Schema loading from the wire

Cap'n Proto provides a schema loader, which can be used to dynamically
load schemas during runtime. To port this functionality to pycapnp,
a new class is provided `C_SchemaLoader`, which exposes the Cap'n
Proto C++ interface, and `SchemaLoader`, which is part of the pycapnp
library.

The specific use case for this is when a capnp message contains
a Node.Reader: The schema for a yet unseen message can be loaded
dynamically, allowing the future message to be properly processed.

If the message is a struct containing other structs, all the schemas for
every struct must be loaded to correctly parse the message. See
https://github.com/DaneSlattery/capnp_generic_poc for a
proof-of-concept.

Add docs and cleanup

Add more docs

Reduce changes

Fix flake8 formatting

Fix get datatype
This commit is contained in:
Rowan Reeve
2023-02-21 08:31:12 +02:00
committed by DaneSlattery
parent b439993b1f
commit a5c29a74d2
9 changed files with 106 additions and 4 deletions

View File

@@ -0,0 +1,12 @@
#pragma once
#include "capnp/dynamic.h"
#include "capnp/schema.capnp.h"
/// @brief Convert the dynamic struct to a Node::Reader
::capnp::schema::Node::Reader toReader(capnp::DynamicStruct::Reader reader)
{
// requires an intermediate step to AnyStruct before going directly to Node::Reader,
// since there exists no direct conversion from DynamicStruct::Reader to Node::Reader.
return reader.as<capnp::AnyStruct>().as<capnp::schema::Node>();
}

View File

@@ -1,7 +1,7 @@
from capnp.includes.capnp_cpp cimport (
Maybe, PyPromise, VoidPromise, RemotePromise,
DynamicCapability, InterfaceSchema, EnumSchema, StructSchema, DynamicValue, Capability,
RpcSystem, MessageBuilder, Own, PyRefCounter
DynamicCapability, InterfaceSchema, EnumSchema, StructSchema, DynamicValue, Capability,
RpcSystem, MessageBuilder, Own, PyRefCounter, Node, DynamicStruct
)
from capnp.includes.schema_cpp cimport ByteArray
@@ -30,3 +30,7 @@ cdef extern from "capnp/helpers/rpcHelper.h":
cdef extern from "capnp/helpers/serialize.h":
ByteArray messageToPackedBytes(MessageBuilder &, size_t wordCount)
cdef extern from "capnp/helpers/deserialize.h":
Node.Reader toReader(DynamicStruct.Reader reader) except +reraise_kj_exception

View File

@@ -11,4 +11,4 @@ kj::Array< ::capnp::byte> messageToPackedBytes(capnp::MessageBuilder & message,
kj::ArrayOutputStream out(result.asPtr());
capnp::writePackedMessage(out, message);
return heapArray(out.getArray()); // TODO: make this non-copying somehow
}
}