Introducing the Rerun SDK for C++
The ability to log streams of multimodal data from C++ and visualize it live with Rerun has been our most requested feature since before the public launch in February. The C++ SDK is finally out, but getting here the right way has been a long road.
If you're eager to get started, the quick start guide is right here.
We designed the C++ API's to be easy to use, efficient, and consistent with our API's in Rust and Python. Another key goal was to make it easy to write adapters to log data in your own custom formats to Rerun. With the first iteration of our C++ API released, we're really looking forward to hear from the community on how well that works, and what we can do to improve it further.
Here is an example of creating and logging a random tensor in Python, Rust, and C++:
"""Create and log a tensor in Python."""
import numpy as np
import rerun as rr
tensor = np.random.randint(0, 256, (8, 6, 3, 5), dtype=np.uint8) # 4-dimensional tensor
rr.init("rerun_example_tensor", spawn=True)
# Log the tensor, assigning names to each dimension
rr.log("tensor", rr.Tensor(tensor, dim_names=("width", "height", "channel", "batch")))
Which should look like this for all languages (ignoring different random numbers):
You can integrate Rerun into your CMake-based project by adding the following snippet to your CmakeLists.txt file:
include(FetchContent)
FetchContent_Declare(rerun_sdk URL
https://github.com/rerun-io/rerun/releases/latest/download/rerun_cpp_sdk.zip)
FetchContent_MakeAvailable(rerun_sdk)For more information on working with the C++ APIs, see the C++ quick start or the logging tutorial.
While this release is all about C++, Rerun 0.10 also includes an in-app getting started guides using Rerun's markdown support.
As a reminder, Rerun is under active and high paced development. While the C++ SDK is feature complete, i.e. everything you can log in Python and Rust can be logged in C++, several improvements to the C++ SDK are already planned for 0.11.
The road to C++ the-road-to-c
Rerun is currently used by researchers and engineers in fields like computer vision, robotics, and AR/XR. The norm in these fields is that running in production (on the edge) means C++. It's therefore no surprise that C++ bindings have been our most requested feature. We've still held off releasing them until now.
Held back by complexity held-back-by-complexity
Within the first few months of building Rerun in earnest, it became clear that manually maintaining nice, well documented, and consistent APIs between just Python and Rust was very hard. Too many forces naturally pulled them apart. If we didn't make some major changes first, adding C++, which is much harder to work with than Rust, would slow us to a crawl.
Most of the complexity arises from that the Rerun SDKs lets users log many kinds of data, which could be represented in many different formats in their code. Rerun uses Apache Arrow for in-memory data and Arrow IPC for in-flight data. The SDKs need to make it super easy to get user data for all those types into Rerun's Arrow based format, while both feeling consistent across languages and feeling native to each language separately.
The solution was a new code generation framework the-solution-was-a-new-code-generation-framework
Five months ago, we made the decision that the only sustainable way forward was code generation. We built a system that took Rerun types defined in a new Interface Definition Language (IDL), and used them to generate working SDK code in Rust and Python. The framework is entirely written in Rust, uses the Rust build system as a build graph for the whole generation process, and probably deserves a post or two on its own.
In total it took about four months get the code generation framework in place, move all SDK and engine code onto it, and move over to the more type oriented API we released in Rerun 0.9. To make sure it would work for new languages, we tested it on parts of the C++ SDK in parallel.
Since 0.9 we've used the new framework to quickly bring C++ to feature parity, and make sure that the new SDK produces 100% compatible Arrow payloads with the two existing APIs.
Cross language equivalence testing cross-language-equivalence-testing
An added benefit of the architecture where IDL specified type definitions produce SDK code, that in turn produce Arrow IPC streams, is that it makes it easy to compare outputs across languages. We built a tool that compares multiple .rrd files (Rerun's file format holding the Arrow IPC streams). That tool is used as part of a small framework that compares the output of code snippets in Python, C++, and Rust, and makes sure they all produce the exact same in-memory Arrow representation.
We use this cross-language equivalence testing framework to test all the SDKs, including the code examples used for documentation. Having these tests set up made all the difference in being able to deliver the C++ SDK so quickly after the big 0.9 release.
Making third party types loggable with ComponentBatchAdaptor making-third-party-types-loggable-with-componentbatchadaptor
When using your own types with Rerun in C++, you have the option to define completely custom archetypes and components just like in Python and C++. However, we also want to make is as easy and performant as possible to use your own data types together with Rerun's built-in types.
For example, say you represent your point clouds as std::vector<Eigen:Vector3f>. You would then like to be able to log a point cloud to Rerun like:
std::vector<Eigen:Vector3f> points3d_vector = ...
rec.log("points", rerun::Points3D(points3d_vector));We can do this with ComponentBatchAdaptor. Here is how it could look for the example above:
// Adapters so we can log Eigen vectors as Rerun positions:
template <>
struct rerun::ComponentBatchAdapter<rerun::Position3D, std::vector<Eigen::Vector3f>> {
ComponentBatch<rerun::Position3D> operator()(const std::vector<Eigen::Vector3f>& container) {
return ComponentBatch<rerun::Position3D>::borrow(container.data(), container.size());
}
ComponentBatch<rerun::Position3D> operator()(std::vector<Eigen::Vector3f>&& container) {
throw std::runtime_error("Not implemented for temporaries");
}
};If you have your own types laid out similarly enough to the matching Rerun type, logging can usually be done without incurring any extra copies. Check out this basic example to see how to use Rerun with Eigen and OpenCV, and learn more about using ComponentBatchAdapter here.
Key current limitations key-current-limitations
ComponentBatchAdaptor is not as expressive and performant as is should be componentbatchadaptor-is-not-as-expressive-and-performant-as-is-should-be
You can't currently write adaptors for single component batches like images or tensors. This means logging these types are both unergonomic and produce an unnecessary copy. The adapters aren't currently able to model data with strides, which means logging strided data also requires an extra copy. We will work on these shortcomings over the next few releases.
No built-in adapters for common libraries like Eigen and OpenCV no-builtin-adapters-for-common-libraries-like-eigen-and-opencv
We plan to add built-in adapters for common types in Eigen and OpenCV over the next releases. If there are other libraries you think should have built-in adapters, we're happy for both suggestions and pull requests.
No hosted API documentation no-hosted-api-documentation
While the general documentation treats C++ as a first class citizen, we don't yet have hosted API documentation like we do for Python and Rust. We should have this resolved within the next few releases but in the meantime you can find the API documentation directly in the header files. There are also C++ code examples for all our types at https://www.rerun.io/docs/reference/types/archetypes.
Rerun only supports C++17 and later rerun-only-supports-c17-and-later
If you need C++14 or C++11 support, please let us know.
Try it out and let us know what you think try-it-out-and-let-us-know-what-you-think
We're really looking forward to hear how this first iteration of our C++ SDK works for you. Start with the C++ quick start guide and then join us on Github or Discord and let us know what you like and what you'd hope to see change in the future.
Also, huge thanks to everyone who tried out early builds and came with feedback, suggestions, and bug reports!