Simple runtime reflection in C++

While I was working on my game engine, I started learning about scripting and scripting systems, and I thought it would be nice to add some scripting capabilities to it. Being able to create objects or to call engine functions from scripts, to separate game logic from the engine innards, to make modification to the game without the need to hardcode stuff and recompile everything seemed like nice features to have. Besides, all game engines worthy of the name allow the use of a scripting language. I picked Lua as a scripting language/environment, since it’s fairly popular in game development, it’s lightweight and I had already used it in the past. Soon into the process of binding my engine code to Lua I realized that needed to get informations about the properties of objects on the engine side of my code (which data members they have, which member functions, constructors and so on and so forth), and, msot importantly, that I needed that informations at runtime. Say, when the game loads a level it runs a script that has a list of entities (and their components) to be spawned: I soon ran into the problem of creating an object by knowing only it’s name from a string in the script. After a little (I have to admit, rather unsuccesful) thinkering, I decided to code a little reflection library myself.

Reflection and introspection, a program reasoning about itself

A more appropriate title for this article should be “Simple runtime introspection in C++”. The terms reflection and introspection are used somewhat interchangeably, even if their meanings are not quite the same. Let’s clear things up first by providing some definitions: in programming jargon, introspection is the ability of a program to find out information about its structure at runtime; reflection takes this a step further by enabling the structure of a program to be modified at runtime. In order to do this, the program must have a representation of itself (its structure and its objects) available during its execution. This kind of information is called metadata. The C++ language supports the OOP paradigm, so it’s a natural choice for metadata to be represented as objects, called metaobjects. The word meta derives from the Greek μετα-, meaning “after” or “beyond”. It is data that provides information about other data, in other words, it is data about data.

In this article I’ll use the term reflection to indicate both reflection and introspection, safe in the knowledge that we’ve agreed upon what is what.

Reflecting a Type (saving type information)

When we write a C++ program, type information is embedded inside the source code, and the compiler uses those informations to perform type checking, but when the compiler is done with our source files all those informations about types are gone, erased from the executable, never to be seen again. As of now, C++ doesn’t provide language support for reflection as other languages do (for example Java, C# or dynamic languages like Python). As we said, if we want to retrieve that information during program execution we need to store it into some metaobjects containing the necessary metadata, before that information is removed from the program source.

When we store information about a type inside another object we say that the type has been reflected: a type becomes a runtime object, and the properties of that type become members of the corresponding object .It is said that the type and its properties, which are abstract language concepts are reified, and thus can be manipulated during the program execution. Once the type has been exposed to the reflection system we can attach other metadata about that type to its reflected counterpart, in the form of metaobjects. Each type has its own runtime representation in the form of a unique TypeDescriptor object. The TypeDescriptor acts as a container of metadata/metaobjects:

A TypeDescriptor has functions to add metadata as well as member functions to retrieve those metadata. When a type is reflected, all its constructors, data members, member functions, base classes and type conversion operators can be attached to it. Free functions can be attached as well, which will come in handy when the reflection system is used to serialize objects, or to create glue code between the application and a scripting system.

It all starts by reflecting a type, giving a name to the reflected type:

The Reflect function calls TypeFactory::ReflectType() and returns an object of type TypeFactory<Type>, which is a class template parameterized by the type to be reflected. The returned typeFactory object is a variable template, so exactly one Typefactory exists for each reflected type. Each TypeFactory is, unsurprisingly, a static factory and contains static member functions that are used to reflect the type as well as to attach metadata to the reflected type:

Each static member function returns the TypeFactory object so many calls can be concatenated when we reflect a type and add type informations to it, as in the named parameter idiom. A TypeDescriptor associated with a type can be retrieved by calling the Resolve function (each type or function in the reflection system lives in the namespace Reflect). This function is overloaded to accept the name of the reflected type as a string argument, an instance of the reflected type, or the name of the type as a template type argument:

Any as in “any kind of object”

Any is a type whose objects can contain any kind of objects, storing a copy of the object by default. The any object uses small buffer optimization (SBO) to limit allocations, and it is responsible for the management of any associated resource. A non-managing version of any is the Handle class, which doesn’t permorm copies and is just a reference wrapper around the object. An Any can be constructed from an Handle, so the resulting any acts like a reference to the object (an Handle can be constructed from an Any as well). Any stores a type-erased object in a void*, but has a TypeDescriptor as well, and manages conversions and casts from the type of the stored object to any other type, using the type information inside the metadata of the reflection system (full implementation on GitHub):

Any objects are used to pass arguments to Constructor and MembeFunction metaobjects, and are returned from those same metaobjects when Invoked.

Constructors

The constructor metaobject abstracts the real constructor of a reflected type and it’s invoked to create a new instance of that type. The Constructor base class exposes an interface to create a new instance of a type and to query the type of the constructed object as well as the type of the constructor arguments. The ConstructorImpl class template derives from Contructor and is parameterized by the type of the object to construct and the constructor arguments, and implements the NewInstance method, which returns an Any containing the new object:

There are two contructor overloads, one taking an array of Any as parameter, while the other is a templated constructor that takes a variadic list of arguments. The NewInstanceImpl member function checks the validity of the arguments: if the arguments are the same as the parameters of the constructor, or if they can be converted to those types, the function returns an Any containing the new instance, otherwise it returns an empty/invalid Any.

Data members, getters and setters

The DataMember metaobject represent a member variable, or field, of the reflected type. It contains the name of the reflected field and the type descriptors of the type of the field and of the class it belongs to, as well as member functions to retrieve them. Setting and getting the value of an object’s field is a matter of invoking the Set and Get methods of the DataMember metaobject, passing an Handle to an Any object that contains the object whose field we are accessing. The accessor methods Get and Set are pure virtual functions:

Two derived classes implement the accessor methods, depending on how the data member is stored: one class stores the raw pointer to data member and every access uses the provided Any object and the raw pointer to member. The class is a template, parameterized by the type of the data member (Type), and the type of the class it belongs to (Class), so the stored pointer type is Type Class::*. The Set member function uses tag dispatch to detect at compile time if the code is trying to set a const data member, and if so it raises a static assertion:

To be able to reflect a data member this way we need access to a public field, but good encapsulation practice madates that data members be declared as private in a class definition. Getter and setter methods are generally used to access the private fields in a properly abstracted data type. A DataMember metaobject can also be created when the type has private member variables that can be accessed through accessor functions. This requires a little bit of C++17 templates, since the getter and setter functions are passed as template arguments to auto deducted template non-type parameters:

In this way we can reflect a type’s fields, wheter they’re public data members or encapsulated, private fields. The DataMember metaobject’s Set and Get methods  can be used to access the member, to change or to get its current value.

Member Functions

We can attach member functions or free functions to a reflected type using the MemberFunction metaobject. Member functions can then be invoked on an instance of the object (in case of member functions), or with an empty any object (in case of free functions). Being able to attach free functions to the meta type is very convenient when we use the reflection system to serialize data or to expose types to a scripting system, as we’ll se later on.

As with the DataMember class, the base MemberFunction class is an ordinary (non-template) class, that exposes an interface to query the properties of the reflected function (number and type of parameters, return type), and to invoke the function. A bunch of class templates derive from it, one for each type of function that can be reflected: member functions, const member functions and free functions. For each derived class, a partial template specialization is provided for void return type. I’ll show the implementation for a non const member function, the other implementations can be found along with the complete code on the GitHub repo:

Base classes and conversions

The C++ language defines a bunch of implicit type conversions, and allows user-defined conversions as well (converting constructors, type conversion operators). Derived-to-base conversions are allowed as well, and a pointer or reference to a derived class can be assigned to a pointer or reference to a base class. The Base and Conversion metaobjects allow these conversions between metatypes.

The Base metaobject has a Cast method that performs a static cast to the base class and returns a void*:

Analogously he Conversion metaobject has a Convert method that returns an Any containing the converted object:

Reflection at work

Let’s see a concrete example of how to use a simple reflection system like this. Let’s define a type called Player and another type Vector3D:

Now we register these types with the reflection system (we reflect them), and attach to them all the metaobject that we need. We also reflect the type const char* as “cstring” and attach a conversion metaobject to std::string, and the type double as “double”, with a conversion to int. These reflected types are needed to pass a pointer to const char or a double when a std::string or a int are needed:

Now we’re able to construct an instance of Player dynamically at runtime with a string containing the name of the reflected type (“Player”). We can also set and get reflected object fields, and call the object’s member functions with a set of suitable arguments and those functions can return values:

The output is:

And that’s it. A simple reflection system that can be used for many useful things, such as serializing/deserializing data, or binding application code to a scripting environment.

Leave a Reply

Your email address will not be published. Required fields are marked *