C++ Reflection: Type MetaData: Introduction

Thinking about how to take things a step further in terms of what your tools can do for you is really important in increasing productivity. This is where a form of reflection can come in very handy. I call the reflection system I'm working on by shorthand of "meta" or "MetaData", though the proper terminology for it would be something more like Class MetaData or Type Reflection. So when I say MetaData I'm referring to data about data, and specifically data about the types of data within your C++ code.

Having a MetaData system allows for information about your data types to be saved during run-time for use during run-time. The C++ compiler compiles away a lot of information, and a lot of this information is very useful. So, a MetaData system saves this information from being compiled away.

So what is the use of such a system? Where here are the things I've constructed so far: simple and powerful debugging features; automated serialization for any type registered with Meta; automic function binding to Lua; Variant and RefVariant class types (objects that can hold any type of data registered within Meta). However these aren't the only things I'll be using the MetaData system for. One could also apply the MetaData to make simple object factories with hardly any code, or property viewing tables for editors with ease. I'm sure there's even more uses as well that I just haven't quite come to terms with. In a way, systems like this can generate tools and functionality for the developer whenever a new class or data type is introduced.

Before we start I want to let the reader know that such a system can be very efficient, efficient enough to run wonderfully within a real-time application like a game. I myself don't know a lot about efficiency or optimization at a low-level, though I do know other programmers who've made such Reflection systems that are very, very fast. So make sure not to have any pre-built misconceptions about a C++ Reflection system before moving on.

I started learning about how to construct a MetaData system from this post. You can take a look if you like, though it won't be necessary if you just want to learn what I'm trying to teach here. In that post, by a fellow student here at DigiPen, he goes over how to get started with MetaData, but leaves a lot to yet be learned. I'll be modelling my post after his quite a bit, as he does have good content structure for an introduction post.

I'll attempt to document what I've learned here as honestly, there's not any resources on constructing a MetaData system like this anywhere as far as I know. The closest thing I could find was from a Game Programming Gems book, the 5th one chapter 1.4, although it required all classes that wanted to participate to inherit from the MetaData class. This isn't really sufficient if you want to reflect class or structure types you don't have source code access to, and it doesn't support the built-in types at all.

Getting Started
To start lets talk about the overall structure of what a MetaData system looks like. I think it's going to best to draw a diagram of what will be achieved from this article:

Diagram of entire MetaData system layout.

In the above diagram the MetaData objects are the key object in which all important operations are performed. A MetaData object is a non-templated class, allowing them to be stored in a data structure. There is one MetaData object for each type of data registered to the system, and the MetaData object of a corresponding type is a representation of that type. It stores information about whether or not it is a Plain Old Data type (POD), the size of the type, it's members and methods, and name. Inheritance information can be stored along with multiple inheritance information, though I haven't even bothered adding that feature in yet as it's not actually very useful and quite difficult to do properly.

The MetaCreator is a templated class that manages the creation of a single unique MetaData instance. It then adds its instance into the MetaManager which contains it in some sort of map.

The MetaManager is a non-templated class that contains all of the created MetaData instances, and can perform search operations on them to find specific types. I use a map of strings to instances, so I can search by string identifier. I've also placed some other small utility functions into my MetaManager as well.

Client Code
Before we get started writing anything, I'd like to try to show some example client code to exemplify why I've taken all this time to make a MetaData system in the first place.

GameObject *obj = FACTORY->CreateObject( "SampleObject" );

// FACTORY->CreateObject code
GameObject *ObjectFactory::CreateObject( const std::string& fileName )
{
  SERIALIZER->OpenFile( fileName );
  GameObject *obj = DESERIALIZE( GameObject );
  SERIALIZER->CloseFile( );
  IDObject( obj );
  LUA_SYSTEM->PushGameObject( obj );
  return obj;
}

// LUA_SYSTEM->PushGameObject code
void LuaSystem::PushGameObject( GameObject *obj )
{
  lua_getglobal( L , "GameObjects" );
    
  lua_pushinteger( L, obj->GetID( ) );
  LuaReference *luaObj = (LuaReference *)lua_newuserdata( L, sizeof( LuaReference ) );
  luaL_setmetatable( L, META_TYPE( GameObject )->MetaTableName( ) );
  luaObj->ID = obj->GetID( );
  luaObj->meta = META_TYPE( GameObject ); // grab MetaData instance of this type
    
  lua_settable( L, 1 ); // Stack index 1[3] = 2

  // Clear the stack
  lua_settop( L, 0 );
}

As you can see I have a nice DESERIALIZE macro that can deserialize any type of data registered within within the Reflection. My entire serialization file (includes in and out) is only about 400 lines of code, and I implemented my own custom file format. I also have a LuaReference data type, which contains a handle and a MetaData instance, and allows any class to be sent to Lua via handle. Because of my Meta system I can write very generic and powerful code pretty easily.

Getting Started
The first I recommend starting with is to reflect the size of a type, and the name of a type. Here's what a very simple  MetaData class could like:

//
// MetaData
// Purpose: Object for holding various info about any C++ type for the MetaData reflection system.
//
class MetaData
  {
  public:
    MetaData( std::string string, unsigned val ) : name( string ), size( val ) {}
    ~MetaData( )

    const std::string& Name( void ) const { return name; }
    unsigned Size( void ) const { return size; }

  private:
    std::string name;
    unsigned size;
}

This simple class just stores the size and name of a type of data. The next thing required is the ability to create a unique instance of MetaData. This requires a templated class called the MetaCreator.

template <typename Metatype>
class MetaCreator
{
public:
  MetaCreator( std::string name, unsigned size )
  {
    Init( name, size );
  }

  static void Init( std::string name, unsigned size )
  {
    Get( )->Init( name, size );
  }

  // Ensure a single instance can exist for this class type
  static MetaData *Get( void )
  {
    static MetaData instance;
    return &instance;
  }
};

You should be able to see that by passing in a type, perhaps <int> to the MetaCreator, a single MetaData instance becomes available from the Get function, and is associated with the MetaCreator<int> class. Any type can be passed into the Metatype typename. The MetaCreator constructor initializes the MetaData instance. This is important. In the future you'll have MetaData for a class that contains MetaData for POD types. However because of out-of-order initialization, some of the types required to construct your class MetaData instance might not be initialized (as in the Init function will not have been called) yet. However if you use the MetaManager<>::Get( ) function, you can retrieve a pointer to the memory location that will be initialized once the MetaCreator of that specific type is constructed. It should be noted that the construction of a MetaCreator happens within a macro, so that there's absolutely no way of screwing up the type registration (the inside of the macro will become quite... ugly).

Lastly you'll need a place to store all of your MetaData instances: the MetaManager!

//
// MetaManager
// Purpose: Just a collection of some functions for management of all the
//          various MetaData objects.
//
class MetaManager
{
public:
  typedef std::map<std::string, const MetaData *> MetaMap;

  // Insert a MetaData into the map of objects
  static void RegisterMeta( const MetaData *instance );

  // Retrieve a MetaData instance by string name from the map of MetaData objects
  static const MetaData *Get( std::string name ); // NULL if not found

  // Safe and easy singleton for map of MetaData objects
  static MetaMap& GetMap( void )
  {
    // Define static map here, so no need for explicit definition
    static MetaMap map;
    return map;
  }
};

And there we have a nice way to store all of our MetaData instances. The Get function is most useful in retrieving a MetaData instance of a type by string name. Now that we have our three major facilities setup, we can talk about the macros involved in actually registering a type within the MetaData system.

  //
  // META_TYPE
  // Purpose: Retrieves the proper MetaData instance of an object by type.
  //
#define META_TYPE( TYPE ) (MetaCreator<TYPE>::Get( ))

  //
  // META
  // Purpose: Retrieves the proper MetaData instance of an object by an object's type.
  //
#define META( OBJECT ) (MetaCreator<decltype( OBJECT )>::Get( ))

  //
  // META_STR
  // Purpose : Finds a MetaData instance by string name
  //
#define META_STR( STRING ) (MetaManager::Get( STRING ))

  //
  // DEFINE_META
  // Purpose : Defines a MetaCreator for a specific type of data
  //
#define DEFINE_META( TYPE ) \
  MetaCreator<TYPE> NAME_GENERATOR( )( #TYPE, sizeof( TYPE ) )

So far so good. Using the DEFINE_META macro it's pretty easy to add a type to the MetaData system, simply do DEFINE_META( type );. The decltype might be confusing, as it's new. decltype simply returns the type of an object. This allows the user to retrieve a MetaData instance that corresponds to an object's type, without knowing what the object's type is; this lets very generic code be easily written.

NAME_GENERATOR is a bit tricky. Every single instance of a MetaCreator needs to be constructed at global scope- this is the only way to get the Init( ) function to be called, without having to place your DEFINE_META macro in some sort of code scope. Without the constructor of the MetaCreator calling Init, the only way to have any sort of code run by using the DEFINE_META macro is to place it within some scope that is run sometime after main executes. This makes the use of the DEFINE_META macro harder. If you create the MetaCreator at global scope, then you can have the constructor's code run before main executes at all, making the DEFINE_META macro very easy and simple to use.

So then comes the issue of "what do I call my MetaCreator?" arises. The first thing you might think of is, just call it MetaCreator and make it static. This hides the MetaCreator at file scope, allowing the DEFINE_META macro to be used once per file without any naming conflicts. However, what if you need more than one DEFINE_META in a file? The next solution I thought of was to use token pasting: ## operator. Here's an example usage of the token pasting technique:

DEFINE_META( TYPE ) \
  MetaCreator<TYPE> Creator##TYPE( )( #TYPE, sizeof( TYPE ), false )

The only problem with this strategy is that you cannot pass in type names with special characters or spaces, as that won't result in a proper token name. The last solution is to use some sort of unique name generation technique. There are two macros __LINE__ and __FILE__ that can be used to generate a unique name each time the macro is used, so long as it is not used twice on the same line of the same file. The __LINE__ and __FILE__ definitions are a bit tedious to use, but I think I have them working properly, like this:

#define PASTE( _, __ )  _##__
#define NAME_GENERATOR_INTERNAL( _ ) PASTE( GENERATED_NAME, _ )
#define NAME_GENERATOR( ) NAME_GENERATOR_INTERNAL( __COUNTER__ )

You have to feed in the __COUNTER__ definition carefully in order to make sure they are translated by the preprocessor into their respective values. A resulting token could look like: GENERATED_NAME__29. This is perfect for creating all of the MetaCreators at global scope on the stack. Without the use of some sort of trick like this, it can be very annoying to have to use a function call to register your MetaData.

Alternatively there are __FILE__ and __LINE__ macros, but they aren't really necessary as the __COUNTER__ does everything we need. The __COUNTER__ however is, I believe, not actually standard.

So far everything is very straightforward, as far as I can tell. Please ask any questions you have in the comments!

It should be noted that const, &, and * types all create different MetaData instances. As such, there is a trick that can be used to strip these off of an object when using the META macro. This will be covered in a subsequent article.

Example Uses
Now lets check out some example client code of what our code can actually do!

DEFINE_META( int );
DEFINE_META( float );
DEFINE_META( double );
DEFINE_META( std::string );

void main( void )
{
  std::cout << META_TYPE( int )->Name( ); // output: "int"
  std::cout << META_TYPE( float )->Size( ); // output: "4"

  std::string word = "This is a word";
  std::cout << META( word )->Name( ); // output: "std::string"
  std::cout << META_STR( "double" )->Size( ); // output: "8"
  if(META( word ) != META_TYPE( int ) // address comparison
  {
    std::cout << "This object is not an int!";
  }
}

And there we have it! An extremely easy to use DEFINE_META macro that stores name and size information of any type, including class and struct types.

Future Posts
For future posts I look forward to writing about automated Lua binding, Variants and RefVariants, automated Serialization, factory usage, messaging, and perhaps other topics as well! These topics are really rather huge, so please by patient in that it may take a while to cover everything.

Link to second article in series.