Skip to content

Commit b3ed67a

Browse files
AustinBrunkhorstAustinBrunkhorst
authored
Update README.md
1 parent 79dc710 commit b3ed67a

File tree

1 file changed

+53
-298
lines changed

1 file changed

+53
-298
lines changed

‎README.md

+53-298
Original file line numberDiff line numberDiff line change
@@ -1,341 +1,96 @@
11
# C++ Reflection
22

33
## Preface
4-
I set out this summer (*2015*) to implement a flexible reflection system for the game project I'm working on. This repository contains a skeleton for parts of the system that I prototyped throughout the summer. With the proper dependencies and build system setup, you should have enough to integrate into your engine / application without much fuss.
4+
I worked on a complete reflection pipeline starting in the summer of 2015 for a game project / editor. My intent by creating this repository was to share my experience and how I came about developing it. The response from the community motivated me to make it a tad bit more official by allowing others to consume and build it easily, rather than just giving you code and saying "fill in the pieces".
55

6-
## Quick Intro
7-
As a statically typed language, C++ wasn't designed to facilitate runtime type information. Instead, it's crazy fast and optimization friendly. Games are performance critical applications - it is for this reason that C++ is basically the standard backend.
6+
I created a blog where I talk more in detail about the process and try to share my experiences as best as possible. You can find the blog here -
87

9-
Type introspection is crucial for complex / large code bases that need to interface with tools (*i.e. a game editor*). Unless you're a team of all programmers (*I'm sorry if that's the case*) it is effectively impossible to iterate upon a larger game without some set of tools to abstract away code (*especially in 3D*). Without type introspection, you can expect to copy and paste a lot of boilerplate code. This is **absurdly** tedious and undesirable.
8+
http://austinbrunkhorst.com/blog/category/reflection/
109

11-
The good news is that there are *tons and tons* of great resources out there for "extending" C++ to include meta information within your code base. The most common approaches you'll find are as follows:
10+
## Building
11+
There are three buildable sections in this repository - [Runtime](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Source/Runtime), [Parser](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Source/Parser) and [Examples](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Examples). I setup an environment for building using CMake - yes it's insane but it's also awesome so let's just go with it. All examples are assuming you're working from the root of this repository.
1212

13-
+ Using macros and templates to simplify the craziness that is writing the aforementioned boilerplate code.
14-
+ Parsing your code to **generate** the crazy boilerplate code.
13+
### Requirements
14+
- [LLVM 3.8.0](http://llvm.org/releases/download.html)+ (for libClang)
15+
- [Boost 1.59](http://www.boost.org/users/history/version_1_59_0.html)+
16+
- A C++11 compliant compiler - I've tested on MSVC 14, G++ 4.8 and
17+
Clang++ 3.6.
1518

16-
The latter technique isn't adopted nearly as much as the former, but feels like it's becoming much more common. With that said, I chose to use the generation technique. I'm pretty glad I did.
19+
#### Runtime
20+
There are no dependencies in the runtime so building is pretty straightforward.
1721

18-
The purpose of this repository is to be a simple jumpstart reference for those interested in implementing the generation method in their own code base.
1922

20-
Here are a few links that cover more specifics on the concept (*specifically in the realm of C/C++*)
2123

22-
+ [GDC: Physics for Game Programmers - Debugging](http://www.gdcvault.com/play/1020065/Physics-for-Game-Programmers-Debugging) (*it's relevant, I swear!*)
23-
+ [GDC: Robustification Through Introspection and Analysis Tools (Avoiding Developer Taxes)](http://www.gdcvault.com/play/1015586/)
24-
+ [Game Engine Metadata Creation with Clang](http://www.myopictopics.com/?p=368)
25-
+ [Parsing C++ in Python with Clang](http://eli.thegreenplace.net/2011/07/03/parsing-c-in-python-with-clang/)
24+
*Create a build directory.*
2625

27-
## Goals
28-
29-
##### Make the pipeline as hands off as possible.
30-
Specifically, you shouldn't have to jump through a bunch of hoops just to expose your code to the reflection runtime library. Make changes to your code, recompile, and the changes are reflected immediately (*yep, it was intended*).
31-
32-
No extra button clicks or steps to synchronize the reflection data.
33-
34-
##### Provide rich functionality in the runtime library.
35-
If we're going through all this trouble in the first place, might as well make it worth while!
36-
37-
##### Avoid huge build times.
38-
We're effectively compiling our code twice. First to parse our code base and understand it as intricately as a compiler does, then to *actually* compile it as per usual (*with the addition of our generated source*).
39-
40-
This is one of the downsides to the generation approach. Instead of manually writing these macros inline with our source, we're using the brains of a compiler. However, we would much rather sacrifice a little bit shorter build times for the luxury of cleaner, less cluttered code.
41-
42-
Unfortunately, this also implies creating a much less intuitive build pipeline. Don't worry though! I have some nifty diagrams for you.
43-
44-
## Pipeline
45-
46-
In our engine (_we call it **Ursine Engine**, because we're dangerously clever and played on the fact that our team name is Bear King_), we use [CMake](http://www.cmake.org/) for managing most aspects of the overall build pipeline. CMake is a horribly wonderful tool that I've come to love despite hating it at the same time. It allows us to generate solutions for most IDEs that anyone on the team likes to use, although currently, everyone is using Visual Studio 2015 (_finally **some** C++14, baby!_).
47-
48-
CMake makes this pipeline surprisingly simple which was a relief. I won't go into much specific detail, but I'll provide relevant snippets of the integration into our engine a little later when I describe the code in this repository.
49-
50-
Here's a diagram of the entire pipeline from writing the source, to building your game / application.
51-
52-
![Pipeline Diagram](http://imgur.com/V7RLPAj.png "Pipeline Diagram")
53-
54-
## Code
55-
56-
The respository has two parts - [Parser](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Source/Parser) and [Runtime](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Source/Runtime).
57-
58-
+ **Parser** is for the command line source parsing tool. (_requires [Boost 1.59.0](http://www.boost.org/users/history/version_1_59_0.html) and [libclang 3.7.0](http://llvm.org/releases/download.html)_)
59-
+ **Runtime** is for the reflection runtime library.
60-
61-
### CMake Prebuild Example
62-
63-
This is basic example of adding the prebuild step to an existing target in CMake.
64-
65-
```CMake
66-
set(PROJECT_NAME "Example")
67-
68-
# assume this contains header files for this project
69-
set(PROJECT_HEADER_FILES ...)
70-
71-
# assume this contains source files for this project
72-
set(PROJECT_SOURCE_FILES ...)
73-
74-
# generated file names, in the build directory
75-
set(META_GENERATED_HEADER "${CMAKE_CURRENT_BINARY_DIR}/Meta.Generated.h")
76-
set(META_GENERATED_SOURCE "${CMAKE_CURRENT_BINARY_DIR}/Meta.Generated.cpp")
77-
78-
# create the project target
79-
add_executable(${PROJECT_NAME}
80-
${PROJECT_HEADER_FILES}
81-
${PROJECT_SOURCE_FILES}
82-
# make sure the generated header and source are included
83-
${META_GENERATED_HEADER}
84-
${META_GENERATED_SOURCE}
85-
)
86-
87-
# path to the reflection parser executable
88-
set(PARSE_TOOL_EXE "ReflectionParser.exe")
89-
90-
# input source file to pass to the reflection parser compiler
91-
# in this file, include any files that you want exposed to the parser
92-
set(PROJECT_META_HEADER "Reflection.h")
93-
94-
# fetch all include directories for the project target
95-
get_property(DIRECTORIES TARGET ${PROJECT_NAME} PROPERTY INCLUDE_DIRECTORIES)
96-
97-
# flags that will eventually be based to the reflection parser compiler
98-
set(META_FLAGS "")
99-
100-
# build the include directory flags
101-
foreach (DIRECTORY ${DIRECTORIES})
102-
set(META_FLAGS ${meta_flags} "\\-I${DIRECTORY}")
103-
endforeach ()
26+
mkdir Build && cd Build
27+
28+
*Generate a build system using any [desired generator](https://cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html) in CMake.*
10429

105-
# include the system directories
106-
if (MSVC)
107-
# assume ${VS_VERSION} is the version of Visual Studio being used to compile
108-
set(META_FLAGS ${META_FLAGS}
109-
"\\-IC:/Program Files (x86)/Microsoft Visual Studio ${VS_VERSION}/VC/include"
110-
)
111-
else ()
112-
# you can figure it out for other compilers :)
113-
message(FATAL_ERROR "System include directories not implemented for this compiler.")
114-
endif ()
30+
cmake -G "<Desired Generator>" ../Source/Runtime
31+
32+
*Build - you can use any IDE if applicable to the generator, but you can also just build straight from CMake.*
11533

116-
# add the command that invokes the reflection parser executable
117-
# whenever a header file in your project has changed
118-
add_custom_command(
119-
OUTPUT ${META_GENERATED_HEADER} ${META_GENERATED_SOURCE}
120-
DEPENDS ${PROJECT_HEADER_FILES}
121-
COMMAND call "${PARSE_TOOL_EXE}"
122-
--target-name "${PROJECT_NAME}"
123-
--in-source "${CMAKE_CURRENT_SOURCE_DIR}/${PROJECT_SOURCE_DIR}/${PROJECT_META_HEADER}"
124-
--out-header "${META_GENERATED_HEADER}"
125-
--out-source "${META_GENERATED_SOURCE}"
126-
--flags ${META_FLAGS}
127-
)
128-
```
34+
cmake --build . --target MetaRuntime
35+
36+
#### Parser
37+
There are more moving parts in this because the parser actually has dependencies and builds to an executable. Don't worry though! I'm here to walk you through this.
12938

130-
### String Templates
131-
Generating code is usually a pretty ugly process.
39+
##### Install LLVM 3.8 for LibClang
40+
**Windows** - download the [32 bit](http://llvm.org/releases/3.8.0/LLVM-3.8.0-win32.exe) or [64 bit](http://llvm.org/releases/3.8.0/LLVM-3.8.0-win64.exe) pre-built binaries.
13241

133-
Instead of writing the characters manually (i.e. `output += "REGISTER_FUNCTION(" + name + ")"`), I wanted to use *"String Templates"*. That is why I chose [Mustache](https://mustache.github.io/). I found a simple [header only implemenation](https://github.com/kainjow/Mustache), which is included in the [Parser](https://github.com/AustinBrunkhorst/CPP-Reflection/blob/master/Source/Parser/Mustache.h) section.
42+
**Unix based systems** - find the appropriate package. On Linux Mint I just did the following.
13443

135-
In the **Generate Source Files** section of the pipeline diagram, you'll notice two steps. *"Compile"* and *"Render"*. Compile simply takes all of the types that we've extracted and [compiles the data to be referenced in Mustache](https://github.com/AustinBrunkhorst/CPP-Reflection/blob/master/Source/Parser/ReflectionParser.cpp#L217-L220). Render actually renders the templates and writes them to the configured output files.
44+
sudo apt-get install libclang-3.8-dev
13645

137-
In the [Templates](https://github.com/AustinBrunkhorst/CPP-Reflection/tree/master/Templates) folder of the repository, you'll find the mustache template files referenced in the reflection parser.
46+
The installation should be located in `/usr/lib/llvm-3.8`
13847

139-
### Type Meta Data
140-
One of the biggest features that I wanted to implement in the runtime library is being able to add meta data to types at compile time.
48+
Once installed, set an environment variable `LLVM_ROOT` to the root of the installation directory. You can skip this step, but an environment variable makes the CMake command simpler.
14149

142-
If you've ever used C#, you know they have a pretty groovy reflection system built into the language. I really like their syntax for [Attributes](http://www.tutorialspoint.com/csharp/csharp_attributes.htm), which is a way to add meta data to language types / constructs.
50+
##### Install Boost 1.59
14351

144-
The closest I could get to this style, was with the use of Macros. C++11 introduced [Attribute Specifiers](http://en.cppreference.com/w/cpp/language/attributes) as a way to hint compilers on intended behavior or add language extensions. Unfortunately, compiler support varies widely, and as mentioned it's only managed at compile time.
52+
This part sucks, but we've gotta do it. Download the [sources](https://sourceforge.net/projects/boost/files/boost/1.59.0/) and build it using [these instructions](http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html#easy-build-and-install).
14553

146-
Luckily for us, Clang supports the attribute `annotate( )`. You can extract the contents of an annotation with [libclang](https://github.com/AustinBrunkhorst/CPP-Reflection/blob/master/Source/Parser/MetaDataManager.cpp#L10-L17).
54+
Once installed, set an environment variable to `BOOST_ROOT` like we did for LLVM.
14755

148-
The syntax for this attribute look something like this.
56+
*Create a build directory.*
14957

150-
__attribute__(( annotate( "Hey look at this annotation!" ) ))
58+
mkdir Build && cd Build
15159

152-
You might be thinking, *"But it's only for Clang.. how will this work in MSVC?"*
60+
*Generate a build system using any [desired generator](https://cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html) in CMake.*
15361

154-
More good news! libclang preprocesses source files, so we can use preprocessor directives. In the source parsing tool, I define `__REFLECTION_PARSER__` [before compiling](https://github.com/AustinBrunkhorst/CPP-Reflection/blob/master/Source/Parser/Main.cpp#L126). We can use this to make a nice solution for all compilers.
62+
cmake -G "<Desired Generator>" ../Source/Parser
15563

156-
```C++
157-
#if defined(__REFLECTION_PARSER__)
158-
# define Meta(...) __attribute__((annotate(#__VA_ARGS__)))
159-
#else
160-
# define Meta(...)
161-
#endif
162-
```
64+
*If you skipped creating environment variables, you'll have to define variables for resolution in CMake directly - just add these two switches in the command above.*
16365

164-
We would use it like so.
165-
166-
```C++
167-
Meta(Mashed)
168-
int potatoes;
169-
```
66+
-DLLVM_ROOT=<PATH> -DBOOST_ROOT=<PATH>
17067

171-
Now that I could annotate code, I needed to define how I would interact with it. Initially I assumed key value pairs separated by commas, like so.
172-
173-
```C++
174-
Meta(Key = Value, Key2, Key = "Yep!")
175-
```
176-
177-
But after reviewing this approach with my teammate [Jordan](http://www.jordanellis.me), he came up with the brilliant idea of doing exactly what C# does, and that is using user defined types as annotations, queryable at runtime. So I came up with this.
178-
179-
```C++
180-
struct Mashed : public MetaProperty { };
68+
*Build - you can use any IDE if applicable to the generator, but you can also just build straight from CMake.*
18169

182-
Meta(Mashed)
183-
int potatoes;
184-
```
70+
cmake --build . --target MetaParser
18571

186-
Heres how it works - I treat all values delimited by commas as constructors. If a value doesn't have parenthases, it's assumed to be a default constructor.
72+
#### Examples
73+
You will need to follow the same steps for setting up the dependencies explained in the **Parser** build instructions.
18774

188-
For each constructor, I then extract the arguments provided. When I generate the source, I simply paste the extracted arguments as a constructor call of the provided type. The value is converted to a `Variant` and accessible at runtime. This allows us to do some really awesome things.
75+
Just like the other two targets you'll do the following -
18976

190-
```C++
191-
enum class SliderType
192-
{
193-
Horizontal,
194-
Vertical
195-
};
77+
*Create a build directory.*
19678

197-
struct Slider : public MetaProperty
198-
{
199-
SliderType type;
200-
201-
Slider(SliderType type)
202-
: type( type ) { }
203-
};
204-
205-
struct Range : public MetaProperty
206-
{
207-
float min, max;
208-
209-
Range(float min, float max)
210-
: min( min )
211-
, max( max ) { }
212-
};
213-
214-
Meta(Range(0.0f, 1.0f), Slider(SliderType::Horizontal))
215-
float someIntensityField;
216-
```
217-
218-
One of the coolest things about this, aside from type safety, is that Visual Studio correctly syntax highlights the contents of the macro and also provides intellisense! It's a beautiful thing. Here's a more complete example of interfacing with it at runtime using the runtime library.
219-
220-
```C++
221-
struct SoundEffect
222-
{
223-
Meta(Range(0.0f, 100.0f))
224-
float volume;
225-
};
226-
227-
int main(void)
228-
{
229-
// you can also use type meta::Type::Get( "SoundEffect" ) based on a string name
230-
Type soundEffectType = typeof( SoundEffect );
231-
232-
// the volume field in the SoundEffect struct
233-
Field volumeField = soundEffectType.GetField( "volume" );
79+
mkdir Build && cd Build
23480

235-
// meta data for the volume field
236-
MetaManager &volumeMeta = volumeField.GetMeta( );
237-
238-
// getting the "Range" property, then casting the variant as a range
239-
Range volumeRange = volumeMeta.GetProperty( typeof( Range ) ).GetValue<Range>( );
240-
241-
// 0.0f
242-
float min = volumeRange.min;
243-
244-
// 100.0f
245-
float max = volumeRange.max;
246-
247-
return 0;
248-
}
249-
```
250-
251-
### Function Binding
252-
253-
Another notoriously difficult or convoluted process in managing reflection info in C++ is dynamically invoking functions / methods.
254-
255-
The most common approach is to store raw function / method pointers and calculate the offsets of arguments. The result is a ton of templates and difficult to follow operations. Not to mention, I can't imagine it's fun to debug!
81+
*Generate a build system using any [desired generator](https://cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html) in CMake.*
25682

257-
In libclang, you're able to easily extract signatures from functions. With this in mind, I came up with the most simple approach that I could think of. **Wrapping function calls in generated lambdas**.
258-
259-
Here's a simple demonstration of the concept.
260-
261-
```C++
262-
int foo(int a, float b, double c)
263-
{
264-
return 0;
265-
}
266-
267-
auto fooWrapper = [](int a, float b, double c)
268-
{
269-
return foo( a, b, c );
270-
};
271-
272-
// same behavior as foo( 0, 1.0f, 2.0 );
273-
fooWrapper( 0, 1.0f, 2.0 );
274-
```
275-
276-
In the context of our runtime library, here's an example of something that might be generated for a class method.
277-
278-
```C++
279-
class DemoClass
280-
{
281-
public:
282-
int someMethod(int a)
283-
{
284-
return a;
285-
}
286-
};
287-
288-
auto someMethodWrapper = [](Variant &obj, ArgumentList &args)
289-
{
290-
auto &instance = obj.GetValue<DemoClass>( );
83+
cmake -G "<Desired Generator>" ../Examples
29184

292-
return Variant {
293-
instance.someMethod(
294-
args[ 0 ].GetValue<int>( )
295-
)
296-
};
297-
};
298-
```
299-
300-
That's it! It's much less compilicated than the previously mentioned approach. This concept is also applied to fields / globals with their getters and setters.
301-
302-
There are some downsides though:
303-
+ Larger code size. For each generated lambda, the compiler has to generate a bunch of symbols behind the scenes.
304-
+ Larger compile times.
305-
+ Decent amount of indirection just for one function call.
306-
307-
You shouldn't have to worry **too** much however. If like most people, you use reflection for editor functionality, not a physics simulation. Performance in most cases is not critical.
308-
309-
Here's a more complete example of dynamically calling functions with the runtime library.
310-
311-
```C++
312-
struct SoundEffect
313-
{
314-
float volume;
315-
316-
void Load(const std::string &filename);
317-
};
318-
319-
int main(void)
320-
{
321-
Type soundEffectType = typeof( SoundEffect );
322-
Field volumeField = soundEffectType.GetField( "volume" );
85+
*Build - you can use any IDE if applicable to the generator, but you can also just build straight from CMake.*
32386

324-
// the runtime supports overloading, but by default returns the first overload
325-
Method loadMethod = soundEffectType.GetMethod( "Load" );
87+
cmake --build .
32688

327-
// creates an instance of a sound effect
328-
Variant effect = soundEffectType.Create( );
89+
Remember to add the extra switches for defining LLVM_ROOT and BOOST_ROOT if you skipped adding them as an environment variable.
32990

330-
// effect.volume is now 85
331-
volumeField.SetValue( effect, 85.0f );
91+
All of the examples build to a simple executable that demonstrates the specific features/functionality in use.
33292

333-
// 85
334-
volumeField.GetValue( effect );
93+
I don't have immediate intentions of documenting the interfaces and such with the runtime library, but hopefully the examples cover all parts of the runtime and people can get the whole picture there. My blog posts cover the development process and the reason I made the decisions I did.
33594

336-
// effect.Load is called
337-
loadMethod.Invoke( effect, std::string { "Explosion.wav" } );
95+
Feel free to contact me with any questions or concerns! Also, pull requests are very welcome!
33896

339-
return 0;
340-
}
341-
```

0 commit comments

Comments
 (0)