I didn’t want the developer to declare his routes by writing C++ code
...in a web framework in C++, where presumably the developer would need to write C++ anyway to make use of it?
This article is quite long and complex
No kidding. And to me, all that complexity just screams "you're doing it wrong."
I read this article all the way to the end, and was disappointed to find that, after all that code, it ends with a single macro call --- in C++ --- and leaves the reader wondering about the original goal of using JSON.
Years ago I spent a brief amount of time maintaining Java code that was written in this reflection-heavy, everything-configurable (XML and even some ad-hoc text formats, but similar idea) style, and it was not at all enjoyable to get everything set up correctly nor debug the monstrosity. On a somewhat related note, http://discuss.joelonsoftware.com/default.asp?joel.3.219431....
I’d prefer to have a simple JSON configuration file... followed by a pile of unnecessary complexity in the form of new semantics on how to declare and tie things together.
I highly doubt that this is making author's developers life easier.
More likely, it's just an over-engineering indulgence, that happens to make one's job more secure.
I prefer the C++ configuration over the reflective one. It plays nicely with IDEs and tooling; I can use a debugger on it; I can run code while generating it; it gets verified by type-checking; etc. The only thing it doesn't have is the familiarity bonus of JSON.
So, in the end, you have a system which allows a JSON data structure to call arbitrary C++ functions. Is that a good thing? That JSON is now highly trusted. And do you really want to program in JSON?
This is essentially a rather complicated way to build an interpreter for another language. To display web pages. Sort of like PHP.
This might be worth the trouble if you were using it to allow programmable shaders in a renderer. Those are usually composed of functional blocks strung together, they have to go very fast, and they're composed by people who are artists, not programmers, often through some GUI. There, it might not be overkill.
A lot of people commented about the JSON configuration file and I'd like to clarify this part of the article.
The routing configuration through a JSON configuration file was an idea I had at the beginning and it lead me to reflexion. I found the subject quite challenging and interesting, and I decided to work on it, resulting in a reflexion library.
However, when I came back to the original library, the mvc framework, I did not decide to go further concerning this routing configuration for multiple reasons:
1. the additional cost for each route call, resulting in decreased performance.
2. even if configuration would have been easier with JSON (or whatever) configuration file, the developer would have to register his controllers and actions for reflexion: lots of complexity for no real gain, as you have mentioned. And of course, if we want to avoid registering step, we would fall back in the dlopen solution as explained in the article, not really more appreciable.
I will maybe do an update to clarify this because lots of people seem to give a real importance to this part while it was only a way for me to introduce the subject. The goal of this article was mainly to describe briefly possible implementations of reflexion in C++, especially the one I have worked on.
I appreciate effort given by author, but sorry it seems like "how to guide":
How to kill your time by making simple things complex, then how to kill more time by maintaining it, then how to kill more and more time for adding one small feature.
So here's my advice on the writing an article like this: Show the result first, then explain how you arrived there!
Otherwise some readers will end up frustrated with the solution after absorbing tons of detail, and others will have given up even though they might have liked the result.
Thank you for your advice! This is my real first article (I've never written articles as long as this one before, and I've never written articles in English earlier), so these kind of feedbacks are really interesting :)
Yikes.. I'm going to have to reread this later to grok it better. I think it's nice that the language has added all these new features (though the macros at the end tell me it's not enough..) - but I keep feeling like the direction of the language post-C++11 has been steered by library developers with these more and more difficult to digest features. Features that you don't end up using every day, and so they never really sink in.
There are plenty of simpler things I'd like them to tackle before. Just in the past week of work two examples come to mind (just to illustrate that there are still issues)
- I was trying to get some function calls to inline and had to fight the compiler for half the day to get it to inline them for me (I still don;t understand why it's okay that the compiler ignored the inline keyword. Can't we assume that I as the developer will know what's best if I explicitly put the keyword there?)
- Then the next day I had the opposite problem where a branch was being inlined even though I wanted a if(/rare event/){ jump to some method}. Again the language provides no tools for me to accomplish that
There are tons of issues with the standard library containers and data locality that have been ignored too.
C++ is THE high performance language - can we spend some time on performance and stop pretending that there isn't room for improvement on that front?
It makes sense to consider what you make configurable especially when it is this expensive. I can't imagine routes change that much and when they do they probably change with code.
This seems like the kind of yak shaving that can really ruin a perfectly good project.
That isn't to say configuration doesn't have its place! One of the reasons I love python and interpreted languages is it is very easy to do when you need it.
I had to write some introspection system for a project a while ago. It wasn't the same use case and calling by name wasn't even required. It was more about handling random object from an unified interface issue.
Anyway, here is an extreme oversimplified introspection engine I just wrote. It only implement creating wrapper "generic" object by class name. While we both used some of the same tricks, I have a few more that make introspection simpler. My implementation also work on embedded systems without a working "dynamic_cast" and C++ runtime type identifier.
Edit: Just to clarify, this is a much smaller subset of what you implemented in the blog post, but it can be extended into a full introspection system using more lambdas maps and variadic template tricks.
There are use cases for which reflection is very useful. Things like automatic serialization are an obvious example. There's also automatic printing of complex types, automatic guis (think unity).
There are a few use cases which aren't very obvious like having a vector for each element in a structure instead of having a vector of the structure. This greatly improves cache locality and can be a great gain in performance for some applications.
Of course this is more introspection which is a specific case of reflection but all these examples are definitely not 'ugly hacks' and could greatly improve C++ as a language.
Then why do the solutions people use to work around the absence of reflection always seem to suck so much? They're (in some combination) verbose, highly un-DRY, require spectacular feats of type-system-fu, and at the end of the day they deliver capabilities that come up laughably short of their competition in other languages. I'm still waiting for a member of the "Reflection is Useless" crowd to show me a serialization framework on par with what I find in Java (100% DRY, 100% automatic for POD, 100% override-able without a type repository, 100% devoid of compiler/linker trickery). Or a desktop framework that offers UI design capabilities comparable to the introspection-based ones I find in ObjC, C#, or whatever you want to call the home-brew C++-with-reflection that Qt uses.
The best apprach is to have "reflection" (really, access to the compiler AST or IR) at compile time and generate code from this, since that minimizes the run-time overhead.
See Rust's serde for an example of that for serialization.
Yes. And don't just think about how messy the homegrown alternatives are - note how much effort people go to add them in! It's just such a useful thing to have - serialization; logging; UI generation; automatic scripting language bindings - and that's why people bother.
I applaud your effort and I actually like your result but this just shows how much we need real reflection in C++17. I like to use C++ for its performance and the recent standards were a big leap in the right direction but reflection just isn't one of C++'s strengths.
I found myself recently needing to embed an open source library which used a similar mechanism for its plugin system. The goals of the plugin design of this system were to make it possible to cleanly register plugins for a range of different types in a manner which preserved type safety and to be able to use the same registration mechanism to define any of these plugins within arbitrary compilation units so that plugins could be easily added by consumers of the system -- using the same mechanism as used for the built in plugins... Including for plugins loaded at runtime.
The plugin registration mechanism involves a pre processor macro that authors invoke on each plugin class after definition -- this macro declares that the specified plugin implements a special factoryinstance template class. The factoryinstance template class is paramterized by the plugin type and the specific plugin implementation and provides a default constructor that retrieves the class name of the plugin implementation class via the qt meta object system and adds it to a central registry for each plugin type. This template class also contains a static member variable of the self same factoryinstance template class. This member variable gets initialized by default by the compiler during static initialization which causes the default constructor for the specialized type to be called which causes the plugin to be added to a registry with correct name and correct type information intact ...
This was all well and good however had a very annoying consequence for me as i wanted to embed some of the functionality of this system into a reusable static archive (the original system authors were only concerned with shared library scenarios). At the end of the day no code linking against this archive will make any reference to any symbols of any of these template class implementations -- so the linker will strip all these symbols away and remove all the static initialization calls -- leaving an empty plugin registry in the runtime environment of
The final binary ...
For now I work around the issue by requiring that each consumer of the static library pass linker flags to force load the whole static -- this works however makes the binaries larger than needed as each binary must include all plugins and the linker is not able to include just the ones used ... At some point I'm going to need to figure out the smallest set of changes to this registration mechanism to adapt this library to the embedded use case such that a consumer of the library will include only the plugins they use ... I haven't figured out what that looks like yet ...
This issue was kind of a mind warping ... Sometimes the more I learn about C++, the more insane it seems ...
I wonder how long it'll be before C++ loses its performance advantages. It used to be low level enough that just using it meant you probably got a pretty good performance boost. But if they keep adding features like variants and reflection, and if people actually start using them, it seems like performance will necessarily decline.
Or, if not, other languages will look at C++ to see how they handle these features and keep performance, and improve their compilers.
Variants and static compile-time reflection will both be a boon to runtime performance. The former means less heap allocation and better locality and the latter means less work at runtime and better in-memory layout (for example, instead of parsing JSON to a hash map you can copy JSON values directly in to your objects)
If C++ "loses its performance advantages" it will entirely be due to the culture and not the language itself, because you can still do inline assembler in C++. I've long believed that the reason why lower-level languages have performance advantages is because their paucity of "rich" features means their users often find simpler, more efficient ways of solving problems.
I've noticed this even between C++ and C - without classes, inheritance, and so forth, you tend to think a lot more about whether you need something class-like before going ahead and doing it, and many times a simple function is all you need.
Adding new features almost certainly invites plenty of advocates who will opine about how much better they are (which is true in some situations), encourating their use, even if not actually necessary. This is problematic when these features make it easy to generate large amounts of inefficient code.
People often think of C++ as a Object oriented language. That a very old fashioned way at looking at it. Most modern C++ code avoids OOP like the plague. For instance the author of STL (The C++ standard template library) calls OOP, philosophically unsound, a hoax.
No need to worry. They only adds zero cost abstractions. As in, no abstractions or features are added that have a runtime performance cost compared to the optimal hand crafted solution. The standards committee is very anal about this. Also the "you dont pay for what you dont use" rule applies. (Exceptions might be an exception)
Compile time has gotten worse though with some new features. Hopefully modules will help cut it back a bit.
I think the term "zero cost abstraction" is a bit misleading. Abstraction means that developers don't have to think about some lower level aspects that they would have given some thought otherwise. That tends to result in inefficiencies.
For instance, C programmers will think a lot more about the size of buffers, how big they need to be, can they be fixed size, can they be reused, etc, because expanding them is often a manual task. In C++ it's so easy to do things like this even though it could be done much more efficiently:
string parse(const string& data) {
stringstream result;
for (char c : data) {
if (someCondition(c)) {
result << transform(c);
}
}
return result.str();
}
I dont think I agree. Abstraction in my view is (mostly) about removing implementation details that dont change from one implementation to another. Its about removing boilerplate until just the fundamental features of the implementations remain. It would be a bad abstraction if details (low level or otherwise) that do change, are removed.
Im not sure I get your example. Are you implying that this C++ code is inefficient because the allocation for result is not done in advance? If you know the size of result in advance, there is no reason to use a stream for result. You can simply use std::string and reserve.
>If you know the size of result in advance, there is no reason to use a stream for result.
Oh yes there is a reason. The reason is that streams are a convenient abstraction because there is very likely an operator<< for whatever transform(c) returns. And why is there an operator<<? Because streams abstract from the type of sink (string, file, network).
You are absolutely right, that this can be implemented a lot more efficiently. That's exactly my point. Abstractions lure us into using inefficient solutions without thinking.
You say "If you know the size of result in advance [...]". Well, exactly. Do you? That's something C programmers think about long and hard but you don't have to think about it if you use streams or even string abstractions naively.
This isn't just a case of "you can write bad code in any language". It's what abstractions do. Allow us to ignore stuff that is unnecessary if the goal is simply to arrive at a correct but less efficient solution.
...in a web framework in C++, where presumably the developer would need to write C++ anyway to make use of it?
This article is quite long and complex
No kidding. And to me, all that complexity just screams "you're doing it wrong."
I read this article all the way to the end, and was disappointed to find that, after all that code, it ends with a single macro call --- in C++ --- and leaves the reader wondering about the original goal of using JSON.
Years ago I spent a brief amount of time maintaining Java code that was written in this reflection-heavy, everything-configurable (XML and even some ad-hoc text formats, but similar idea) style, and it was not at all enjoyable to get everything set up correctly nor debug the monstrosity. On a somewhat related note, http://discuss.joelonsoftware.com/default.asp?joel.3.219431....