Baked-in compiler to generate extra functions

[Ganado]

Let's say I had a program, and I wanted it to compile more code, generated at run-time. (Yes, I know I could use a scripting language, but for purposes of knowledge, let's say I have to do it the way I describe.)

Obviously, I can do something like the equivalent of system("g++ generated_code.cpp -o generated_code");, and force the user to have to download GCC (or some other compiler) separately to get this feature to work. (Or perhaps ship it with my download, if licensing lets you. Not the point.)

But I was wondering if there are more-eloquent solutions that exist in the real world? Can anyone point me to any? Perhaps a game engine or something has a compiler (NOT interpreter, preferably native C++ but it could be some .NET thing) baked into it that then spits out game code? Just an example, I'm not even trying to make a game. What I'm actually trying to do is compile some math equations that the user generates, and it just got me curious.

[JLBorges]

> What I'm actually trying to do is compile some math equations that the user generates

NativeJIT (library for just-in-time compilation of expressions) may be of interest.
https://github.com/BitFunnel/NativeJIT

[helios]

Using libclang to do in-memory C++ compilation and run-time linking:
https://github.com/Helios-vmg/Borderless/tree/master/src/plugin-core/Cpp
Some documentation:
https://github.com/Helios-vmg/Borderless/blob/master/doc/Plugins.txt#L131

[keskiverto]

What I'm actually trying to do is compile some math equations that the user generates, and it just got me curious.

Compile an equation?

One of the basic homework examples is the simple calculator that handles only
lhs op rhs where op is + - * /
That program does simple input parsing.

The next step is a parser for more complex equations. It constructs a tree of objects. There will be a way to evaluate the equation, possibly by recursive (virtual or visitor) method.

A compiler takes a further step and stores the tree of objects as a serie of CPU instructions.


Do you really need JIT binary (interesting topic in itself), or would OO-solution suffice?

[dhayden]

Just because a program is created with a compiler doesn't mean that it can compile code itself. It's a little like asking if an automobile can create a pickup truck because perhaps the car has an automobile factory under the hood somewhere.

Some interpreted languages can do this sort of thing since the interpreter is a necessary part of the runtime system.

[Ganado]

JLBorges,
> The expression isn't known until runtime.
> The expression will be evaluated enough times to amortize the cost of compilation.
> Latency and throughput demands require low cost for compilation.
Yes, that sounds very much like what I was looking for. I will try this out this weekend (time allowing).

helios,
> The C++ implementation is provided by Clang, acting as a JIT compiler. The
> performance of the generated code is identical to that of code generated by the
> command line compiler.
Thanks, I'll explore your code too, which could be equally useful.
That also sounds promising. I will test it out when I have time.

keskiverto,
I've made an expression parser using trees and the shunting yard algorithm, but I could never get it to be as fast as compiled code. I admit I have not used a library someone else has made for run-time expression parsing, so I should do more benchmarks. I could also never get a scripting language (Python) to be close in speed to C++ when dealing with per-pixel operations with math equations. Given this, I was really curious as to whether there were compilation-based solutions instead of interpretation, even if just for the sake of curiosity (but I am more confident than not that it would have real speed benefits, as well).
But yes, perhaps I should try out something like https://github.com/cparse/cparse and do some benchmarks with it, to see exactly what speed I'm losing or gaining compared to compilation.

dhayden,
With all respect, I never implied that. Sorry if it seems that way. But of course, what you said is correct.

Thanks all. Probably won't be able to actually test out code for a few days but I appreciate the fast replies.

[helios]

I could also never get a scripting language (Python) to be close in speed to C++ when dealing with per-pixel operations with math equations.

Interesting that you mention this, because the reason I integrated libclang into my project was because luajit generated some awful code in certain cases. My use case was implementing user-defined graphical filters, such as Floyd-Steinberg dithering.

If you already have a parser, an interesting alternative would be to generate machine code.
https://github.com/asmjit/asmjit/
It would be a good chance to learn a bit about x86 Assembly, if you don't know any already.