- Forum
- General C++ Programming
- Compile C++ code at runtime [g++]
Compile C++ code at runtime [g++]
My program needs to compile various source files at runtime.
What is the most elegant way to compile cross platform with g++ from within my program? Is there a gcc-library I can use?
I know that I could use popen() to open a Unix pipe and call g++ as command line tool. But first it isn't really cross platform and second it doesn't seem elegant to me. I could do it this way, but I first wanted to know from someone more experienced than me if this is really the appropriate way.
What is the most elegant way to compile cross platform with g++ from within my program? Is there a gcc-library I can use?
I know that I could use popen() to open a Unix pipe and call g++ as command line tool. But first it isn't really cross platform and second it doesn't seem elegant to me. I could do it this way, but I first wanted to know from someone more experienced than me if this is really the appropriate way.
You can use sysem-specific functions, wrap them into cross-platform layer and use resulting library in your program.
Just wondering whether you might be better off with a cross platform IDE like QtCreator?
Also, Qt is independent of the compiler - you could use clang for example. And there are Linux versions.
Also, Qt is independent of the compiler - you could use clang for example. And there are Linux versions.
I'm not so sure that's what you really need to do.
What exactly is your situation, and what are you trying to accomplish?
(Because, unless there is some independent process, like a human or an external server, providing unknown source code, you really don't need to be a compiler.)
Are you making a plugin system?
What exactly is your situation, and what are you trying to accomplish?
(Because, unless there is some independent process, like a human or an external server, providing unknown source code, you really don't need to be a compiler.)
Are you making a plugin system?
Sorry for my late answer, but I was busy last month.
@Duoas
Kind of. It'll be a system for small apps (with some kind of game engine and a self designed database server, which is already working). Also I've written my own IDE and designed a language that is a slight superset of C++ and XML/CSS. So my plan is to preprocess that code (already implemented) and then pass it to g++ (to compile a shared object).
This obviously has to be done in runtime ;-)
I was just curious if there was any dl-API for g++.
@TheIdeasMan
I'm using Codeblocks as IDE. Actually I am programming an IDE. Or what exactly is your point?
I'm now sticking with MiiNiPaa solution. I originally wanted to use preprocessor directives, but using dls seems more "customizable" to me.
@Duoas
Kind of. It'll be a system for small apps (with some kind of game engine and a self designed database server, which is already working). Also I've written my own IDE and designed a language that is a slight superset of C++ and XML/CSS. So my plan is to preprocess that code (already implemented) and then pass it to g++ (to compile a shared object).
This obviously has to be done in runtime ;-)
I was just curious if there was any dl-API for g++.
@TheIdeasMan
I'm using Codeblocks as IDE. Actually I am programming an IDE. Or what exactly is your point?
I'm now sticking with MiiNiPaa solution. I originally wanted to use preprocessor directives, but using dls seems more "customizable" to me.
Last edited on
Writing a front-end for GCC is a particularly serious headache, so other solutions would be better.
If you weren't already so invested in your current solution I would suggest using a scripting language library like Tcl or Python.
If you weren't already so invested in your current solution I would suggest using a scripting language library like Tcl or Python.
Actually most of the work I've already done isn't very specific for a particular base language. Indeed I already planned (or at least thought of) supporting more than one "base language". So I'm not forced to use C++.
But I'm wondering what's the big problem about it? Isn't qmake doing a similar thing? (Interpreting a few symbols in the source, somehow compiling the XML files and then pass it all to a C++ compiler).
I'm not doing any fundamental changes in the language. There is just some special syntax for accessing the database and for using XML based data structures. This is what I mean with slight superset.
Also C++ is a rich and very performant language unlike most scripting languages. Which is very important to me. Although some scripting languages are said to be very performant nowadays (especially python or javascript) there is an overhead that you can't argue away. And it can impact the performance. In some cases even in smaller programs.
Can you provide me with reasons why this is a bad idea?
But I'm wondering what's the big problem about it? Isn't qmake doing a similar thing? (Interpreting a few symbols in the source, somehow compiling the XML files and then pass it all to a C++ compiler).
I'm not doing any fundamental changes in the language. There is just some special syntax for accessing the database and for using XML based data structures. This is what I mean with slight superset.
Also C++ is a rich and very performant language unlike most scripting languages. Which is very important to me. Although some scripting languages are said to be very performant nowadays (especially python or javascript) there is an overhead that you can't argue away. And it can impact the performance. In some cases even in smaller programs.
Can you provide me with reasons why this is a bad idea?
It's not that it's a bad idea, just a very expensive one (in terms of work you must do).
If you were to use Tcl, for example, you would have a very small library to link into your application, which you can bind tightly, and which has very good performance.
As for scripted-vs-compiled performance, that is often a false dichotomy. No matter which choice you make, the code must be at some point "compiled". Things like Tcl and Python "byte-code compile" -- producing opcodes the interpreter understands. Things like Java do something similar, but produce opcodes that the JVM understands (and the JVM must be present on your user's machine to be useful).
But mainly, what use-cases have you got that show that performance in the user plug-in code is going to be a problem? What exactly will people doing with it?
And for the original question about GCC -- alas, that's your only real solution. If you want to see an existing project that does more or less what you want, check out CriTcl.
http://wiki.tcl.tk/critcl
What this entails, though, is that you handle specific platforms specially (like Windows, on which is not safe to assume an accessible compiler, so you would have to provide one as part of your package) and fall back to using GCC on unknown platforms, and if that fails, either try to interpret the script yourself or just tell your users they're out of luck.
All of this, IMHO, is bad karma for an application that should "just work".
In other words, unless your users know that they must have a working, accessible, and sane compile-time environment as part of your application's deployment, then it will be something that will frustrate your users.
Whereas, using a language designed for these kinds of tasks, like Tcl (or whatever else you may have in mind -- I hear Lua's nice) makes things go nicely.
Hope this helps.
If you were to use Tcl, for example, you would have a very small library to link into your application, which you can bind tightly, and which has very good performance.
As for scripted-vs-compiled performance, that is often a false dichotomy. No matter which choice you make, the code must be at some point "compiled". Things like Tcl and Python "byte-code compile" -- producing opcodes the interpreter understands. Things like Java do something similar, but produce opcodes that the JVM understands (and the JVM must be present on your user's machine to be useful).
But mainly, what use-cases have you got that show that performance in the user plug-in code is going to be a problem? What exactly will people doing with it?
And for the original question about GCC -- alas, that's your only real solution. If you want to see an existing project that does more or less what you want, check out CriTcl.
http://wiki.tcl.tk/critcl
What this entails, though, is that you handle specific platforms specially (like Windows, on which is not safe to assume an accessible compiler, so you would have to provide one as part of your package) and fall back to using GCC on unknown platforms, and if that fails, either try to interpret the script yourself or just tell your users they're out of luck.
All of this, IMHO, is bad karma for an application that should "just work".
In other words, unless your users know that they must have a working, accessible, and sane compile-time environment as part of your application's deployment, then it will be something that will frustrate your users.
Whereas, using a language designed for these kinds of tasks, like Tcl (or whatever else you may have in mind -- I hear Lua's nice) makes things go nicely.
Hope this helps.
First of all: This code I want to compile aren't plugins. They are more or less small applications. The main program doesn't do much without them. It's somehow like a small game engine or a engine for simulations. Most things can be set up in config files. You don't need a single line of code to create a working application. but you can use lots of entry points to replace things or to extend it.
I didn't say anything about the compilation/interpretation process in particular. I just said scripts are sometimes way slower than a lower level programming language. Probably this can have many reasons. I'm no expert, but I came across these two by myself (correct me if I'm wrong):
1) Optimization:
A script can't be optimized the same way as a C++ Programm due to paradigms like dynamic typing. The compiler has a hard time optimizing the code if the variable types changes all the time. E.g. look at Mozilla's "asm.js" which led to an incredible performance boost, just by making ECMAscript type-safe. On the other hand I observed that many VM's are able to optimize things in runtime, something that c++ can't do. But in my observation this hasn't much impact.
2) Virtual Machine:
As you mentioned scripted languages always come with a VM (mainly for memory management). These are always generating some extend of overhead. In my personal opinion this needn't lead to a remarkable performance change, but at least in some languages it really does. I gave up programming in Java because of the high memory consumption.
I just want to put this in numbers:
I implemented a short algorithm that adds up millions of numbers. This isn't a standard task in most programs, but as my engine is able to do physical simulations in real time and as most of the algorithms should be replaceable by the user such calculations definitely matter to me.
With the exact same implementation in both languages I got an average of 8.7 seconds in python and 0.5 seconds in C++. This is a remarkable difference. (My own self test might not be representative, but other professional tests show similar results. Just look at this http://readwrite.com/2011/06/06/cpp-go-java-scala-performance-benchmark . These aren't scripting languages, but it shows the big difference to other languages. Also you can play with this website: http://benchmarksgame.alioth.debian.org/u64/benchmark.php?test=fannkuchredux&lang=all&data=u64)
My engine is among others capable of rendering 2D/3D content in real time (GPU) and simulating physics (CPU, at a later time perhaps also GPU). As I said lots of critical parts should be replaceable. Maybe (if necessary) the user should even be able to optimize some parts with ASM. But I'm not quite sure yet about that one (because of portability). Because this is real time even a performance difference of about 10% would matter to me.
Actually there are two types of users. There are developers and end-users. Only the developers will need a working build-environment. And only the developers will use the IDE. When they're ready the IDE automatically cross-compiles the necessary libs and executables. Then all is uploaded to a repository on my webserver. (The repository/packaging system is already working. Only the compiling system is missing yet.) The end users just download and install these binary files (through the packiging service). So they don't need the build system.
I think making sure that g++ is installed under Linux is not a big deal. Under Windows and OS X I could package it as you said. This not a big deal either as I wrote a packaging service anyway.
For other platforms: I think that developers that are not working under Windows/OS X or Linux are able to install g++ on their own. Also Linux/Windows/OS X is the minimum portability that I want to achieve.
Also consider that I'm a non-professional, just-for-fun OpenSource programmer. My main question isn't: "What is the use/cost factor?", my main question is: "Does it make fun?" I don't want do something that doesn't make sense though. So do you still think I should use scripts?
I didn't say anything about the compilation/interpretation process in particular. I just said scripts are sometimes way slower than a lower level programming language. Probably this can have many reasons. I'm no expert, but I came across these two by myself (correct me if I'm wrong):
1) Optimization:
A script can't be optimized the same way as a C++ Programm due to paradigms like dynamic typing. The compiler has a hard time optimizing the code if the variable types changes all the time. E.g. look at Mozilla's "asm.js" which led to an incredible performance boost, just by making ECMAscript type-safe. On the other hand I observed that many VM's are able to optimize things in runtime, something that c++ can't do. But in my observation this hasn't much impact.
2) Virtual Machine:
As you mentioned scripted languages always come with a VM (mainly for memory management). These are always generating some extend of overhead. In my personal opinion this needn't lead to a remarkable performance change, but at least in some languages it really does. I gave up programming in Java because of the high memory consumption.
I just want to put this in numbers:
I implemented a short algorithm that adds up millions of numbers. This isn't a standard task in most programs, but as my engine is able to do physical simulations in real time and as most of the algorithms should be replaceable by the user such calculations definitely matter to me.
With the exact same implementation in both languages I got an average of 8.7 seconds in python and 0.5 seconds in C++. This is a remarkable difference. (My own self test might not be representative, but other professional tests show similar results. Just look at this http://readwrite.com/2011/06/06/cpp-go-java-scala-performance-benchmark . These aren't scripting languages, but it shows the big difference to other languages. Also you can play with this website: http://benchmarksgame.alioth.debian.org/u64/benchmark.php?test=fannkuchredux&lang=all&data=u64)
My engine is among others capable of rendering 2D/3D content in real time (GPU) and simulating physics (CPU, at a later time perhaps also GPU). As I said lots of critical parts should be replaceable. Maybe (if necessary) the user should even be able to optimize some parts with ASM. But I'm not quite sure yet about that one (because of portability). Because this is real time even a performance difference of about 10% would matter to me.
| In other words, unless your users know that they must have a working, accessible, and sane compile-time environment as part of your application's deployment, then it will be something that will frustrate your users. |
Actually there are two types of users. There are developers and end-users. Only the developers will need a working build-environment. And only the developers will use the IDE. When they're ready the IDE automatically cross-compiles the necessary libs and executables. Then all is uploaded to a repository on my webserver. (The repository/packaging system is already working. Only the compiling system is missing yet.) The end users just download and install these binary files (through the packiging service). So they don't need the build system.
I think making sure that g++ is installed under Linux is not a big deal. Under Windows and OS X I could package it as you said. This not a big deal either as I wrote a packaging service anyway.
For other platforms: I think that developers that are not working under Windows/OS X or Linux are able to install g++ on their own. Also Linux/Windows/OS X is the minimum portability that I want to achieve.
Also consider that I'm a non-professional, just-for-fun OpenSource programmer. My main question isn't: "What is the use/cost factor?", my main question is: "Does it make fun?" I don't want do something that doesn't make sense though. So do you still think I should use scripts?
A number of your assumptions are not correct, but I'm not going to take time to address them now.
It sounds like you are creating a production environment. And I think it would be fair to require your users to have the GCC as a dependency.
Just make sure that your tools also help your users identify (select, package, and distribute) output dependencies (DLLs, etc).
It sounds like you are creating a production environment. And I think it would be fair to require your users to have the GCC as a dependency.
Just make sure that your tools also help your users identify (select, package, and distribute) output dependencies (DLLs, etc).
Try using C# with Mono. You can integrate C# into your program (And Unity uses this as an interface to it's C++ core.). C#'s performance is pretty decent.
Topic archived. No new replies allowed.