Library Design and Organization Question

[DeXecipher]

Is it faster and more effective to design a seperate copy of an engine for (each) different platform or to handle all platforms in one version (source) of an engine (multimedia framework) in one big project (distributed for different compilers)?

Im asking this because recently broke apart my Game Engine in order to better design it and divide it into more manageable modules.

[DeXecipher]

I would really appreciate any experienced persons opinion. please help.

[DeXecipher]

Can any experienced person, who has made or worked on a project answer my question?

[DeXecipher]

please help

[JLBorges]

> design a seperate copy of an engine for (each) different platform
> or to handle all platforms in one version

Probably neither - each is too extreme. An approach like this would perhaps be more flexible:

Since the engine logic would be the same once platform differences are abstracted away, have a single engine class which uses object composition to handle different platforms.
http://en.wikipedia.org/wiki/Composition_over_inheritance

For instance, if the handling of the joystick is different across platforms, abstract it away in an interface:

struct joy_stick // pure interface
{
    virtual ~joy_stick() ;

    virtual void x_axis_range( int from, int to ) = 0 ; // set x-axis range
    virtual std::pair<int,int>  x_axis_range() const = 0 ; // get x-axis range
    virtual int x() const = 0 ; // get x-axis value

    // etc

};

Have an implementation of the interface, one for each platform. For instance:

struct sparc_solaris_joy_stick : joy_stick
{
    virtual void x_axis_range( int from, int to ) override ;
    virtual std::pair<int,int>  x_axis_range() const override ;
    virtual int x() const override ;

    // etc

};

And then use an abstract factory to create the components for the game engine. http://sourcemaking.com/design_patterns/abstract_factory

struct my_game_engine
{
    my_game_engine() : js( component_factory::instace().create_joystick() ) /* ... */
    {
        // ...
    }

    void do_something()
    {
        int xpos = js->x() ;
        // etc
    }

    std::shared_ptr<joy_stick> js ;
};

[Disch]

You are asking a specific question here, but then the reason you say you are asking is kind of unrelated.

The question:

Is it faster and more effective to design a seperate copy of an engine for (each) different platform or to handle all platforms in one version (source) of an engine (multimedia framework) in one big project

You don't need multiple copies of the same engine. What you really want here is to separate/remove platform specific code from your engine and interface with it through an abstracted layer.

I/O is typically the only place where platforms differ (unless you're doing extremely low level stuff, but since you're talking about a game I'm going to assume that isn't the case here). Fortunately I/O for most computer-style devices are the same. There's the concept of a keyboard, a mouse, and joypads for input. For output, there's concepts of windows, streaming audio, textured polygons, etc.

The underlying concepts are the same across platforms, but the implementation to make them work is what differs. For example on Windows, if you want to get keyboard input you can use GetAsyncKeyState. But of course that exists only on windows and not on other platforms. So if you want your engine to be portable, you need to abstract that.

This is typically done by putting OS specific calls behind a wrapper. Continuing with that simple keyboard example, you could create your own GetKey function:

bool GetKey(int key)
{
  if(GetAsyncKeyState(key) & 0x8000)
    return true;
  else
    return false;
}

This is a big step towards portability because this routine can be easily swapped out with a different implementation to work on Linux or whatever platform you want:

#ifdef WIN32  // if compiling for Windows...
bool GetKey(int key)
{
  //... do the windows implementation
}
#else // otherwise (assuming Linux... I forget what's defined for Linux... maybe __unix__?)
bool GetKey(int key)
{
  //... do the linux implementation that does the same thing
}

Do all of your I/O that way, and your code becomes [mostly] portable (there are other things that can make your code non-portable, but I'm trying to keep this brief).


But it can be even easier than that! Several libraries like SFML and SDL already do this for you. They have multiple different implementations... one for each platform they support. But the interface for the library is always the same.


As for coding for different compilers, this isn't much of an issue, really. Ideally you'd want the same code to compile the same way on all mainstream compilers. The best way to accomplish this is:

1) Stick to C++ standards and don't try to do anything that's undefined.
2) Don't use compiler-specific extensions
3) Pick which version of which compilers you want to support, and don't use features not supported in those compilers (for example if you want to support older compilers, you might not want to use C++11 functionality).
4) Compile your code on all target compilers and make sure it compiles OK.




The unrelated bit:

Im asking this because recently broke apart my Game Engine in order to better design it and divide it into more manageable modules.

Breaking a large project into smaller modules is often a good idea. But it might also be unnecessary. It's really a judgement call you have to learn to make for yourself.

Though I'm not sure what this has to do with different versions or different OS's/compilers. There should only be one copy of the game engine.

[JLBorges]

Favour abstraction over conditional compilation.
Except in the most trivial cases, avoid this kind of stuff like the plague.

#ifdef WIN32  // if compiling for Windows...
bool GetKey(int key)
{
  //... do the windows implementation
}
#else // otherwise (assuming Linux... I forget what's defined for Linux... maybe __unix__?)
bool GetKey(int key)
{
  //... do the linux implementation that does the same thing
}

Why this is a bad idea, even in C, has been known for a long time; since the early days of C.
A common C alternative (makefiles / configure scripts) is illustrated in this discussion:
http://stackoverflow.com/questions/1851181/why-should-ifdef-be-avoided-in-c-files

[Disch]

Yes yes, of course I agree. I suppose I was oversimplifying my example.

[DeXecipher]

I agree that my way is hard but so far I have been abstracting by having by letting a namespace "system" be my low-level layer for various subsystems. Its a system where all other namespaces are dependant only upon the platform detection and mapping using macros "if defs" and such. Im doing all this from scratch using only the basics of the c/c++, new/delete for memory, platform specific c-style interface inside a system dependant wrapper, abstracted data structures built on top of this layer (meaning function calls and classes which map helper functions). It is slow as hell, but its the only way I've been thinking about it because I want to do most things aside from io/physics/ and super lowel stuff for things such as Direct X and OpenGl.

for example

namespace MainEngineNamespace
{
   //this is like platform layer
  #define DEBUG //or don't
  #define WIN32_LEAN_AND_MEAN//or not because it will detect macros in headers for
  //the ide/compiler you use.
  //detect the platform define as well as the version of compiler
 //define interrupts an error handling tags (what is a failure and was is not)
 //define types
 //define whether we are building or loading a DLL

 namespace SubSystemSys{/*library dependant code here*/}//all subsequent /
//following systems will be dependant upon this layer
  //pipeline of dependancy insures stability and consistency
  namespace SubsystemA{}//(basic file IO and keyboard and mouse "if supported" perhaps)
 namespace SubsystemB{}//Memory based systems, File System, Memory Pool/Manager
  namespace SubsystemC{}//Data structures perhaps, arrays, strings, vectors, trees, etc. 

 namespace SubsystemD{}//Gui
  namespace SubsystemE{}//(Graphics perhaps) Im stopping at graphics, lol. thats //enough >_> for me to get something going.
 namespace SubsystemF{}//ok Networking goes here and i'm done.
}

keep in mind that this will not be all in one file, I intend to divide these subsystems into Static and Dynamic (DLL) libraries.

I don't care too much about only working on working on one platform as a one man so far but I understand the infeasibility in terms of time, but it will be extremely flexible the way I do it. All the user would have to do is define WINDOWS or WIN32_LEAN_AND_MEAN or _XBOX and the library will detect it and run for ANY platform, the only difference would be versions for different compilers for optimization.

Windows isnt too different from MAC in its architecture. Let me know if I'm not wrong but aren't they both derivatives of Unix? Its slow and "hardcore" but it works and I appreciate the feedback as I have not read them all but I will take them into consideration the methods you guys describe. I've been mimicking techniques I've seen in WINAPI, DirectX, Opengl, and Ogre design.

In short, I'm using an iterative process, learning what works, and it requires knowing the API to some extent.

This is my first project with a scopable outcome and I'm going all out on it but I have experienced failure and it has taught me not to stop until I have something.

Just to clarify all of my subsystems will be implemented in different dlls and wrap helper functions onto this "system" namespace that I have created, this achieves abstraction and control, but its slow for one person.

[DeXecipher]

As for the composition idea, I've been reading that as well and it seems like it would simplify the process of development if you have enough experience of how to get different components to work together in a non-limiting way. I think the drawback to that is that you can't really code too much as you go (you rely on soundly designed systems that all work together). I think that would require a lot more experience than I have. I'm only experienced with memory, gui, and basic file io not super complex maths and graphics methods. I'm working on this while I'm in school in addition. But If I showed you my engine, the beginnings, you would see that I am using a component based design in the upper most layers (user level), a top down approach, but a pipeline like design for subsystems that are stacked or coupled.

[DeXecipher]

Also, having an implementation of each interface for each platform is not too short from doing it through inheritance for each level. The way I'm doing it I stack everything that is needed in a system dependant layer controlled by #ifdef PLATFORM_X == [platform here]
then use X set of methods for this platform, damn ad-hocness flaw :(.

I stack everything into large source files, which make the source files larger of course but everything will be consistent.

[DeXecipher]

Finally addressing the multiple copies of the game engine problem, There will be one copy of the game engine I just expressed my meaning in the wrong way. It will be one version of the engine what I mean is that I will make sure that the same code that compiles on MinGW will compile on VS studio. Some Engines, Irrlicht for example don't compile on different IDEs right from unboxing them but they end up splitting components up for each platform while simultaneously having to keep up with the subtle differences between what code each compiler will accept.

[JLBorges]

> I think the drawback to that is that you can't really code too much as you go...

Yes. Interfaces need to be designed early. And ideally they should be immutable (a fundamental limitation of languages like C++ and Java which kludge run-time polymorphism atop a static type system).


> I stack everything into large source files ...

Which is not a great design idea; it promotes tight coupling.

[DeXecipher]

Tight coupling deals with software systems that have mutual dependencies, my design is a waterfall/iterative model. But i get that Interfaces should be designed earlier. So you are saying that it is ideal to impose limits on the software earlier.

[JLBorges]

> So you are saying that it is ideal to impose limits on the software earlier.

No; what I meant is that interfaces should be closed for modifications and open for extensions.
Martin's version of the "The Open-Closed Principle".
http://en.wikipedia.org/wiki/Open/closed_principle

[DeXecipher]

@JLBorges
Ok now I get what you are saying, thanks for the link. It seems I have to reevaluate my coding philosophy a bit with this project in order to make it faster. All this time I've been going back and forward between the design and implementation for a particilar platform in order to simplify the abstraction between the interface and the particular api. Wrapping an abstraction layer on helper functions might not be too feasible in terms of the time it takes.

Now it makes sense, much sense to me.

By making the interfaces immutable from the beginning I force a design which allows me to focus upon the implementation after design and not interchangeably.
Its a top down style, similiar to what I've been doing but I think it should be faster because I forces only decisions based upon some immutable interface.

But how would you know how to design your interfaces optimally if you've never stepped on a lower level before? Does that come with practice or is it learned or based upon some axioms of expected behavior? Much what a top-down interface is, perhaps?

thanks.

[DeXecipher]

One last question, what if my abstraction (in a c-style interface) uses conditional compilation but it does so in a way that I could further divide it into the implementation for my platform.

For example, this is sort of what I've been doing, in the (header files):

#ifdef (_WIN32 || WIN32_LEAN_AND_MEAN || WIN32 )
#define PLATFORM (ENGINE_PLATFORM_WINDOWS)
#else //some other platform
#define PLATFORM(ENGINE_PLATFORM_UNDEFINED)
#endif 

I define my own PLATFORM and map any premises (previously declared defines) into
a platform and assume the appropriate execution based upon that.

[JLBorges]

> By making the interfaces immutable from the beginning I force a design which allows me
> to focus upon the implementation after design and not interchangeably.

Run-time polymorphism in a statically typed language requires programming to precisely typed immutable interfaces. If we do that, implementations of these interfaces are interchangeable.

For example, the following is possible because std::ostream is programmed to the (generic) interface of std::streambuf and the (object-oriented) interface of std::numpunct<char>.

#include <iostream>
#include <string>
#include <locale>
#include <fstream>
#include <sstream>

void foo( std::ostream& stm, int i = 123456789, double d = 123.45,
           const std::string& str = "hello world" )
{
   stm << "an int: " << i << "\nthen a double: " << std::fixed << d
       << "\nand finally a string: '"  << str << "'" << std::endl ;
}

int main()
{
    // an ostream which uses an implementation of std::streambuf
    // which sends output to stdout
    std::ostream stream( std::cout.rdbuf() ) ;

    foo(stream) ; // this output goes to stdout

    // a custom implementation of std::numpunct<char>
    struct my_numpunct : std::numpunct<char>
    {
        protected:
            virtual char do_thousands_sep() const /*override*/ { return '-' ; }
            virtual std::string do_grouping() const /*override*/
            { return "\x02\x02\x02\x02\x02" ; }
    };

    // switch implementation from std::numpunct<char> to my_numpunct
    stream.imbue( std::locale( stream.getloc(), new my_numpunct ) ) ;

    foo(stream) ; // stream now uses my_numpunct to format numbers

    // an implementation of std::streambuf hich sends output to a file
    std::filebuf fbuf ; fbuf.open( "out.txt", std::ios::out ) ;

    // switch to a different implementation
    stream.rdbuf(&fbuf) ;
    foo(stream) ; // this output goes to the file

    // an implementation of std::streambuf hich sends output to a string
    std::stringbuf sbuf ;

    // switch the implementation once again
    stream.rdbuf(&sbuf) ;
    foo(stream) ; // this outputr goes to memory (string)
    std::clog << "the string has " << sbuf.str().size() << " characters\n" ;

    // switch to a homegrown strewambuf implementation which sends output to a socket
    // ...

    // etc.
}

> But how would you know how to design your interfaces optimally
> if you've never stepped on a lower level before?

I don't. So I explore, write a prototype. The designing of interfaces is typically an iterative process.
Maybe design in bridges (early abstraction) http://sourcemaking.com/design_patterns/bridge
And use adapters (late-abstraction) http://sourcemaking.com/design_patterns/adapter




> One last question, what if my abstraction (in a c-style interface) uses conditional compilation
> but it does so in a way that I could further divide it into the implementation for my platform.

AFAIK (and my knowledge of C is limited, very limited), conditional compilation for declarations in a header is an acceptable programming practice.

[DeXecipher]

thank you.