System calls abuse, and prevention

[Ganado]

I've seen it written sometimes in various places that using a system() call opens a "gaping security hole" or has the "potential for abuse", something along those lines.

My question is under what circumstances this potential of abuse becomes reality?

One thing I read (iirc) was that if char literals are in a program, it is easy to modify these outside, allowing someone to replace the text inside the system call with something else. This makes sense, but I can think of a myriad of programs that can do functionality outside of what C++ itself gives by just itself, so how is this functionality gained if not through system calls?

One prominent example I am questioning is the use of "mkdir", conveniently both a windows and unix/linux command. If I have some installer or other type of similar program that can deal with your OS's file system in some way, how would there be any way to make a directory without using system("mkdir"+name) or the like. There are probably thousands of programs or simple installers that have the capability of creating a whole tree of directories or files, so do all of these programs have this seemingly-inherent problem with system()?

[giblit]

My question is under what circumstances this potential of abuse becomes reality?

system("mkdir"+name) You just answered yourself there. When they enter their "name" they can put any commands they want and the system will call those commands.

[Ganado]

Thanks for the reply, is the fact that someone could possibly? go into an executable file and find the string literal "mkdir" and change it to something else, or is it only run-time issues like the user typing in another command instead of a real name that's the problem?
How do so many installers or other programs prevent using security-risking system calls to do things like make a tree of directories, or delete some "temp" file that usually only exist during run-time?

(ex: after compiling a program that has system("mkdir test"), I can indeed find the string literal "mkdir test" inside the binary.)

[giblit]

I think the biggest issue is when you call a program to be executed. Since you could theoretically call any program then. http://www.cplusplus.com/forum/articles/11153/

Though the run time thing like you had also is a big issue since I could theoretically do anything I wanted then.

It pretty much just leaves a hole in your security.

[Ganado]

Okay the security part makes sense now, but I was still lost on how to still give functionality without compromising security.

and thus:
I did some more searching, and I had totally forgotten about OS's have their own functions available to use, in a C or C++ way instead of through the command line.

so linux has int mkdir(const char *, mode_t); in #include <sys/stat.h> ,
and windows has CreateDirectory(name, attributes).

So basically I shouldn't use a system() call to do something, I should rather find the functions the OS's library already provides. I hope I have that right understanding.

[giblit]

Yeah I would suggest using the native ones since system just calls them anyways AFAIK.