Shared Memory Security

[sp25]

Hello,
I am implementing IPC between 2 windows processes using shared memory (memory mapped file).

The main reason for selection of this is to have the fastest possible mechanism for sharing data.

1. Is there a way to make this memory area secure and accessible only by certain processes.
2. Are there recommendations for making this whole scheme "tamper proof"?

3. Are there methods through which I can implement some level of authentication without hurting performance?

Would help if there are examples. I haven't done any similar implementation earlier.

Thanks

[helios]

Is there a way to make this memory area secure and accessible only by certain processes.

A normal process will not be able to gain access unless explicitly given access by your program, but a process with administrative privileges can do whatever it wants, up to and including debugging your programs and reading arbitrary bytes in their memory spaces. There isn't much you can do to prevent this.

You could implement a cryptographic communication scheme between the two processes, but if the friggin' system memory is untrusted, where are you going to store the keys? If an attacker running in the same system intercepted the key exchange (e.g. by hooking a pipe write function or a semaphore function), all it would have to do would be to suspend the programs and scan their memory spaces looking for a contiguous set of n bytes that decrypt a known message encrypted by the other key.

Some programs implement anti-debugging techniques. A possibility might be to try to detect when your data may have been compromised and either quit or start over. Needless to say, this approach is far from fool-proof; there is such a thing as anti-anti-debugging.

[JLBorges]

> Is there a way to make this memory area secure and accessible only by certain processes.

This is one way:

Create the file mapping as an un-named kernel object.
(The last argument to CreateFileMapping() is the name of the object; pass nullptr as the name.)

Other processes will not be able to open the file mapping object by themselves.

Duplicate the handle for each of the processes of interest
(and pass the value of the duplicated handle to those processes).
https://msdn.microsoft.com/en-us/library/windows/desktop/ms724251(v=vs.85).aspx

[helios]

An attacker may have hooked CreateFileMapping() and would be able to duplicate the handle for a different process before returning control to the caller. The other process could at this point read the shared memory at any moment.

Additionally, a process may list handles open in another process regardless of whether they are named or not. DuplicateHandle() also works with hSourceProcessHandle arguments that refer to remote processes.

[JLBorges]

> An attacker may have hooked CreateFileMapping() and would be able to duplicate the
> handle for a different process before returning control to the caller.

If you are naive enough (or ignorant enough) to compromise your system to the extent that an attacker can hook system service requests from any process, your program is compromised before any of your code even starts executing.

Tip: Get out of the habit of doing everything you do on the machine as root (administrator). And limit access to the TCB.


> DuplicateHandle() also works with hSourceProcessHandle arguments that refer to remote processes

Tip: Read up on process security and access rights. (In particular, the PROCESS_DUP_HANDLE access right.)

[helios]

I get the feeling that OP wants to protect the shared memory from being accessed by a legitimate user, not necessarily by outside attackers. I don't think it makes sense to try to protect your program's memory from being read if you're already assuming that something from outside the system is capable of even attempting to read it.

Tip: Read up on process security and access rights. (In particular, the PROCESS_DUP_HANDLE access right.)

Yeah, like I said, administrator processes can do whatever they want. It's not difficult to open a process with PROCESS_ALL_ACCESS.