Hi,
I would advise you to stick with detouring NtOpenProcess to protect your process, because majority of the rootkit and malware developers would easily bypass your OpenProcess hook fairly easily. On that note even NtOpenProcess hook can be bypassed fairly easily however it takes a bit more work.
Example trick to bypass NtOpenProcess Hook (Ring3):
x64 Systems:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
|
; Programmer: OrionMaster (Cplusplus Forums)
;
; Basic bypass example of NtOpenProcess via Assembly Routine (x64)
; Not Tested! However I believe it should work.
.code
OpenProcess proc
pushad ; Save all Registers
mov eax, [NtOpenProcess Ordinal]
xor ecx, ecx ; Equivalent to mov ecx, 0
mov dword ptr [ esp + 0x4] , [Operand 1]
mov dword ptr [ esp + 0x8] , [Operand 2]
mov dword ptr [ esp + 0xC] , [Operand 3]
mov dword ptr [ esp + 0x10] , [Operand 4]
call 033:[X86SwitchTo64BitMode]
popad ; Reinstate all registers before esp being incremented
add esp, 4
retn 10
OpenProcess endp
end
|
x86 Systems:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
|
; Programmer: OrionMaster (Cplusplus Forums)
;
; Basic bypass example of NtOpenProcess via Assembly Routine (x86)
; Not Tested! However I believe it should work.
.code
OpenProcess proc
pushad ; Save all registers
mov eax, [NtOpenProcess Ordinal]
mov dword ptr [ esp + 0x4] , [Operand 1]
mov dword ptr [ esp + 0x8] , [Operand 2]
mov dword ptr [ esp + 0xC] , [Operand 3]
mov dword ptr [ esp + 0x10] , [Operand 4]
call [KiFastSystemCall]
popad ; Reinstate all registers
ret
OpenProcess endp
end
|
As you see with these few tricks we would completely avoid a trace of the detour and yet still behave and function like the real NtOpenProcess.
However we can hook System Call stubs which means nothing in user mode can bypass nor remove the detour. There are two system call running in x86 before switching or entering kernel mode stubs in Windows:
- KiFastSystemCall (Located ntdll.dll)
- X86SwitchTo64BitMode (Located TEB+0xC0)
Nevertheless, it still does not matter what compiler you are depending on as the processes itself needs to posses the libraries in which your function derive from one way or another. I would be pretty skeptical with that as it not many processes which uses say,
MsRdc.dll, this means not all code can be run without the process being short of a single or more dynamic link libraries. Therefore it is not a viable method to do "instant" injections using GCC or MINGW.
As for the callback, I would say it would terribly fail leaving all processes crashed, as supposed to it being used in Device Drivers.
Hope I helped,
OrionMaster