Hi,
The callback is for NtTerminateProcess and not for OpenProcess, therefore it is fair to say the callback should have the parameters of NtTerminateProcess. That being said it is extremely important you copy exactly the same parameters of NtTerminateProcess in terms of data types.
The Callback would not work as the Callback looks like OpenProcess and
NOT like NtTerminateProcess. Furthermore, unless you are planning on making this hook local, which I see no point from security perspective,
GetCurrentProcessId() will never work but crash the process it is injected in as VS uses call gates instead of direct CALL's into the address of the function.
Then the code snippet:
Would not be the correct way to inform the user that the termination attempt failed, you must return
STATUS_ACCESS_DENIED rather than a
false
. As if the function really failed you would expect it to give a suitable error value not 1 or 0 so all you are doing is messing up the developers time and making them annoyed either remove your AV.
There is a lot of work to be dones here as the callback mechanism is all wrong and the entire indentation is wrong as you could have just done:
if(pid==GetCurrentProcessId()) return false;
Next, you need to get used to using assembly routines rather than using large C\C++ code here especially with handling System Calls. The reason it is easier to use Assembly is because you need to assign values to the callback and to return you need to patch in the return address.
When handling callbacks you need to be 100% sure that you don't use any functions as the entire address space changes and trying to access functions from 3rd party processes will lead to a process collapse due to Access Violation.
I also suggest you delete the thread on Rohitab about the terminating AV processes as it is making me think you are blackhat and if I see thread like that which are influencing black hats I sorry to say I cannot help.
GL