"Hooking" means that you
replace a certain function with your own implementation, so that whenever the hooked function gets called
anywhere in the program your function gets invoked instead of the original.
There exists libraries for that:
https://ntcore.com/files/nthookengine.htm
https://github.com/microsoft/detours/wiki
...usually you do this inside of your own process though!
Executing some code in the context of
another process is possible via "DLL injection":
https://resources.infosecinstitute.com/topic/using-createremotethread-for-dll-injection-on-windows/
In a nutshell, you use
CreateRemoteThread()
to start a new thread within the context of the
other process. And then you let that new thread invoke
LoadLibraryA()
, in order to load (or "inject")
your DLL into the
other process. This finally gives you the chance to execute your own code, i.e. the code in your DLL, within the context of the
other process – because the DLL's
DllMain()
function will be executed when it gets loaded.
This code then may install a hook, or do whatever you like...