How to send real time screenshare over tcp windows socket?

[LukeProducts]

Hello all,

i just wondered how i could screenshare over a tcp socket connection.
I want the client to send real time pictures to the server, which then displays it to a new window. I do not know how to get real time screenshots and how to manage them to be sent in real time and also no know how to create a window serversided displaying the screens real time result.
I do know how to take one snapshot from the clients computer, but it isnt fast at all.

I cannot use Libraries like OpenVC, because i need the client to be small sized and also always fail on installing OpenVC, as i use mingw64 and i dont know how to add libraries to that except to put the headers in the mingw folder manually.

Can anyone help me with some code snippets or so?

greetings and thanks in advance

Luke

[jonnin]

for real time, you don't want to send images, you want to send a video stream. The reason is that images are big, even when compressed, while video has a temporal compression element that is aware of what parts of the screen changed and can only send the changes or occasional full frame to catch up here and there. You can mimic that with your images, I suppose, but its a home grown approach to what is already done in video streaming. You need a real time stream, not a massively buffered one, though.

It sounds like you need to stop what you are doing and figure out how to use a library first. This is not going to happen in real time without a library -- does not have to be CV, but you want something even if its just mpeg or h264 type encoder. You likely need libraries all around, from networking to graphics card interface to video or image compression/handling to image display.

to get the screen fast you grab it from the graphics card directly, don't use a 'take one screenshot' tool or command. If the screen is running a very high resolution, you can halve or quarter it with a quality resize in both directions to improve speed at the cost of image quality.

Don't be afraid to drop frames if you fall behind your timer or to adapt the FPS to meet it.

this sounds a little XYish. What exactly are you trying to do that requires real time instead of near real time or even normal, laggy screenshare like everyone else uses for such things?

[LukeProducts]

Hi jonnin,

thank you very much for your reply. I don't care about a few laggs, by real time I meant that the transfer of a frame takes preferably not 10 seconds or so :). A real time stream sounds exciting.
What exactly do you mean by that?
Do I have to change how I send data, e.g. replace the recv() or send() function, or do you mean a specific handling of the received and sent data of the two network participants?
If i have to rely on a library here, which one would that be exactly and how do i use it?

Thank you jonnin

Luke

[jonnin]

by stream, I meant using a video format instead of sending one image at a time. Like how you get spammed by video attacks when you go to most web sites, those videos are not sent one picture at a time but are compressed as a video stream.

this part is about the data, not the transmission. No matter what you do, you will send it as packets over a socket, though your video streaming library may or may not do that for you.

I do not know the best streaming library to use right now. The last time I did this was probably 2008... and I can't recall what it was. It was easy enough to use that I got the data from camera to remote computer in just a couple of days, so whatever they have now is surely really easy to use and get set up. You won't have a camera, so you need that screen capture instead.

[kigar64551]

It sounds like you want to re-invent VNC 😏

https://en.wikipedia.org/wiki/Virtual_Network_Computing

https://github.com/ultravnc/UltraVNC


Really, this would be a huge project to develop from the scratch. You'd need at least:

1. Figure out how to capture the screen in real-time

2. Compress the captured frames, using some sort of inter-frame compression, to get the data down to a manageable size

3. Define a network-protocol that your client and sever will use to communicate (should include authentication+encryption)

4. Actually implement and test all of the above