Not sure where to start, so I figured I'd start here. Basically I'm looking for a program that allows real time effects/processing of a video file being outputted to a screen. I want to control effects like distortion, filters, etc with an Arduino unit with various sensors
You can still find free lib on the net , but you must understand that this kind of software is not given for free . It generates millions to cies for developping special effects . You will have to pay for a decent one , but I believe that there is hidden gems on the net which can do simple task for montage or editing .
No no, the Arduino is piping data into a PC that will be sending a video feed to a projector. I wanted to add distortion effects and such to the video in real time. I'm wondering if maybe this could somehow be achieved in openGL perhaps utilizing the Three.js library?
Yes, as to the feasibility of doing it in real time. I also forgot about the frame rate.
It's not the same to process 1080p @ 60 Hz (373 MiB/s) than QVGA @ 23 Hz (5 MiB/s).
A modern GPU can apply the fragment shaders quickly enough, but the problem is getting the pixels to video memory quickly enough. Its even harder if the pixels have to be reformatted in CPU before being sent.
@helios
That amount of data is trivial for modern GPUs :)
I'd be worried about the throughput of the Arduino connection with the PC if anything.
To expand on helios' last sentence, I believes he recommends storing it in a format that makes the driver essentially do a memcpy of your texture data straight to VRAM (This is typically BGRA or RGBA).