Ways to take screenshots and analyze them through program?

[brokenbot]

Hey, I'm looking for a way that I can make a program that takes screenshots of a screen analyzes them and makes decisions off that. Or On the fly analyzing. I'm not sure which is better/faster etc. Is there any libraries that cater to this. Do I have to make my own etc? Thanks