Interacting with Websites

[mackabee]

Could anyone point me to some references or tutorials that deal with interacting with websites? For instance, writing a program to click a button or interact with some sort of control.

What libraries are available? Is this just some basic javascript interaction? I just need a general idea of what to learn to accomplish such tasks.

Thanks!

[Denis]

Do you want to write c++ program that click different controls on a web site?

[mackabee]

Yes, but without opening a browser or controlling mouse and keyboard movements. Say for instance a webpage has a textbox and a submit button. I want to write a c++ program which simulates filling the textbox and pressing the button.

I'm just wondering if there is some sort of library that already does the work for me. I'd rather not have to write a sockets application that initiates a connection with the web server and manually have to craft the packets by hand.

[Incubbus]

In other words... You want to spare yourself the fun^^...

[S6k9GNh0]

It depends on each website. For instance, take this website for an example. When you press the Submit button, what actually happens? If you look at the source you'll see.... erm... a lot of javascript I don't understand since I don't do well with website scripting and I don't feel like tracing it to its actual workings. any site that submits data, you'll find that its just sending a packet of some kind and you can actually find these packets in a packet analyzer. In a PHP based site, you can literally send data to the remote file and it will return data that you can physically read as HTML or however. Try it out using CuRL or something.

[mackabee]

I might just write my own. I just got a little information and it looks as simple as creating a connection and messing with POST and GET.

[S6k9GNh0]

It is in most cases.

[moorecm]

You mean a CGI in C++? If so, it can be done, but good luck finding a host that will let you run it...

[S6k9GNh0]

A personal server will. Some providers allow you to give them source and to compile it themselves if its small.

[mackabee]

Thanks moorecm! I have always seen "CGI" pop up from time to time but never took the time to really look it up and understand it. That's exactly what I need to learn =).

I plan on just scheduling the program to run on my own computer every couple of hours.

[mackabee]

So I can write my C++ application using CGI that will interact with any other server?

This line from the RFC is what's confusing me:

"The Common Gateway Interface (CGI) [22] allows an HTTP [1], [4]
server and a CGI script to share responsibility for responding to
client requests."

I don't want to run a webserver. I am the client.

[Galik]

You might want to look at Curl: http://curl.haxx.se/

That could be used to make GET and POST requests etc... but javascript is another matter.

I have no idea how you might do that. Maybe check out the mozilla embedded interface?

http://www.mozilla.org/projects/embedding/

That might have something you could use.