crawler

[ashish verma]

have anyone tried making a crawler in c++?
if he/she can help me

[guestgulkan]

I'm sure there is no end of example code out there on the web.
Google is your friend.

[modoran]

Look at wget source code, this already do it.

[Galik]

I've written a crawler in C++. I used libcurl to download the web-pages and boost::regex to extract the links and paging information.

cURL & libcurl
http://curl.haxx.se/

boost::regex
http://www.boost.org/doc/libs/1_46_1/libs/regex/doc/html/index.html

[S6k9GNh0]

Do not make multiple threads of the same topic.