Different outputs using URLDownloadToFile

[am250478]

Hi,
first time here, new programmer.
I am trying to download online html pages and then search after specific strings.
I implemented code with URLDownloadToFile function that did good job, then I found that on specific internet sites I get wrong output.
Instead of full html page, I get only header html file with meta data, style, color, display, width and directions to different browsers.
If I try "view page source" on the browser, I get the full html page.
It's https site, maybe implemented by js.
Please help.
Thank you.

[salem c]

1. Get wireshark -> https://www.wireshark.org/
2. Get Firefox or Chrome browser, if you're not already using one of these.
3. Read this -> https://wiki.wireshark.org/TLS#Using_the_.28Pre.29-Master-Secret
4. Open a browser with SSLKEYLOGFILE environment variable set, and point wireshark to the same file as directed in 3.
5. Start a wireshark trace.
6. Visit website in the browser you started in step 4.

Now you can follow exactly what your browser does when visiting https://www.cplusplus.com

If you use this knowledge to scrape a site contrary to it's terms of service, that's YOUR PROBLEM not mine.

[kbw]

It's best to use an external downloader (like curl or wget) rather than trying to track varying web (non) standards in a C/C++ program. Then you can post-process the downloaded content.