Use std::string with Win32 API?

[WetCode]

Ok so i was trying to pass the buffer from ReadFIle() to a std::string like so.

	string sBuffer;
	sBuffer.reserve(fileSize);
	// Read fileName and fill cBuffer wit contetnts.
	if (ReadFile(hFileIn, &sBuffer, fileSize, &BytesRead, FALSE) == 0)
	{
		cout << "Read File failed!.." << endl;
		cout << "error code: " << GetLastError() << endl;
		CloseHandle(hFileIn);
		cin.get();
		return 0;
	} else {
		cout << "[OK]\tReading file " << fileName << endl;
	}

I do not think ths is the correct method to use but do anyone have any idea on how to do this.?
EDIT: Btw this code gives me error code: 998

MSDN:
ERROR_NOACCESS

    998 (0x3E6)

    Invalid access to memory location.

And i guese its not wery hard to understand way by some of you senior programmers here, but if anyone have time to explain. I whould realy much aprichiate it.

Thanks.

[modoran]

It cannot be done like that with std::string. You can try this instead:

char* buf = new char[fileSize];
string sBuffer;
	
	// Read fileName and fill cBuffer wit contetnts.
	if (ReadFile(hFileIn, buf, fileSize, &BytesRead, FALSE) == 0)
	{
		cout << "Read File failed!.." << endl;
		cout << "error code: " << GetLastError() << endl;
		CloseHandle(hFileIn);
                            delete [] buf;
		cin.get();
		return 0;
	} else {
                           sBuffer.assign (buf, fileSize);
                           delete [] buf;
		cout << "[OK]\tReading file " << fileName << endl;
	}

Not related to your problem, from my knowledge ReadFile() not work with a buffer larger than 8192 bytes (I am not sure why and couldn't find any documentation about that, but it happens for me).

[andywestken]

Actually, it is possible to use a std::string with Windows API calls. But you have to resize them rather reserve them (see notes below for caveats!)

#define USING_WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <iostream>
#include <string>
using namespace std;

int main(void)
{
	std::string path(MAX_PATH, '\0'); // fill with null chars

	GetModuleFileName(NULL, &path[0], path.length());

	cout << "module path (before resize) = \"" << path << "\n" << endl;

	// this is needed to adjust the size to what it actually is,
	// so operator<<, etc. behave correctly (the string works
	// with what it knows the length to be, not the null term.)
	path.resize(strlen(&path[0]));

	cout << "module path = \"" << path << "\n" << endl;

	return 0;
}

module path (before resize) = "c:\test\test.exe

                                                   "
module path = "c\test\test.exe"

Note that:

- using std::strings like this is offically sanctioned (i.e. part of the standard) in C++11, as std::string's storage is now mandated to be contiguous, but not in C++03.

- while not previously part of the standard, all major implementations of std::string already use contiguous storage due to other requirements of the standard, so it was already a de facto standard (which C++11 picked up). See Herb Sutter's comments on this (link below.)

(It's certainly safe to use this previously non-standard approach with the std::string that ships with Visual C++. And if you're using C++11, it's totally fair game!)

- but usually, the C++ standard container of choice in this circumstance is std::vector<char>, as std::vector's storage has always been guaranteed to be contiguous, allowing you to safely use, e.g.

// The preferred way

std::vector<char> buf(fileSize);

// etc

ReadFile(hFileIn, &buf[o], buf.size(), &BytesRead, FALSE);

// etc 

.

Andy

PS See Herb Sutter's response to the question “What is with std::string?” on this page:
Cringe not: Vectors are guaranteed to be contiguous
http://herbsutter.com/2008/04/07/cringe-not-vectors-are-guaranteed-to-be-contiguous/

[andywestken]

@modoran

from my knowledge ReadFile() not work with a buffer larger than 8192 bytes

This is not true. There are restrictions on read size when using unbuffered i/o, but even then you can use multiples of the sector size.

There is this post which refers to the value 8192...

ReadFile Kernel32 Maximum Buffer Size
http://stackoverflow.com/questions/7646320/readfile-kernel32-maximum-buffer-size

But the OP is corrected in the answer:

There is no such limit. You are mistaken.

Clearly you are limited by address space and the requirement that the buffer is a contiguous block of virtual memory. On a 32 bit system each process can only address 2GB of virtual memory. What's more you will not be able to allocate a 2GB contiguous block of memory.

But these are general limitations. The ReadFile API will happily read into as big a buffer as your can allocate.

Andy

Example reading blocks of 10b from a movie file

#define USING_WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>
using namespace std;

void ReadFileTest()
{
    // A 37.7 MB movie
    const char filePathTest[] = "C:\\Test\\test.flv";

    HANDLE hFile = NULL;

    hFile = CreateFile(filePathTest,
                       GENERIC_READ,
                       0, // no sharing
                       NULL,  // default security
                       OPEN_EXISTING,
                       FILE_ATTRIBUTE_NORMAL,
                       NULL); // no template

    if (hFile == INVALID_HANDLE_VALUE) 
    {
        cout << "CreateFile failed\n";
        return;
    }

    const DWORD bufferSize = (10 * 1024 * 1024); // 10 MB
    std::vector<char> dataBuffer(bufferSize);
    DWORD dwBytesRead = 0;
    BOOL bRet = FALSE;

    cout << hex << showbase;

    do
    {
        bRet = ReadFile(hFile,
                        &dataBuffer[0],
                        dataBuffer.size(),
                        &dwBytesRead,
                        NULL); // no overlapped structure
                               // (needed when using overlapped
                               // (asynchronous) i/o)

        cout << "dwBytesRead = " << dwBytesRead << endl;
        if(dwBytesRead < dataBuffer.size())
            cout << "!buffer not fully filled!" << endl;
    }
    while(bRet &&  dwBytesRead > 0);

    if(!bRet) 
    {
        cout << "ReadFile failed\n";
    }

    CloseHandle(hFile);
}

dwBytesRead = 0xa00000
dwBytesRead = 0xa00000
dwBytesRead = 0xa00000
dwBytesRead = 0x7c4855
!buffer not fully filled!
dwBytesRead = 0
!buffer not fully filled!

[WetCode]

Wow thanks for all the great answers realy aprichiate it.
Am shure google can find me a answer for this follow up qustion.
But if i wanted to store part of a file in a buffers or more likly more files.

This whould be the correct way to do this?
std::vector <vector<char>>dataBuffer(int amountOfFiles);
and accessing a file buffer like so. dataBuffer[fileIndex][binaryIndex]?
and then i make space for the indevidual files using.

dataBuffer[fileIndex].resize(file1Size);
dataBuffer[fileIndex].resize(file2Size);

And passing it to ReadFIle like this?

ReadFile(hFile,
                        &dataBuffer[1][0],
                        dataBuffer[1].size(),
                        &dwBytesRead,
                        NULL);

Thanks again for great replys.

Edit: Or do i need to use resize at all?

[andywestken]

Looks about right.

Note the C++03 doesn't like >> in nested template declarations; you need a space between the >s

// C++03 and C++11 :-)
std::vector <vector<char> > dataBuffer(int amountOfFiles);

but C++11 would be happy enough with your version

// C++03 :-(  C++11 :-)
std::vector <vector<char>> dataBuffer(int amountOfFiles);

And yes, you definitely do need to resize before use.

Andy

[WetCode]

I did not know that i think its time to update to VC++ 2012.
Am stile using 2010 and if i remember correctly VC++ 2010 dont have fully implimented the C++11 Standard.
Thanks again for a great reply.

Cheers

[andywestken]

Actually, as some has pointed out to me on another thread, VC++ have been able to deal with >> since VC++ 2005.
http://www.cplusplus.com/forum/beginner/102670/#msg552881

But it was non-standard behaviour pre-C++11

Andy