Passing LPTSTR Parameter to DLL gives Access Violation in C++ project

[pollux33]

I'm calling a function in an undocumented DLL which unpacks a file.

There must be a mistake in the way the header is declared / allocated but can't figure out what is going wrong.

Project character set in VS 2010 is Unicode.

Can call the DLL function succesfully from C# with the snippet below (but I need to make it work in c++):

[DllImport("unpacker.dll", EntryPoint = "UnpackFile", PreserveSig = false)]
internal static extern IntPtr UnpackFile(byte[] file, int fileSize,
[MarshalAs(UnmanagedType.LPWStr)] StringBuilder header, int headerSize);

If I uncomment of move one the headers an Acccess Violation pops up. The function also returns 0 which it doesn't in C#.

Any thoughts?

Code in the VC++ 2010 project:

// unpacker.cpp : Defines the entry point for the console application.

#include "stdafx.h"
#include <windows.h> 
#include <fstream>
#include <iostream>

using namespace std;

typedef void* (__stdcall *UnpackFile)(unsigned char*, int, LPTSTR, int);

int _tmain(int argc, _TCHAR* argv[])
{
    LPTSTR header; 
    //Move this line and get a access violation on the _UnpackFile(filetounpack... line

    static unsigned char *filetounpack; //Buffer to byte array with the file to unpack
    int filelen; //variable to store the length of the file
    HINSTANCE dllHandle; // Handle to DLL
    UnpackFile _UnpackFile; // Function pointer
    ifstream filetoread; //Stream class to read from files
    static LPTSTR header2;  //Buffer for the header 2nd


    filetoread.open ("c:/projects/testfile.bin", ios::in | ios::binary|ios::ate); 
    filelen = filetoread.tellg(); //read the length
    filetounpack = new unsigned char [filelen]; //allocate space

    filetoread.seekg (0, ios::beg); //set beginning
    filetoread.read ((char *)filetounpack, filelen); //read the file into the buffer
    filetoread.close(); //close the file

    dllHandle  = LoadLibrary(_T("unpacker.dll"));

	if (dllHandle)
	{
		_UnpackFile = (UnpackFile)GetProcAddress(dllHandle, "UnpackFile");

		if(_UnpackFile)
		{
			//header = new _TCHAR[filelen]; //Allocate memory for header 
			header2 = new _TCHAR[filelen]; //Allocate memory for header

			void* tmp = _UnpackFile(filetounpack ,filelen ,header2 ,filelen);

			delete[] filetounpack;
			delete[] header;
			delete[] header2;
		}
		else
		{
			std::cout << "Could not retrieve address function!" << std::endl;
		}


		FreeLibrary(dllHandle); 
	}
	else
	{
		std::cout << "DLL Failed To Load!" << std::endl;
	}

    return 0;

}

[pollux33]

Bump! No gurus?

[kbw]

You're passing an LPSTR that points to null instead of a StringBuilder in parameter 3.