MJPEG Streaming

[ziggyspaceman]

Hi
I am looking to add MJPEG streaming capabilities to a project, however I've run into a problem & can't seem to get MJPEG streaming code below to work properly.

I was just wondering if there's any one here knows how to do this sort of thing?, basically when I run code below it waits for a connection on port 1234, when I connect using VideoLan(VLC) it connects OK exchanges headers but when it comes to getting JPEG image data it simply does nothing except “Error: socket error while sending image data”

1 2 3 4 5 6 7 8 9

Server response

Initializing winsock... initialized. Creating socket... created. Binding socket... bound. Putting socket in listening mode... done. Waiting for incoming connection... accepted. Connected with 192.168.0.3:49806. Error: socket error while sending image data Cleaning up winsock... done.

1 2

VideoLan(VLC)

Your input can't be opened: VLC is unable to open the MRL 'http://192.168.0.3:1234'.

Any help would be greatly appreciated.
Regards
Steve

#include <iostream>
#include <string>
#include <sstream>
#include <fstream>

#define WIN32_MEAN_AND_LEAN
#include <winsock2.h>
#include <windows.h>

using namespace std;

class ROTException
{
public:
    ROTException() :
         m_pMessage("") {}
    virtual ~ROTException() {}
    ROTException(const char *pMessage) :
         m_pMessage(pMessage) {}
    const char * what() { return m_pMessage; }
private:
    const char *m_pMessage;
};
//---
int length;
char * buffer;

const char HEAD_RESPONSE[] =
{
    "HTTP Code: 200 OK\r\n"
    "Content-Type: multipart/x-mixed-replace;boundary=myboundary\r\n"
	"--myboundary\r\n"
};

const char HEAD_RESPONSE_PER_IMAGE[] =
{
	"Content-Type: image/jpeg\r\n"
	"Content-Length: 69214\r\n\r\n"

};

const char SEPARATOR[] =
{
    "\r\n"
	"--myboundary\r\n"
};
//---

const int  REQ_WINSOCK_VER   = 2;	// Minimum winsock version required
const int  DEFAULT_PORT      = 1234;// Listening port
const int  TEMP_BUFFER_SIZE  = 128;

string GetHostDescription(const sockaddr_in &sockAddr)
{
	ostringstream stream;
	stream << inet_ntoa(sockAddr.sin_addr) << ":" << ntohs(sockAddr.sin_port);
	return stream.str();
}

void SetServerSockAddr(sockaddr_in *pSockAddr, int portNumber)
{
	// Set family, port and find IP
	pSockAddr->sin_family = AF_INET;
	pSockAddr->sin_port = htons(portNumber);
	pSockAddr->sin_addr.S_un.S_addr = INADDR_ANY;
}


void HandleConnection(SOCKET hClientSocket, const sockaddr_in &sockAddr)
{
	// Print description (IP:port) of connected client
	cout << "Connected with " << GetHostDescription(sockAddr) << ".\n";

	char tempBuffer[TEMP_BUFFER_SIZE];

	// Read data
	while(true)
	{
		int retval;
		retval = recv(hClientSocket, tempBuffer, sizeof(tempBuffer), 0);
		if (retval==0)
		{
			break; // Connection has been closed
		}
		else if (retval==SOCKET_ERROR)
		{
			throw ROTException("socket error while receiving.");
		}
		else
		{
		    /////////////////////////////////////////////////////
		    //
		    //This is the part that sends response back to client
		    //
		    /////////////////////////////////////////////////////
            //Send header to client
            //---------------------
            /*
            HTTP Code: 200 OK
            Content-Type: multipart/x-mixed-replace; boundary=myboundary
            --myboundary
            */
			if (send(hClientSocket, HEAD_RESPONSE,sizeof(HEAD_RESPONSE), 0)==SOCKET_ERROR)
				throw ROTException("socket error while sending header");


            //Main loop
            //---------
            while(1){

                /*
                Read image data, just used a single image
                to keep things simple [Binary format]
                */
                ifstream is;
                is.open ("lena.jpg", ios::binary);
                // get length of file:
                is.seekg (0, ios::end);
                length = is.tellg();
                is.seekg (0, ios::beg);
                // allocate memory:
                buffer = new char [length];
                // read data as a block:
                is.read (buffer,length);


                //Send per image header to client
                //-------------------------------
                /*
                Content-Type: image/jpeg
                */
                if (send(hClientSocket, HEAD_RESPONSE_PER_IMAGE,sizeof(HEAD_RESPONSE_PER_IMAGE), 0)==SOCKET_ERROR)
                throw ROTException("socket error while sending header per image");
                //Sleep(500);
                //Send image data
                //---------------
                if (send(hClientSocket, buffer, length, 0)==SOCKET_ERROR)
				throw ROTException("socket error while sending image data");
                //Sleep(500);
                //Add a separator after image data
                //--------------------------------
                /*
                --myboundary
                */
                if (send(hClientSocket, SEPARATOR,sizeof(SEPARATOR), 0)==SOCKET_ERROR)
				throw ROTException("socket error while sending separator");

                //Close img file
                //--------------
                is.close();
                Sleep(500);
            //////////////////////////////////////////////////////
		    //end of loop
		    //////////////////////////////////////////////////////
            }

		}
	}
	cout << "Connection closed.\n";
}

bool RunServer(int portNumber)
{
	SOCKET 		hSocket = INVALID_SOCKET,
				hClientSocket = INVALID_SOCKET;
	bool		bSuccess = true;
	sockaddr_in	sockAddr = {0};

	try
	{
		// Create socket
		cout << "Creating socket... ";
		if ((hSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == INVALID_SOCKET)
			throw ROTException("could not create socket.");
		cout << "created.\n";

		// Bind socket
		cout << "Binding socket... ";
		SetServerSockAddr(&sockAddr, portNumber);
		if (bind(hSocket, reinterpret_cast<sockaddr*>(&sockAddr), sizeof(sockAddr))!=0)
			throw ROTException("could not bind socket.");
		cout << "bound.\n";

		// Put socket in listening mode
		cout << "Putting socket in listening mode... ";
		if (listen(hSocket, SOMAXCONN)!=0)
			throw ROTException("could not put socket in listening mode.");
		cout << "done.\n";

		// Wait for connection
		cout << "Waiting for incoming connection... ";

		sockaddr_in clientSockAddr;
		int			clientSockSize = sizeof(clientSockAddr);

		// Accept connection:
		hClientSocket = accept(hSocket,
						 reinterpret_cast<sockaddr*>(&clientSockAddr),
						 &clientSockSize);

		// Check if accept succeeded
		if (hClientSocket==INVALID_SOCKET)
			throw ROTException("accept function failed.");
		cout << "accepted.\n";

		// Wait for and accept a connection:
		HandleConnection(hClientSocket, clientSockAddr);

	}
	catch(ROTException e)
	{
		cerr << "\nError: " << e.what() << endl;
		bSuccess = false;
	}

	if (hSocket!=INVALID_SOCKET)
		closesocket(hSocket);

	if (hClientSocket!=INVALID_SOCKET)
		closesocket(hClientSocket);

	return bSuccess;
}

int main(int argc, char* argv[])
{
	int iRet = 1;
	WSADATA wsaData;

	cout << "Initializing winsock... ";

	if (WSAStartup(MAKEWORD(REQ_WINSOCK_VER,0), &wsaData)==0)
	{
		// Check if major version is at least REQ_WINSOCK_VER
		if (LOBYTE(wsaData.wVersion) >= REQ_WINSOCK_VER)
		{
			cout << "initialized.\n";

			int port = DEFAULT_PORT;
			if (argc > 1)
				port = atoi(argv[1]);
			iRet = !RunServer(port);
		}
		else
		{
			cerr << "required version not supported!";
		}

		cout << "Cleaning up winsock... ";

		// Cleanup winsock
		if (WSACleanup()!=0)
		{
			cerr << "cleanup failed!\n";
			iRet = 1;
		}
		cout << "done.\n";
	}
	else
	{
		cerr << "startup failed!\n";
	}
	return iRet;
}

[kbw]

Please use the code formatting tag for your code, it really helps.

I don't know the protocol, and the code looks mostly ok but ...

Code like:
send(hClientSocket, HEAD_RESPONSE,sizeof(HEAD_RESPONSE), 0)
sends the terminating null too. You probably should just send:
send(hClientSocket, HEAD_RESPONSE,sizeof(HEAD_RESPONSE)-1, 0) or
send(hClientSocket, HEAD_RESPONSE,strlen(HEAD_RESPONSE), 0)

HandleConnection should close the socket when it's done. If it did that, it'd be completely self contained and could be spun of as a thread in a future tweaked implementation.

[ziggyspaceman]

Thanks for the reply, tried those code changes but still not working.

Basically I am trying to do what an IP camera does when a browser connects to it & download MJPEG stream, browser sends a GET request to server, the server then responds with

HTTP Code: 200 OK
Content-Type: multipart/x-mixed-replace; boundary=myboundary

Which basically tells the browser to keep connection open & that it going to send a series of jpeg images within “--myboundary” tags, now it only needs to send the above headers once since the browser keeps connection open. It simply keep's sending image data within tags like so...

--myboundary
Content-Type: image/jpeg
Content-Length: 67856

<binary JPEG image data>
--myboundary
Content-Type: image/jpeg
Content-Length: 52856

<binary JPEG Image data>
--myboundary 
…

The problem for me is that I've done very little socket programming in C++ so i'm not sure how to do the above correctly.

The coding above seems to work when I connect using a client software written in c++ and all the data sent to the client seems OK, but when it come to VideoLan media player it can connect to server OK but then does nothing except errors on both sides.

“stuck”
Steve:(

[kbw]

I still don't have the answer, but have a few more comments.

I recomend using wireshark to view the network communication. You can see what the working connection does and compare that with the one that doesn't work. Initially, you'll see everything, but if use use a capture filter on port 1234 as "port 1234", you'll only see traffic using that port. Time spent learning how to use wireshark isn't wasted time.

It's cheaper to get the file size using stat. It keeps your file access code clean and is more efficient because it uses file metadata rather than accessing the file itself.

As you're attempting to send the whole file at once, you should consider that send might not send all you've given it. It actually returns the number of bytes sent, so you need to check that and loop around sending the remainder.
e.g.

    for (int offset = 0; offset < bufsz; )
    {
        int nbytes = send(s, buf + offset, bufsz - offset, 0);
        if (nbytes == -1)
            throw "send failed";
        offset += nbytes;
    }

[ziggyspaceman]

WireShark sure I'll give it a try.

Anyhow tried just sending header nothing else & tried connecting from web browser

HTTP Code: 200 OK\r\n
Content-Type: multipart/x-mixed-replace;boundary=myboundary\r\n

did what it should do by keeping connection open so that's good.

Next let it send the above header along with the next set of headers, but browser didn’t like it

Error 325 (net::ERR_RESPONSE_HEADERS_TOO_BIG): Unknown error.

I must admit I didn't write the code myself ,copied it from a tutorial & adapted it to suit my needs, so I don't myself have any idea what the problem could be.

Thanks very much for your input, I would very much like to solve this problem basically there does n't seem to be any coding examples out there to do this, well none I can find anyhow.

Best Regards
Steve