Question on Intel Hex Records...

[ceruleus]

Hello C++ Forums.

I come to you today with a question about Intel Hex Records. I am currently refactoring some old code that sketchily reads in some Intel Hex Records for the purpose of sending data over the air to devices with a wireless stack to upgrade their firmware. As I started to go through this horribly ugly code, I started thinking there must be some library that does this well and is tested.

In case you don't know what an Intel Hex Record is -
http://en.wikipedia.org/wiki/Intel_HEX
http://www.lucidtechnologies.info/intel.htm

So I googled around for a bit and I couldn't find any libraries or sample code that does this stuff well. I'm leaving now and I'm going to rewrite this thing in the morning unless one of you guys can help me out!

I can give you some sample code if you really want but what I've got is pretty useless or else I'd post it.

Thanks!

**Here's hoping for a Disch and/or Duoas reponse!**

EDIT - Forgot to mention sscanf is out of the question! The main reason I'm doing this refactor is because to run this code using sscanf to parse the data of each record of a file that is about 400k in size takes a total of about 6 seconds. I am 100% sure that it is the sscanf slowing things down, and the time requirements to parse a file around 400k is just a few miliseconds.

[cire]

sscanf is likely faster than the C++ equivalent. Have you actually profiled the code?

[ceruleus]

@cire

I have not profiled the code legitimately. In fact I have never done that and I'm very interested in doing it. Can you point me to a nice tool that does profiling?

I have, however, used QueryPerformanceCounter on windows to do the timing myself. It is definitely sscanf that's taking forever, which makes sense. I just need to tear through a bytestream and convert the ascii to hex.

[S G H]

I know nothing bout Intel Hex Records but let me remember you all what sscanf can do, you can with your own hand-made code (and thats gonna be also faster)

[cire]

http://developer.amd.com/tools/heterogeneous-computing/amd-codeanalyst-performance-analyzer/

is a reasonable choice.

Yes, one would expect string manipulation to take the most time in a program that's doing string manipulation almost exclusively. I guess the question is whether or not you think you can improve on sscanf. Coding up a simple (representative) test with an alternative implementation and timing it -vs- a sscanf implementation should give you an idea of any performance gains you might get and tell you whether it's worth refactoring the program.

[S G H]

Well I think I already did my best against std::string on my own library, my own string class outperforms that - Calculated doing 1mln iterations on the same equal operation for both classes.
Also my own string class can interoperate with other type of my own strings (Wide to Ascii, to 32-bit chars, to 64-bit chars if supported by the compiler, also a custom struct can be used thanks to C++'s Templates Power. You interested? Check out the SGHLib on Sourceforge, you should be able to find a link on my user page on this forum.

[ceruleus]

@cire - I think you may be misunderstanding my question. Not your fault as I am probably being very unclear as to what I'm trying to accomplish here, and the fact that I haven't provided any code isn't helping anything...

So, to be clear, I have determined that sscanf is doing it's job too slowly. All I need sscanf to do is take in some buffer, read an ASCII value and convert it to hex for me. So I wrote a quicker conversion function (but it's shitty).

inline signed short ConvertAsciiHexCharToInt( char byAsciiChar )
{
	// 0 - 9
	// A - F
	// a - f
	char byRetValue = -1;

	if( byAsciiChar >= '0' && byAsciiChar <= '9' )
	{
		byRetValue = byAsciiChar - '0';
	}
	else if( byAsciiChar >= 'A' && byAsciiChar <= 'F' )
	{
		byRetValue = byAsciiChar - 'A' + 0x0A;
	}
	else if( byAsciiChar >= 'a' && byAsciiChar <= 'f' )
	{
		byRetValue = byAsciiChar - 'a' + 0x0A;
	}

	return byRetValue;
}

inline signed short ConvertAsciiHexToInt( char* pbyBuffer )
{
	signed short wRetValue = -1;

	signed short wUpper = ConvertAsciiHexCharToInt( pbyBuffer[0] );
	signed short wLower = ConvertAsciiHexCharToInt( pbyBuffer[1] );

	// This checks if either sign bit is set (negative) most efficiently...
	if( wUpper | wLower > 0) 
	{
		// MAKEWORD is a macro that will shift these suckers over to high byte and low byte
		wRetValue = MAKEWORD( wUpper , wLower );
	}

	return wRetValue;
}

Now, the code I'm trying to refactor would do something like this previously:

unsigned int dwRecordLength = 0;
sscanf( (const char*)( pbyBuffer + dwPos ), %02X, &dwRecordLength );

I'm trying to replace that with this:

// Get the byte count
char byByteCount = ConvertAsciiHexToInt( pbyBuffer+dwPos );

Remember these are just code snippets, there is a total of about 3 or 4 of these calls all siting in a while( not done parsing file ) loop. For a file about 350KB the first code would take about 6 seconds, with my improvements that's down to a fraction of a second, but still somewhat significant (about 100-350ms or so...)

Okay now hopefully you see why I don't want to use sscanf.

NOW BACK TO MY ORIGINAL QUESTION!!!!
I need to parse Intel Hex Records, that's what these files are
http://en.wikipedia.org/wiki/Intel_HEX
http://www.lucidtechnologies.info/intel.htm

Does anyone know of any good libraries for parsing this stuff? Anyone ever done this before and have any suggestions/tips?

[ceruleus]

@EssGeEich

Thanks! I'll check out your custom string stuff, always interested to see what other people come up with!

[S G H]

Oh, if you don't know how to use them, at the beginning of your main.cpp or wherever you include your files, FIRST define H_FSTR, THEN include SGHLib.h.
You can use StreamA for Ascii, or StreamW for Unicode. If you want another type to be used, use StreamT< (Type) >. It is "Stream" for personal historical reasons, but it is a string. Feel free to check out other things.

[ceruleus]

If anyone is still interested in this, I just realized that instead of using those inline functions I posted I could just make a look up table and use that to do the conversion.

My original question still stands though! Is there a nice, tested library for reading intel hex records in c++?

[cire]

A couple things regarding your code: You shouldn't assume that char is signed or unsigned.

	// This checks if either sign bit is set (negative) most efficiently...
	if( wUpper | wLower > 0) 

Presumably that should be if ( (wUpper | wLower) >= 0 ). > and >= have a higher precedence than |, and 0 seems like it should be a valid value.

[ceruleus]

@cire Thanks for reminding me! The fact that char is not guaranteed to be signed OR unsigned has always been a great excuse to ALWAYS use the signed or unsigned keywords.