Reading file into structre

I am trying to read in a file with the follwing data structure:

typedef struct tld_msg
{
char* header;
uint16_t type;
uint16_t length;
uint8_t data[1];
} tld_msg_t;

The data structure in the file is organized as follows:

header: An ASCII delimiter string of the following form: MSG:XXXXXX\n
type: A 2-byte binary type.
length: A 2-byte binary length, giving number bytes in data packet
N-bytes of data with a possbile '\o' termination
2-byte binary checksum


I do not know how to extract the data out so I can read and use it. Can anyone help?


I have written the following, and can stream the code into a character array.

#include <iostream>
#include <fstream>
#include <cstdlib>
#include "stdint.h"

using namespace std;

ifstream:os_type size;
char * memblock;

typedef struct tld_msg
{
char* header;
uint16_t type;
uint16_t length;
uint8_t data[1];
} tld_msg_t;

int main()
{
tld_msg dataMessage;

ifstream file("test.ips", ios::in | ios:: binary | ios::ate);
if(file.is_open())
{
size = file.tellg();
memblock = new char[size];
file.seekg(0,ios::beg);

file.read(memblock,size);

for(int i =0; i<=size; i++)
{
cout << memblock[i];
}

}

file.close();
cout << "file content is in memory";
delete[]memblock;

}
What's the file format?
It is in something called an IPS file. First packet of data looks like this:


MSG:S

c���e7"@������)@���?��ޅ��r�0�!@�����(@:�-^�?����@��m�!@�˄2-(@Z�v�e�?ֶ� @\��!@�*��h(@&F�K��?�׭T�#]$�!@�]��(@���~�?���H�d}Ѹ�!@�%��(@b%�z�?c]R��w�[�7"@�'�s�)@�F�$��?��Ƌ@ �p"�!@<���*(@�m��t�?���[�Ӿ�c���5��
["@�^�������gz�Rl�@O>`�e"@�I'@no��'�?���m��֖O�@�3�Έ{*@(�ٺ��?���UB5
f��?��'(iH#@��
%�������]�U�4�?�`���!@�_yN�������a0�N�@E.D��"@�ꋌ���������>��@�*!b�3"@��e[�����
�'��?����&Q$@b��smks �9���?��;�[%@��
%�������_����]@�К�c�%@���`�?���_��E)�?��р��#@�^��𿥲�o��pi¿�Ab��J"@@Gu�s�𿥶�c��=�@���s%@@�[}����m���&@�_�O��!@�ꋌ�𿜮�u��}�@����i"@�^�������m,���@�D����#@�x����{}�w2�ڡ�?���+��#@�ꋌ������]|�VP��?�v���{#@@���������y���\`�?�k�4%@9HF֣�����cD�J���z��&@�^�������{�5���@�̻#@�΋�B�����֪u��z]@�����-"@��e[���ުH�W�@�l�i
�"@@Gu�s��������nv���@���˄�%@�x�[��@�pI��*@���؟�?����+���w@P#@@������������K��1ÿ�P���[#@@Gu�s������]��
�t�#@��J������ރp=,8��?��O?��#@9HF֣�����_�[��w@�tҳ"@@Gu�s�������`e��ķ?l���$@@~ϳ����Ξb��B��?�9��#@@�����𿥮���v��]�?�N�}�"@�e�b��������ޤ�0@�����#@��P
������{ch���@Rk�(�*@�B����{}�@)�o�?�)�ȡ%@@~ϳ�����oN ��y@/�i�6#@@���������w�.�a�@�У�#@�ߋH7ѿ����Z~%J}�?�R
�x"@9HF֣�����w-��Z@���
�&"@�|Ӳ����������w���?��q�$@9HF֣�����_9�
z��su�qE@2���&ذ��!@@���������c8��ʥݿ�<Z8�"@���;�~𿵾�Y�y���@�9���M"@�΋�B������q%�G#@��
%�������pp�s�?.j@�#@�|Ӳ�������]�ต�@�0���!@��e[����G�r$@���f�"@9HF֣������д�ܕ�?����E�%@�_yN�������[�]x�V�?���?��#@���?��������
@��]���*@&B�Xƿ{}�{�xX@��JRm�*@�8S&�?���[DM��b�?5,t��#@�|Ӳ��𿥪���~B}��?�{�(��$@�}!伟濜uk{��O<��?U��J&@�x�������y�Z�'�?�Jj�"@�ꋌ������M2�!�@�207l*@H�&]h�?���@��X'I*@@~ϳ𿜲�s^'A@�H�;�t#@@����������� y+>@r:I:�$@U�.J7�����\]��@J[�r�*@� �Gw������{��?�<š��#@@����������u�wUF'@��A?S%@2�4��࿜��m^��ܩ�?������"@��
%�������a��*v�?�n�#S"@@Gu�s�����ִ�'�e�忀��5��&@@Gu�s������]]".�@��w��&@�)
�)�?���\���@��m�l�$@�c���&῔����V�|�@������#@�L�㔔���h����@^�C�]"@�ꋌ�������
�r���?j��+�$@�p�Q忔b&��
@u���*@��~�x���� tj���?��M�$@'������.8T�u@���
u%@�K]ş�?����`]{���?���}��"@9HF֣�����[�9J��?��,$@��[z�>�����Uh�5]��?���p
"@�x�������[�


Not sure how to handle this.
You haven't provided enough information to help you. I have no idea what the file format is.
I am still having problems with this. Here is a description of the file:

A TLD packet contains an ASCII type string, a binary type field, a binary length field, binary data, and a binary checksum at the end.

Coordinates are specified using the WGS84 ellipsoid unless otherwise indicated.
General Packet Structure

For any multi-byte binary quantities (both integers and floating point values), all bytes shall be in little-endian order.

In the case of floating point values, if you are on a machine that does not have little-endian byte ordering, you must swap the bytes before serializing the binary value. Similarly if you are parsing a packet on a non-little-endian machine you must first swap the bytes before considering the floating point value. A TLD packet must contain the following elements:

1.

An ASCII delimiter string of the following form:

MSG:XXXXXX\n

Where the "XXXXXX" should be replaced by a one to six character ASCII identifier. The permissible characters in the identifier are A-Z, a-z, and 0-9. In general the identifier should be descriptive of the type of data contained within the packet. As the "address" space of this identifier is rather small, not every packet type should have its own identifier. An application may reserve its own identifier and have all packets use the same identifier, or it may use a general identifier. There is no restriction that a binary type (defined below) always match a particular ASCII identifier as the ASCII string is used for delimiting messages in persistent storage and for conveinence.

See table at bottom of document for the currently used identifiers and notes on their meaning.
2.

A 2-byte binary type.

Each type must be unique to a particular TLD packet definition. This means that once a type has been used the corresponding data layout and "meaning" of the packet cannot be amended or reused. A binary type is considered "claimed" when the owner adds it to this document (a step you should always take first) and is "defined" once the application for which it has been used is finalized (i.e. you don't have to pick the final format up front, just don't ever change it after you have finalized or released a product or demonstrator with the type in use).

See table at bottom of document for the currently used binary types, their "meaning", and the data format associated with each type.
3.

A 2-byte binary length.

The length field will hold the number of bytes that are contained in the packet not including the checksum at the end (defined below).
4.

N-bytes of data.

The N must match the value in the length field. Remember, that if you are packing a NULL terminated string as one of the fields of data, you must account for the '\0' at the end.
5.

A 2-byte binary checksum.

The checksum is computed over the binary type field, the length field, and all of the data. The checksum shall always be computed with a CRC16 (CCITT) with polynomial of 0x1021. For example, if using the Boost CRC object, you would use the following code:

#include <boost/crc.hpp>
boost::crc_optimal<16, 0x1021, 0xFFFF, 0, false, false> crc_ccitt2;
crc_ccitt2 = std::for_each(buffer, size, crc_ccitt2);
uint16_t crc = crc_ccitt2();

The prototype for the TLD header would look something like this:

typedef struct tld_msg {
uint16_t type; // Type of packet
uint16_t length; // Length of data
uint8_t data[1]; // Start of the data
} tld_msg_t;

Geographic Pose information in Message Data

Several message types describe geographic location and orientation in a world frame. Geographic position is expressed in WGS84 latitude, longitude and altitude coordinates, with latitude and longitude given in decimal radians and altitude in meters.

Orientation is given as a unit quaternion, a 4-dimensional vector that describes a rotation and isn't affected by singularities. The first element is real and the last 3 element are imaginary. The conversion from a quaternion [a,b,c,d] to Euler angles [x,y,z] (Assuming an Aerospace xyz Euler sequence as described in Quaternions and Rotation Sequences, by J. Kuipers) is given by

x = atan2(2*(a*b+c*d),(1-2*(b*b+c*c)))
y = asin(2*(a*c-b*d))
z = atan2(2*(a*d+b*c),(1-2*(c*c+d*d)))

A timestamp counter that is used for all events. The timestamp counter is a 64-bit register, free-running at 66.666MHz (wrap-around > 8700 years), which is reset during power-up.


Set Timestamp counter to real-world time; use the ppsTimestamp message, which is sent once per second, and locate the closest, preceeding GPS timestamp to calculate the number of timestamp counter ticks in real-world time.

Types
Name Description
point_type_t Point types for point_data_t id
0x01 ecef_point_t
0x02 ecef_i_point_t

ecef_point_t various point structures
x double
y double
z double

Name Number Description
point_data 0x0010
id uint8
source uint8
timestamp uint64 66.66 MHz IP-S2 timestamp
numPoints uint16
points[numPoints] point_type_t Array of points
The data doesn't match the file format. And the string end ends with 0D 0A, rather than 0A. So you may have problems with the data.

Anyway, it's mostly fixed length stuff so the only tricky bit is reading the variable length name.

1. Allocate the record with malloc to be sizeof(tld_msg).
2. Read first four chars. They should be "MSG:"
3. Read upto the first 0A. This is the name. If it ends with 0D, remove it. Allocate a string (with strdup) and set the header field to this value.
4. Read the next two bytes into the type field.
5. Read the next two bytes into the length field.
6. Realloc the record to be sizeof(tld_msg) + length - sizeof(data).
7. Read length bytes into data field.
8. Read two bytes into a unt16_t called crc. Use the CRC algorithm to verify the block held in data of size length.
Last edited on
Topic archived. No new replies allowed.