Fail Read Int64 value from binary file created by C++

Question

I m developing a C# CE application to read data from binary files which created by C++ progam to do item validation.

Below is the coding of the C++ program..

 // File Name: Ean2an.bin which is created by struct 
    struct EAN2AN_TYPE 
    {
    __int64     ean:40;     // 5 bytes, up to 12 digits
    __int64     rec_no:24;  // 3 bytes rec no in the c_ItemMaster File, up to 16 million  records
    };

    // After bind data to struct, wil create the binary file
   bool CreateBin_EAN2AN_TYPE()
   {
    if(mn_RecordCount_EAN2AN_TYPE == 0) return false;

    FILE   *binfile;

    qsort(mc_EAN2AN_TYPE, mn_RecordCount_EAN2AN_TYPE, sizeof(struct EAN2AN_TYPE), qsort_EAN2AN_TYPE);
    try
    {
        binfile = fopen(ms_Path_EAN2AN_TYPE, "wb");
        fwrite(&mc_EAN2AN_TYPE, sizeof(struct EAN2AN_TYPE), mn_RecordCount_EAN2AN_TYPE, binfile);
    }
    catch(Exception ^ex)
    {
        TaskProgramLibrary::Message::ERR("Create EAN2AN_TYPE.bin fail!\r\n       " + ex->Message);
    }
    finally
    {
        fclose(binfile);

        mdw_FileSize_EAN2AN_TYPE = FileSize(ms_Path_EAN2AN_TYPE);
    }

    return true;
      }

I tried to read the data by using binary read(based on position) and use bitconverter to convert to int64 or using Marshal.PtrToStructure, but the value return is incorrect. Then i tried to read 5 bytes instead of 8 bytes from the file, but the value return stil incorrect.

Below is the written C# coding
    //Struct created in C#
   [StructLayout(LayoutKind.Sequential)]
        public struct EAN2AN_TYPE
        {
            [MarshalAs(UnmanagedType.I8)]
            public Int64 ean;
            [MarshalAs(UnmanagedType.I8)]
            public Int64 rec_no;
        } 

    //The ways i tried to read in C#
    //1.Read Int64 by Binary 
    private void ReadByBinary()
         {
            using (BinaryReader b = new BinaryReader(_fs))
            {
                while (b.PeekChar() != 0)
                {
                    Int64 x = b.ReadInt64();
                    Console.WriteLine(x.ToString());

                }
            }

        }

   //2.Using Marshal to convert the Intptr to struct's field type
    private object ReadByMarshal(Type iType)
        {
            _oType = iType;// typeof(System.Int64);
            byte[] buffer = new byte[Marshal.SizeOf(_oType)];
            //byte[] buffer = new byte[5];

            object oReturn = null;

            try
            {
                _fs.Read(buffer, 0, buffer.Length);

                GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
                oReturn = Marshal.PtrToStructure(handle.AddrOfPinnedObject(), _oType);
                handle.Free();

                return oReturn;
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }


    //3. Use Binary and use bit converter to convert to Int64
     private void ReadByBinaryAndUseBitConverter()
        {
            using (BinaryReader b = new BinaryReader(_fs))
            {
                byte[] x = b.ReadBytes(8);
                Int64 y = BitConverter.ToInt64(x, 0);
                Console.WriteLine(y);

                byte[] x2 = b.ReadBytes(8);
                Int64 y2 = BitConverter.ToInt64(x2,0);
                Console.WriteLine(y2);
            }

        }


    //4. Use Marshal and convert to struct
    public EAN2AN_TYPE  GetStructValue()
        {

            byte[] buffer = new byte[Marshal.SizeOf(typeof(EAN2AN_TYPE)];

            EAN2AN_TYPE oReturn = new EAN2AN_TYPE();

            try
            {
                //if (EOF) return null;

                _fs.Read(buffer, 0, buffer.Length);
                GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
                IntPtr rawDataPtr = handle.AddrOfPinnedObject();

               oReturn = (EAN2AN_TYPE)Marshal.PtrToStructure(rawDataPtr, typeof(EAN2AN_TYPE));

                handle.Free();

                if (_fs.Position >= _fs.Length)
                    Close();

                return oReturn;
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }

Edit:Upload image for the binary file

Edit:The first 8 bytes value read by C# program

The binary data shown by editor

Anybody have any idea?

Thanks in advance

Answer 1

ean is defined as a 40-bit entity and rec_no is 24-bit, making the entire struct only 64 bits. Your definition of EAN2AN_TYPE is 128 bits, so there's obviously going to be a problem. I question the sanity of whoever wrote the initial code, but your job is to get it back and use it, so you play with what you're dealt.

EDIT: Updated to use your specified data and take into account Ben's complaint

Here are two ways to get the same result. One is a little easier to understand since it does the movement in steps, the other is faster and "more correct". I put your example EAN data into my input to verify the results.

public struct EAN2AN_TYPE
{
    public long ean; // can hold 5 bytes
    public int rec_no; // can hold 3 bytes
}

byte[] incoming = new byte[] { 0x6F, 0x5D, 0x7C, 0xBA, 0xE3, 0x06, 0x07, 0x08 };

Memory Copying:

using(var stream = new MemoryStream(incoming))
using (var reader = new BinaryReader(stream))
{
    // I leave it to you to get to the data
    stream.Seek(0, SeekOrigin.Begin);

    // get the data, padded to where we need for endianness
    var ean_bytes = new byte[8];
    // read the first 5 bytes
    Buffer.BlockCopy(reader.ReadBytes(5), 0, ean_bytes, 0, 5);
    var rec_no_bytes = new byte[4];
    // read the last 3
    Buffer.BlockCopy(reader.ReadBytes(3), 0, rec_no_bytes, 0, 3);

    var ean2 = new EAN2AN_TYPE();

    // convert
    ean2.ean = BitConverter.ToInt64(ean_bytes, 0);
    ean2.rec_no = BitConverter.ToInt32(rec_no_bytes, 0);
}

Bit shifting:

using (var stream = new MemoryStream(incoming))
using (var reader = new BinaryReader(stream))
{
    // I leave it to you to get to the data
    stream.Seek(0, SeekOrigin.Begin);

    // get the data
    var data = BitConverter.ToUInt64(reader.ReadBytes(8), 0);
    var ean2 = new EAN2AN_TYPE();

    // shift into our data
    ean2.ean = (long)(data & ~0xFFFFFF0000000000);
    ean2.rec_no = (int)(data >> 40);
}

Of course you could make the EAN2AN_TYPE a class, feed it in 8 bytes, then have property accessors that do the shifting shenanigans for you as well. I'd do that if this has to be a 2-way thing (i.e. you need to put data into one of those structs to send back to the C app).

Answer 2

Thanks for your reply..

The initial C++ code is written by vendor. I only can try understand by read the C++ code. As my understanding.. it just create binary file and write in the data.. I cant find any encoding/convert part from code ..

I tried to convert the 1st ean(978086288751) to byte[] manually by code. The byte[] is 111 93 124 186 227 0 0 0 which is different for the result i get..

I have tested ctacke suggested code. but i still not able to get the correct ean..

Below is the coding.. (i added in the filestream to read the binary file)

 using (FileStream fileStream = File.OpenRead(_File))
            {
                MemoryStream memStream = new MemoryStream();
                memStream.SetLength(fileStream.Length);
                fileStream.Read(memStream.GetBuffer(), 0, (int)fileStream.Length);

                using (BinaryReader reader = new BinaryReader(memStream))
                {
                    //stream.SetLength(_fs);               
                    // I leave it to you to get to the data
                    memStream.Seek(0, SeekOrigin.Begin);

                    // get the data, padded to where we need for endianness
                    byte[] ean_bytes = new byte[8];
                    // if this is wrong - then change param 4 to '3' to align at the other end
                    Buffer.BlockCopy(reader.ReadBytes(8), 0, ean_bytes, 0, 8);
                    //byte[] rec_no_bytes = new byte[4];

                    byte[] rec_no_bytes = new byte[4];
                    // if this is wrong - then change param 4 to '1' to align at the other end
                    Buffer.BlockCopy(reader.ReadBytes(3), 0, rec_no_bytes, 0, 3);

                    EAN2AN_TYPE ean2 = new EAN2AN_TYPE();

                    // convert
                    ean2.ean = BitConverter.ToInt64(ean_bytes, 0);
                    ean2.rec_no = BitConverter.ToInt32(rec_no_bytes, 0);
                }

            }

//Result read 5 bytes : 17 0 0 0 0 ean : 17

//I changed to

   var ean_bytes = new byte[8];
    Buffer.BlockCopy(reader.ReadBytes(8), 0, ean_bytes, 0, 8);

Result read 8 bytes :17 0 0 0 0 108 94 5 ean :386865365256241169

So sorry i m still a new user.. not able to post any attachment.. Hope you can understand from my interpretation.

Answer 3

It could be a problem with the endianness (if that's a word) of the data. You'll get problems like this if the data was written on a big-endian system, and read as little-endian, or vice-versa.

The other problem is that the two fields are actually packed into one 64-bit value. You may need to read an Int64 and then use bit operations to extract the two fields. All of your code appears to be reading two Int64 values by various means.