How to create a dynamic array of structs reading data from a binary file

Question

I have a struct which describes the data objects stored in a binary file. The struct looks like:

struct Sum_str {
public:
    char    FINISH_T[24]; // U4
    char    DISP_COD; // C1
    char    USR_DESC[128]; // Cn
    char    EXC_DESC[128]; // Cn
};

This is loaded via reading from a binary file and the number of data objects in the file may vary. When I declare the struct as a set limit via a const everything works. Unfortunately the binary can have less or more data than declared.

How can I dynamically grow the array as needed depending on the number of objects stored in the binary file?

Answer 1

You aren't using the correct container if the size of your arrays can be greater than your given maxes of 24, 128, 128. Use a vector instead of arrays if you need your containers to be dynamic in size

struct Sum_str {
public:
    std::vector<char>    FINISH_T; // U4
    char                 DISP_COD; // C1
    std::vector<char>    USR_DESC; // Cn
    std::vector<char>    EXC_DESC; // Cn
};

If you are restricted to the arrays, then only fill the arrays up to their respective lengths. Remember to place null terminating characters in the end of the data so you don't accidentally read past your char arrays when printing the contents. Also use #define for your array sizes instead of magic numbers.

If you are unsure of how many Sum_str structs you need then you should use a container that will fit your requirements. Your requirements require a container that can change in size, therefore you should use a vector

std::vector<Sum_str> my_container;

Answer 2

You do not specify the actual format of the binary file so I am going to assume that it contains an array of the structs you provided.

So your basic algorithm will be something like the following:

• create a std::vector of the struct as in std:vector <Sum_str > myVect;
• next beginning at the proper point in the binary file, I assume offset 0, read a binary record which means to read in a struct as a binary object
• put it into the vector as in myVect.push_back(obj);
• continue this reading of an object and putting into the vector until you reach the proper ending point which I assume is end of file

The primary and dependent assumption is the binary file contains zero or more of what amounts to a binary image of an array of the structures, Sum_str. The reason this is important is that you want to make sure that:

• the size of a Sum_str object in the file is the same size as a Sum_str object in memory
• that the layout and offsets of the members of a Sum_str object in the file is the same layout and offsets of members as a Sum_str object in memory

If the binary file contains only an array of struct data then you may want to estimate the number of structs contained in the file and create the std::vector with an initial capacity that approximates the estimate. See Initial capacity of vector in C++ as well as Using C++ filestreams (fstream), how can you determine the size of a file? which provides a way of counting the size of a file. The tellg() method of getting the file size is not dependable though it looks like with binary files on Windows and Linux it is close enough.

See Reading and writing binary file as well as How to read a binary file into a vector of unsigned chars as well as c++ Read/Write class from/to binary file and C++ binary files and iterators: getting away with a 1:1 using ifstreambuf_iterator? .

First cut - a simple demo using older style C++

I don't have C++11/17 available where I am so here is a version using an older C++ compiler until I can get to a newer version and can do a bit of research.

#include <iostream>
#include <fstream>
#include <vector>

struct Person
{
  char name[50];
  int age;
  char phone[24];
};

int mainx()
{
  Person me = {"Robert", 28, "364-2534"};
  Person book[30];
  int x = 123;
  double fx = 34.54;

  // put in some data for our output to show that it works.
  for (int i = 0; i < 30; i++) {
      book[i] = me;
      book[i].age = i *10 + 5;  // modify the age so we can see it changed.
  }
  std::ofstream outfile;
  outfile.open("junk.dat", std::ios::binary | std::ios::out);
  outfile.write((char *)&x, sizeof(int)); // sizeof can take a type
  outfile.write((char *)&fx, sizeof(fx)); // or it can take a variable name
  outfile.write((char *)&me, sizeof(me));
  outfile.write((char *)book, 30*sizeof(Person));
  outfile.close();

  return 0;
}

int mainy()
{
  Person me = {0};
  std::vector<Person>book;
  int x = 0;
  double fx = 0;
  std::ifstream infile;
  infile.open("junk.dat", std::ios::binary | std::ios::in);
  infile.read((char *)&x, sizeof(int)); // sizeof can take a type
  infile.read((char *)&fx, sizeof(fx)); // or it can take a variable name
  infile.read((char *)&me, sizeof(me));
  for (int i = 0; i < 30; i++) {
      Person buff;
      infile.read((char *)&buff, sizeof(Person));
      book.push_back(buff);
  }

  infile.close();

  std::cout << "x = " << x << std::endl;
  std::cout << "fx = " << fx << std::endl;

  std::cout << "Person me = " << me.name << ", age " << me.age << ", phone " << me.phone << std::endl;

  for (int i = 0; i < 30; i++) {
    std::cout << "Person book = " << i << "  " << book[i].name << ", age " << book[i].age << ", phone " << book[i].phone << std::endl;
  }
  return 0;
}

int main ()
{
    mainx();
    mainy();
    return 0;
}

Addendum - demonstrating using iterators

Here is a second example of the above modified to use iterators. This is not my area of expertise however my testing with Visual Studio 2017 in a console application demonstrates that it works.

See also istream_iterator Class

#include <iterator>
#include <iostream>
#include <fstream>
#include <vector>

struct Person
{
    char name[50];
    int age;
    char phone[24];

    friend std::ostream& operator<<(std::ostream& os, const Person& dt);
    friend std::istream& operator<<(std::istream& os, const Person& dt);
};

std::ostream& operator<<(std::ostream& os, const Person& dt)
{
    os.write((char *)&dt, sizeof(Person));
    return os;
}

std::istream& operator>>(std::istream& os, const Person& dt)
{
    os.read((char *)&dt, sizeof(Person));
    return os;
}

// construct a binary file that contains various objects.
// in the following routine we will read back out from
// the binary file the objects we wrote into it.
int mainx()
{
    Person me = { "Robert", 28, "364-2534" };
    std::vector<Person> book;
    int x = 123;
    double fx = 34.54;

    // put in some data for our output to show that it works.
    for (int i = 0; i < 30; i++) {
        Person x = me;
        x.age = i * 10 + 5;  // modify the age so we can see it changed.

        book.push_back (x);
    }

    // construct out output file with the various objects
    // we want to save. when we read from this file
    // we will want to use the same types of variables
    // in the same order since there is no meta data
    // in this file to indicate object types or object boundaries.
    std::ofstream outfile("junk.dat", std::ios::binary | std::ios::out);
    outfile << x;
    outfile << fx;
    outfile << me;

    // write out the vector of Person objects.
    for (auto x : book) outfile << x;

    outfile.close();

    return 0;
}

// following routine opens the persistent store and reads
// back in the objects that were saved. we need to do
// the reading in the same order they were written.
int mainy()
{
    Person me = { 0 };
    int x = 0;
    double fx = 0;
    std::ifstream infile("junk.dat", std::ios::binary | std::ios::in);
    infile >> x;
    infile >> fx;
    infile >> me;

    // istream_iterator from stream infile
    std::istream_iterator<Person> is(infile);
    // End-of-stream iterator 
    std::istream_iterator<Person> isEof;

    std::vector<Person>book;

    // iterate over the Person objects in the input
    // stream until end of file pushing each object
    // read into our vector.
    while (is != isEof) {
        book.push_back(*is);
        is++;
    }

    infile.close();

    std::cout << "x = " << x << std::endl;
    std::cout << "fx = " << fx << std::endl;

    std::cout << "Person me = " << me.name << ", age " << me.age << ", phone " << me.phone << std::endl;

    int i = 0;
    for (auto x : book) {
        i++;
        std::cout << "Person book = " << i << "  " << x.name << ", age " << x.age << ", phone " << x.phone << std::endl;
    }
    return 0;
}

int main()
{
    mainx();    // write the binary file
    mainy();    // read the binary file
    return 0;
}