What is the best way to slurp a text file into an array/collection of std::string in C++

Question

In the style of What is the best way to read an entire file into a std::string in C++? I want to ask a similar question, except that I want my output to be an array/STL container containing the lines in a text file.

C#/.net has the rather useful File.ReadAllLines() utility function, what would a good C++ (STL) version look like?

possible duplicate of [How do I iterate over cin line by line in C++?](http://stackoverflow.com/questions/1567082/how-do-i-iterate-over-cin-line-by-line-in-c) — Jerry Coffin, Jan 28 '13 at 14:42
Is it a dupe? I have a fixed-sized file to work from. Would @Nawaz's answer work on `cin`? — Mr. Boy, Jan 29 '13 at 09:21
@John: Yes. My answer would work on `std::cin` also. Here is online demo : http://ideone.com/d1Ng1h — Nawaz, Jan 29 '13 at 10:28

Nawaz · Accepted Answer · 2013-01-28T12:11:23.580

6

In C++, you could do this.

Define a struct as:

struct line : std::string 
{
   friend std::istream & operator >> (std::istream & in, line & ln)
   {
      return std::getline(in, ln);
   }
};

then do this:

std::ifstream file("file.txt");
std::istream_iterator<line> begin(file), end;
std::vector<std::string> allLines(begin, end);

Done!

With this approach, you can directly work with iterator-pair begin and end. No need to use std::vector<std::string>. Note that line can implicitly convert into std::string. So you can use begin and end with the Standard algorithms.

For example,

Find the longest line:

auto cmp = [](line const &a, line const& b) { return a.size() < b.size(); };
std::string longestLine = *std::max_element(begin, end, cmp);

Count the lines whose length is greater than 10:

auto cmp = [](line const &a) { return a.size() > 10 ; };
size_t count = std::count_if(begin, end, cmp);

So on. In this way, you can work directly work with begin and end. No need to use std::vector. Save memory.

Hope that helps.

edited Jan 28 '13 at 12:11

answered Jan 28 '13 at 11:50

Nawaz

327,095
105
629
812

2

But that breaks on whitespace not newlines. – Peter Wood Jan 28 '13 at 11:50
Why do you assume such a seldom situation? – leemes Jan 28 '13 at 11:52
1

@leemes: assumption removed. – Nawaz Jan 28 '13 at 11:54
Excellent fix. This now works with algorithms. I'd prefer it without the inheritance from `std::string` though. – Peter Wood Jan 28 '13 at 11:55
1

Nice! With the `line` trick, maybe the OP could even avoid the using `vector` and use the `begin,end` range directly, depending on what he intends to do. – rodrigo Jan 28 '13 at 11:58
1

For beginners more complex, but more elegant than the straight-forward imperative way. I like! – leemes Jan 28 '13 at 11:58
Well now which one am I supposed to pick as the 'right' answer?! Think I'll have to let the votes decide it :) – Mr. Boy Jan 28 '13 at 12:04
Oh, can you explain this behavior? http://ideone.com/TMDpok (I'm using cin as "the file", since ideone can't process files AFAIK) – leemes Jan 28 '13 at 12:04
@leemes The `operator< – Peter Wood Jan 28 '13 at 12:05
@leemes: it should be `operator>>`, not `< – Nawaz Jan 28 '13 at 12:06
Ah, I was too slow. Fixed ideone link, maybe you want to put it in your answer: http://ideone.com/p4N8YP – leemes Jan 28 '13 at 12:07
1

Didn't see that you also fixed the ideone link ;) – leemes Jan 28 '13 at 12:31
This is quite neat as you save memory. Is there a way of re-winding the iterator apart from closing and re-opening the file? – Roman Kutlak Jan 28 '13 at 14:19
@RomanKutlak: No. It is a *single-pass* input iterator, once you read data through it, it cannot go back. Read about it here : http://en.cppreference.com/w/cpp/iterator/istream_iterator – Nawaz Jan 28 '13 at 14:29
2

+1 Finally a solution for the missing "getline" stream iterator! (Though this should be a `typedef basic_line line;`, right? :-) ) – Kerrek SB Jan 29 '13 at 12:09

score 4 · Answer 2 · answered Jan 28 '13 at 11:51

4

The standard idiom:

#include <fstream>
#include <string>
#include <vector>
#include <utility>

std::ifstream infile("file.txt");
std::vector<std::string> v;

for (std::string line; std::getline(infile, line); )
{
    v.push_back(std::move(line));
}

answered Jan 28 '13 at 11:51

Kerrek SB

428,875
83
813
1,025

score 3 · Answer 3 · answered Jan 28 '13 at 11:50

3

std::ifstream ifs(name);
std::vector<std::string> lines;
std::string line;
while (std::getline(ifs, line))
    lines.push_back(line);

answered Jan 28 '13 at 11:50

rodrigo

79,651
7
121
162

score 1 · Answer 4 · answered Jan 28 '13 at 21:29

1

This is my attempt at @Nawaz's idea. I've avoided inheritance from std::string, as I felt a little queasy about it:

#include <iostream>
#include <iterator>
#include <vector>

struct Line 
{
    operator const std::string&() const {return string;}
    std::string string;
};

std::istream& operator>>(std::istream& in, Line& line)
{
    return std::getline(in, line.string);
}

int main()
{
    std::istream_iterator<Line> begin(std::cin), end;
    std::vector<std::string> allLines(begin, end);

    std::cout << allLines.size() << " lines read from file:" << std::endl;
    std::copy(allLines.begin(), allLines.end(),
          std::ostream_iterator<std::string>(std::cout, "|"));
    return 0;
}

answered Jan 28 '13 at 21:29

Peter Wood

21,348
4
53
90

With this, the user wouldn't be able to write `line.size()` for example. And I don't find any reason why `line` shoudn't be derived from `std::string`. Or say why `basic_line` should not be derived from `basic_string`. `basic_line` *is* `basic_string`. Only that they're not polymorphic. – Nawaz Jan 29 '13 at 13:10
I would only use `Line` to marshal data into `std::string`s. – Peter Wood Jan 29 '13 at 13:17
I know that already from your answer. It is just that I don't find any convincing reason why `class basic_line : public std::basic_string {};` is not a good idea. I feel that nobody has done it so far, that is why many find it in the must-be-avoided category. – Nawaz Jan 29 '13 at 13:19
There are [two](http://stackoverflow.com/questions/4205050/inheriting-and-overriding-functions-of-a-stdstring) [questions](http://stackoverflow.com/questions/6006860/why-should-one-not-derive-from-c-std-string-class) about not deriving from `std::string`. – Peter Wood Jan 29 '13 at 13:22
Nothing in those answers applies to this situation. – Nawaz Jan 29 '13 at 15:13
@PeterWood: Welllll.... polymorphism and inheritance are different things. It's true that there are no virtual functions in `std::basic_string`, but... so what? You only need a virtual destructor if you are handling more-derived objects through base interfaces. But no such thing is being done here. In other words, a `line` *is-not* a `string`. It's a line, and it just happens to be identical to a string. – Kerrek SB Jan 29 '13 at 15:13
I'm worried about object slicing. I know it's unlikely to cause a problem as `list` has no member data, but are there any dangers? – Peter Wood Jan 29 '13 at 22:18

What is the best way to slurp a text file into an array/collection of std::string in C++

4 Answers4