String read in from stdin by getline using redirection command "<" from input file does not append correctly

Question

Problem statement

I would like to append a string " are awesome !", to another string "Ice creams" read in from stdin using getline function.

Thus producing "Ice creams are awesome !"

If the input is typed manually, the append operation works flawlessly.

However, if the input is taken from a file through redirection command "<" the append operation does not behave similarly.

Thus producing are awesome ! Ice creams.

Code

Let me demonstrate the problem through code.

#include<iostream>

int main()
{
    // err : will be read from stdin using getline
    // err : is expected to fail in appending
    std::string err;

    // ok  : will be modified internally
    // ok  : is expected to succeed in appending
    std::string ok = "Pancakes";

    std::cout<<"Before executing  std::getline..."<<std::endl;
    std::cout<<"err \t:\t"<<err<<std::endl;
    std::cout<<"ok \t:\t"<<ok<<std::endl;
    std::cout<<"-------------------"<<std::endl;

    // we now use getline to read in string value for err
    // stdin can have any string , let's assume it is "Ice creams"
    std::cout<<"-------------------"<<std::endl;
    std::cout<<"Please enter string manually or < from file "<<std::endl;
    std::getline(std::cin,err);
    std::cout<<"-------------------"<<std::endl;


    std::cout<<"After executing std::getline..."<<std::endl;
    std::cout<<"err \t:\t"<<err<<std::endl;
    std::cout<<"ok \t:\t"<<ok<<std::endl;
    std::cout<<"-------------------"<<std::endl;

    //-------------------------------------------//
    //               THE PROBLEM                 //
    //-------------------------------------------//
    // we try to append to err
    err += " are awesome !";
    // we try to append to ok
    ok  += " are awesome !";

    std::cout<<"After executing append operation..."<<std::endl;
    std::cout<<"Where,the '+=' operator is used..."<<std::endl;
    std::cout<<"err \t:\t"<<err<<std::endl;
    std::cout<<"ok \t:\t"<<ok<<std::endl;
    std::cout<<"-------------------"<<std::endl;


    return 0;
}

Terminal output

The compilation is done using g++ running in a Windows 10 machine with the help of WSL functionality.

C:\test>wsl g++ test.cpp -o a.out -std=c++11

C:\test>wsl ./a.out
Before executing  std::getline...
err     :
ok      :       Pancakes
-------------------
-------------------
Please enter string manually or < from file
Ice creams
-------------------
After executing std::getline...
err     :       Ice creams
ok      :       Pancakes
-------------------
After executing append operation...
Where,the '+=' operator is used...
err     :       Ice creams are awesome !
ok      :       Pancakes are awesome !
-------------------

C:\test>wsl ./a.out < in-test.in
Before executing  std::getline...
err     :
ok      :       Pancakes
-------------------
-------------------
Please enter string manually or < from file
-------------------
After executing std::getline...
err     :       Ice creams
ok      :       Pancakes
-------------------
After executing append operation...
Where,the '+=' operator is used...
 are awesome !  Ice creams
ok      :       Pancakes are awesome !
-------------------

C:\test>

Results

Append produces correct results for manual input from stdin.
Append produces garbled results for input from file in-test.in from stdin.

Note : in-test.in file contains one line with the string "Ice creams".

Note : If it is difficult to notice the error, then let me point you to it.

It is the line are awesome ! Ice creams located at the tail of the Terminal output.

Questions

Why does this happen?
How to avoid it?

Wow ! down voted 2 times within 5 seconds of posting the question and within 5 views. Where is the mistake? — SYN, Aug 20 '18 at 19:51
This is exactly the code you are using? which version of gcc? which OS? there is nothing wrong with the code itself. — Alan Birtles, Aug 20 '18 at 20:05

score 3 · Answer 1 · answered Aug 20 '18 at 19:56

3

Note how the line does't even contain the output supposedly leading "err \t:\t", but that the output of err shows where it's supposed to?

That could only come because the input file contains a leading carriage-return '\r'.

answered Aug 20 '18 at 19:56

Some programmer dude

363,249
31
351
550

score 0 · Answer 2 · answered Aug 21 '18 at 07:03

Aknowledgement

Nice of Some programmer dude to mention that '\r' can be present in the string.

Problem source

Makes sense, Linux, Macintosh and Windows have different line terminators and as such can have some problems when we want to transfer files from one system to another and expect them to work correctly.

Explained beautifully in this question :Difference between CR LF, LF and CR line break types? .

That's exactly what happened. I created the file using Notepad.exe and hence automatically assigned to it a CRLF line terminator. However, since I compile the program using Windows Subsystem for Linux (WSL)(essentially Ubuntu running inside Windows 10), the executable created a.out is a linux executable.

Redirecting a file with CRLF line terminators into the executable stdin caused the problem.

This particular problem possibly will be faced by users migrating to WSL after using MinGW or MSYS2.

Since, MinGW and MSYS2 work great with CRLF encoding but WSL does not.

Fix

Just change the line terminator using a tool called unix2dos or dos2unix.

As the names suggest they convert the text files to be compatible with dos and unix based OS.

Use unix2dos when using files created using Linux/WSL to make them compatible with Windows executable.
Use dos2unix when using files created using Windows to make them compatible with Linux/WSL executables.

Demo

C:\test>wsl g++ test.cpp -o a.out -std=c++11

C:\test>wsl unix2dos in-test.in
unix2dos: converting file in-test.in to DOS format ...

C:\test>wsl file in-test.in
in-test.in: ASCII text, with CRLF line terminators

C:\test>wsl ./a.out < in-test.in
Before executing  std::getline...
err     :
ok      :       Pancakes
-------------------
-------------------
Please enter string manually or < from file
-------------------
After executing std::getline...
err     :       Ice creams
ok      :       Pancakes
-------------------
After executing append operation...
Where,the '=' operator overloading is used...
 are awesome !  Ice creams
ok      :       Pancakes are awesome !
-------------------

C:\test>wsl dos2unix in-test.in
dos2unix: converting file in-test.in to Unix format ...

C:\test>wsl file in-test.in
in-test.in: ASCII text

C:\test>wsl ./a.out < in-test.in
Before executing  std::getline...
err     :
ok      :       Pancakes
-------------------
-------------------
Please enter string manually or < from file
-------------------
After executing std::getline...
err     :       Ice creams
ok      :       Pancakes
-------------------
After executing append operation...
Where,the '=' operator overloading is used...
err     :       Ice creams are awesome !
ok      :       Pancakes are awesome !
-------------------

Results

The executable created by WSL did not work initially with the input file with CRLF line terminators, then
The file's line terminators were changed to that compatible with Linux and hence WSL by using the tool dos2unix, resulting in the successful operation of the program. Hence, the fix works.

String read in from stdin by getline using redirection command "<" from input file does not append correctly

2 Answers2