20

On Linux for removing carriage return we can execute:

sed -i 's/\r//g' <file>

But the same will not work on Mac OS X. Need to prepend $ like:

sed -i $'s/\r//' <file>

And "g" is also not needed.

Why is this so?

Giacomo1968
  • 23,903
  • 10
  • 59
  • 92
Gaurav Fotedar
  • 655
  • 1
  • 6
  • 11
  • (supposition, i have no Mac) shell interpretation/file management. Sed work line by line and normaly the line is ended by CR, not taken by sed. Linux and mac version does not consider the same way the end of the line. g is needed if more than 1 substitution but this is always the last char by default. – NeronLeVelu Feb 07 '14 at 08:01

3 Answers3

36

It is because sed available on OSX doesn't recognize \r as a special character unlike the sed on Linux does.

You can either use it the way you're using:

sed -i.bak $'s/\r//' file

OR this:

sed -i.bak "s/$(printf '\r')//" file

OR else you can use tr on OSX:

tr -d '\r' < file
anubhava
  • 664,788
  • 59
  • 469
  • 547
  • But replacing normal text works just fine without any need for "$"..... so if \r is not considered as special, even then $ should not be required?? – Gaurav Fotedar Feb 08 '14 at 07:58
  • That is true only `\n` and `\r` need prefix of `$` – anubhava Feb 08 '14 at 07:59
  • would someone explain what the `$` prefix does here? I understand $ in most unix shells to expand a shell variable. I don't understand why it has the effect of getting a `\r` to be processed as a carriage return by sed. – Avram Jul 17 '17 at 17:29
  • That is bash syntax to deal with backslash-escaped characters. Check `man bash` and search for `\$'string'` – anubhava Jul 17 '17 at 18:10
8

Another portable and flexible solution is:

sed -i.bak $'s/\x0D//' file

Because the return character has the ASCII code 0D. \x substitutions work for all POSIX releases of sed, and you can match any other troublesome character by looking up the ASCII code. To find other valid substitutions, execute man re_format or view an ASCII Table.

The final /g is needed for Linux because the carriage return (\r) does not end the line. Many Windows "plain text" editors (e.g. Notepad) end each line with both carriage return and new line (\r\n), but Macs with OS 9 (ca. 2001) or earlier ended each line of a "plain text" file with a single \r. If you're cleaning up a Windows file, the /g is not needed on any *X system. If you're on macOS, the /g isn't needed, either, because macOS recognizes the single \r as a line ending.

(A Linux system reading an old Mac file will think that all the text is on one very long line and only convert the first \r. If you're on a Linux system and need to preserve the line breaks from an old Mac file,

sed -i.bak $'s/\x0D/\x0A/g' file

converts each \r to \n.)

William H. Hooper
  • 525
  • 6
  • 7
0

An easier alternative might be to use perl instead of sed. It's available by default on Mac, and obviously accepts the normal syntax you were trying to use:

perl -i -pe 's/\r//g' $Your_File

In case what you really want is to replace Windows CRLF line endings by Unix LF line endings, there is even a safer approach with Perl's \R linebreak escape:

perl -i -pe 's/\R/\n/g' $Your_File

or

perl -i.bak -pe 's/\R/\n/g' $Your_File

which will also save the original file with a ".bak" extension

For details on Perl's \R, see linebreak escape or Brian D Foy's article : The \R generic line ending which even has a couple of fun videos.

mivk
  • 9,883
  • 3
  • 55
  • 54