52

How it is possible to make a dry run with sed?

I have this command:

find ./ -type f | xargs sed -i 's/string1/string2/g'

But before I really substitute in all the files, i want to check what it WOULD substitute. Copying the whole directory structure to check is no option!

Seanny123
  • 6,594
  • 11
  • 56
  • 106
dmeu
  • 2,885
  • 4
  • 22
  • 36

3 Answers3

37

Remove the -i and pipe it to less to paginate though the results. Alternatively, you can redirect the whole thing to one large file by removing the -i and appending > dryrun.out

I should note that this script of yours will fail miserably with files that contain spaces in their name or other nefarious characters like newlines or whatnot. A better way to do it would be:

while IFS= read -r -d $'\0' file; do
  sed -i 's/string1/string2/g' "$file"
done < <(find ./ -type f -print0)
SiegeX
  • 120,826
  • 20
  • 133
  • 152
  • @dmeu please see my updated answer as to how to do this properly; not the dry run but the real-deal. – SiegeX Dec 22 '10 at 21:09
  • 3
    Anything wrong with `find . -type f -print0 | xargs -0 sed ...`? That has the merit of executing sed once for many files instead of once per file. There is some overhead in the shell-only version - not an outrageous overhead, but some. – Jonathan Leffler Dec 22 '10 at 21:34
  • @SiegeX no problem. white space has nothing lost in my files @Jonathan Leffler: please post a different answer if it really works ;-) – dmeu Dec 22 '10 at 21:58
  • @Jonathan nothing wrong with that except the overhead in the subshell created as noted. If you wanted to go that route but forgo the subshell I would use `find . -type f -exec perl -pi -e 's/.../g' {} +` which has the xargs-like benefit and acts just like GNU's `sed -i` but being way more portable. – SiegeX Dec 22 '10 at 22:13
  • Anything 'find' executes is executed once-per-file, just like in the shell loop, and in contrast to xargs which will accumulate arguments until it reaches a limit, and then executes the command (sed in this case) once with many (possibly hundreds) of file names. If you need process substitution, you can do that for `xargs` too: `xargs -0 sed ... < – Jonathan Leffler Dec 22 '10 at 22:20
  • 1
    @Jonathan *"Anything 'find' executes is executed once-per-file"*. Well, that depends on how you terminate `-exec`. If you terminate it with `\;` then yes, you're right. If you terminate it with `+` like I did in my comment then it acts just like xargs would. – SiegeX Dec 22 '10 at 22:23
  • @Jonathan Alternatively, if you'd rather depend on GNU sed then use perl you can accomplish the very same thing with: `find . -type f -exec bash -c 'sed -i "s/foo/bar/g" "$@"' _ {} +` but it's quite messy since you have to pass in the file names as parameters to the shell. – SiegeX Dec 22 '10 at 22:30
  • OK - there's no end to the inventiveness of GNU commands...it is impossible to keep up with all the additions. I didn't know about the trailing `+` option. Since I have to work on machines without the GNU toolset, it is more important for me to know what is portable to other platforms than to know every last possible extension to the the GNU toolset. Having said which, `-print0` and `-0` and `-i` are all GNU extensions too (3 options, 3 commands). – Jonathan Leffler Dec 22 '10 at 22:33
  • @Jonathan You'll be happy to know that terminating `-exec` with a `+` isn't a GNUism but is in the POSIX standard for `find`, see [this link](http://pubs.opengroup.org/onlinepubs/009695399/utilities/find.html) and search for *-exec utility_name* – SiegeX Dec 22 '10 at 22:41
  • Consider me suitably pleased - thank you. I wonder how many of the systems I use have caught up with that detail from POSIX 2004...HP-UX, Solaris seem to be OK; older AIX (5.x) seems not. 4/5 isn't too bad, I suppose. – Jonathan Leffler Dec 22 '10 at 23:17
  • 1
    Interesting, I don't fully understand how/why the syntax works, but it works. Thank you! – sleepycal Jan 15 '14 at 21:41
19

I would prefer to use the p-option:

find ./ -type f | xargs sed 's/string1/string2/gp'

Could be combined with the --quiet parameter for less verbose output:

find ./ -type f | xargs sed --quiet 's/string1/string2/gp'

From man sed:

p:

Print the current pattern space.

--quiet:

suppress automatic printing of pattern space

Murmel
  • 4,141
  • 38
  • 45
  • 1
    I like this. Just a warning: if you're in a git repo you risk matching strings in your `.git` folder, which is probably not what you want to do. To avoid this, start with something a bit smarter than `find ./ -type f`. For example: `ag -l | xargs sed...` (you may need to install ag) – MatrixManAtYrService Apr 21 '19 at 00:00
9

I know this is a very old thread and the OP doesn't really need this answer, but I came here looking for a dry run mode myself, so thought of adding the below piece of advice for anyone coming here in future. What I wanted to do was to avoid stomping the backup file unless there is something really changing. If you blindly run sed using the -i option with backup suffix, existing backup file gets overwritten, even when there is nothing substituted.

The way I ended up doing is to pipe sed output to diff and see if anything changed and then rerun sed with in-place update option, something like this:

if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
    sed -i.bak -e 's/string1/string2/g' $fpath
fi

As per OP's question, if the requirement is to just see what would change, then instead of running the in-pace sed, you could do the diff again with some informative messages:

if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
    echo "File $fpath will change with the below diff:"
    sed -e 's/string1/string2/g' $fpath | diff $fpath -
fi

You could also capture the output in a variable to avoid doing it twice:

diff=$(sed -e 's/string1/string2/g' $fpath | diff $fpath -)
if [[ $? -ne 0 ]]; then
    echo "File $fpath will change with the below diff:"
    echo "$diff"
fi
haridsv
  • 7,212
  • 4
  • 56
  • 57