111

How can Bash rename a series of packages to remove their version numbers? I've been toying around with both expr and %%, to no avail.

Examples:

Xft2-2.1.13.pkg becomes Xft2.pkg

jasper-1.900.1.pkg becomes jasper.pkg

xorg-libXrandr-1.2.3.pkg becomes xorg-libXrandr.pkg

codeforester
  • 28,846
  • 11
  • 78
  • 104
Jeremy L
  • 7,138
  • 4
  • 27
  • 36
  • 1
    I intend to use this regularly, as a write-once use-a-lot script. Any system I'll be using will have bash on it, so I'm not afraid of the bashisms that are quite handy. – Jeremy L Mar 02 '09 at 15:45
  • More generally see also https://stackoverflow.com/questions/28725333/looping-over-pairs-of-values-in-bash – tripleee Mar 04 '18 at 19:19

10 Answers10

203

You could use bash's parameter expansion feature

for i in ./*.pkg ; do mv "$i" "${i/-[0-9.]*.pkg/.pkg}" ; done

Quotes are needed for filenames with spaces.

tripleee
  • 139,311
  • 24
  • 207
  • 268
richq
  • 52,738
  • 20
  • 144
  • 141
  • 1
    I like it too, but it uses bash-specific features... I normally don't use them in favor of "standard" sh. – Diego Sevilla Mar 02 '09 at 15:39
  • 2
    For throwaway one-line interactive stuff I always use this instead of sed-fu. If it were a script, yep, avoid bashisms. – richq Mar 02 '09 at 15:42
  • One issue I've found with the code: What happens if the files were already renamed and I run it again? I get warnings for trying move into a subdirectory of itself. – Jeremy L Mar 02 '09 at 16:02
  • 1
    To avoid re-run errors, you can use the same pattern in the "*.pkg" part, ie "for i in *-[0-9.]*.pkg ; do mv $i ${i/-[0-9.]*.pkg/.pkg} ; done". But the errors are innocuous enough (moving to the same file). – richq Mar 02 '09 at 16:14
  • 4
    Not a bash solution, but if you have perl installed, it provides the handy rename command for batch renaming, simply do `rename "s/-[0-9.]*//" *.pkg` – Rufus Jun 26 '16 at 08:40
  • [My recent answer elsewhere on this page](https://stackoverflow.com/a/49099627/874188) has a remark on how the glob pattern `-[0-9.]*` might not mean what you think or hope. – tripleee Mar 06 '18 at 04:19
  • Update on this answer for recursive go: ```for i in ls **/*.pkg ; do mv "$i" "${i/-[0-9.]*.pkg/.pkg}" ; done``` – crollywood Sep 20 '18 at 07:58
  • What is the technical term / man page where I can look up the details of the replacement business in `{i/-[0-9.]*.pkg/.pkg}` ? – patrick Nov 20 '20 at 22:37
36

If all files are in the same directory the sequence

ls | 
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' | 
sh

will do your job. The sed command will create a sequence of mv commands, which you can then pipe into the shell. It's best to first run the pipeline without the trailing | sh so as to verify that the command does what you want.

To recurse through multiple directories use something like

find . -type f |
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' |
sh

Note that in sed the regular expression grouping sequence is brackets preceded by a backslash, \( and \), rather than single brackets ( and ).

Diomidis Spinellis
  • 17,588
  • 4
  • 52
  • 77
  • Forgive my naïvety, but wouldn't escaping the parentheses with backslashes make them be treated as literal characters? – devios1 Mar 07 '14 at 01:11
  • 2
    In sed the regular expression grouping sequence is \(, rather than a single bracket. – Diomidis Spinellis Mar 08 '14 at 15:00
  • didn't work for me, would be glad to get your help.... The suffix of the FROM file is appended to the TO file: $find . -type d | sed -n 's/\(.*\)\/\(.*\)anysoftkeyboard\(.*\)/mv "\1\/\2anysoftkeyboard\3" "\1\/\2effectedkeyboard\3"/p'|sh >>>>>>>OUTPUT>>>> mv: rename ./base/src/main/java/com/anysoftkeyboard/base/dictionaries to ./base/src/main/java/com/effectedkeyboard/base/dictionaries/dictionaries: No such file or directory – Vitali Pom Jan 25 '18 at 21:56
  • You seem to be missing the backslash in the brackets. – Diomidis Spinellis Jan 27 '18 at 08:01
  • 1
    [Don't use `ls` in scripts.](http://mywiki.wooledge.org/ParsingLs) The first variety can be achieved with `printf '%s\n' * | sed ...` and with some creativity you can probably get rid of the `sed` too. Bash's `printf` offers a `'%q'` format code which could come in handy if you have filenames which contain literal shell metacharacters. – tripleee Sep 19 '18 at 04:49
  • Handling newlines embedded in file names is a noble goal. But the output of `printf` will still fail in this case. It will also fail if the expanded file names don't fit in the command's argument vector. So the approach based on `printf` is even less robust. – Diomidis Spinellis Sep 20 '18 at 12:54
11

I'll do something like this:

for file in *.pkg ; do
    mv $file $(echo $file | rev | cut -f2- -d- | rev).pkg
done

supposed all your file are in the current directory. If not, try to use find as advised above by Javier.

EDIT: Also, this version don't use any bash-specific features, as others above, which leads you to more portability.

Diego Sevilla
  • 27,060
  • 3
  • 52
  • 82
6

Here is a POSIX near-equivalent of the currently accepted answer. This trades the Bash-only ${variable/substring/replacement} parameter expansion for one which is available in any Bourne-compatible shell.

for i in ./*.pkg; do
    mv "$i" "${i%-[0-9.]*.pkg}.pkg"
done

The parameter expansion ${variable%pattern} produces the value of variable with any suffix which matches pattern removed. (There is also ${variable#pattern} to remove a prefix.)

I kept the subpattern -[0-9.]* from the accepted answer although it is perhaps misleading. It's not a regular expression, but a glob pattern; so it doesn't mean "a dash followed by zero or more numbers or dots". Instead, it means "a dash, followed by a number or a dot, followed by anything". The "anything" will be the shortest possible match, not the longest. (Bash offers ## and %% for trimming the longest possible prefix or suffix, rather than the shortest.)

tripleee
  • 139,311
  • 24
  • 207
  • 268
5

We can assume sed is available on any *nix, but we can't be sure it'll support sed -n to generate mv commands. (NOTE: Only GNU sed does this.)

Even so, bash builtins and sed, we can quickly whip up a shell function to do this.

sedrename() {
  if [ $# -gt 1 ]; then
    sed_pattern=$1
    shift
    for file in $(ls $@); do
      mv -v "$file" "$(sed $sed_pattern <<< $file)"
    done
  else
    echo "usage: $0 sed_pattern files..."
  fi
}

Usage

sedrename 's|\(.*\)\(-[0-9.]*\.pkg\)|\1\2|' *.pkg

before:

./Xft2-2.1.13.pkg
./jasper-1.900.1.pkg
./xorg-libXrandr-1.2.3.pkg

after:

./Xft2.pkg
./jasper.pkg
./xorg-libXrandr.pkg

Creating target folders:

Since mv doesn't automatically create target folders we can't using our initial version of sedrename.

It's a fairly small change, so it'd be nice to include that feature:

We'll need a utility function, abspath (or absolute path) since bash doesn't have this build in.

abspath () { case "$1" in
               /*)printf "%s\n" "$1";;
               *)printf "%s\n" "$PWD/$1";;
             esac; }

Once we have that we can generate the target folder(s) for a sed/rename pattern which includes new folder structure.

This will ensure we know the names of our target folders. When we rename we'll need to use it on the target file name.

# generate the rename target
target="$(sed $sed_pattern <<< $file)"

# Use absolute path of the rename target to make target folder structure
mkdir -p "$(dirname $(abspath $target))"

# finally move the file to the target name/folders
mv -v "$file" "$target"

Here's the full folder aware script...

sedrename() {
  if [ $# -gt 1 ]; then
    sed_pattern=$1
    shift
    for file in $(ls $@); do
      target="$(sed $sed_pattern <<< $file)"
      mkdir -p "$(dirname $(abspath $target))"
      mv -v "$file" "$target"
    done
  else
    echo "usage: $0 sed_pattern files..."
  fi
}

Of course, it still works when we don't have specific target folders too.

If we wanted to put all the songs into a folder, ./Beethoven/ we can do this:

Usage

sedrename 's|Beethoven - |Beethoven/|g' *.mp3

before:

./Beethoven - Fur Elise.mp3
./Beethoven - Moonlight Sonata.mp3
./Beethoven - Ode to Joy.mp3
./Beethoven - Rage Over the Lost Penny.mp3

after:

./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3

Bonus round...

Using this script to move files from folders into a single folder:

Assuming we wanted to gather up all the files matched, and place them in the current folder, we can do it:

sedrename 's|.*/||' **/*.mp3

before:

./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3

after:

./Beethoven/ # (now empty)
./Fur Elise.mp3
./Moonlight Sonata.mp3
./Ode to Joy.mp3
./Rage Over the Lost Penny.mp3

Note on sed regex patterns

Regular sed pattern rules apply in this script, these patterns aren't PCRE (Perl Compatible Regular Expressions). You could have sed extended regular expression syntax, using either sed -r or sed -E depending on your platform.

See the POSIX compliant man re_format for a complete description of sed basic and extended regexp patterns.

ocodo
  • 27,324
  • 15
  • 97
  • 113
5

I find that rename is a much more straightforward tool to use for this sort of thing. I found it on Homebrew for OSX

For your example I would do:

rename 's/\d*?\.\d*?\.\d*?//' *.pkg

The 's' means substitute. The form is s/searchPattern/replacement/ files_to_apply. You need to use regex for this which takes a little study but it's well worth the effort.

Simon Katan
  • 628
  • 7
  • 11
  • I agree. For an occasional thing, using `for` and `sed` etc. is fine, but it's much simple and quicker to just use `rename` if you find yourself doing this often. – argentum2f Mar 28 '19 at 15:17
  • Is that you escaping periods? – Big Money Aug 09 '19 at 00:13
  • The problem is that this is not portable; not only do not all platforms have a `rename` command; in addition, some will have a *different* `rename` command with different features, syntax, and/or options. This looks like the Perl `rename` which is fairly popular; but the answer should really clarify this. – tripleee Feb 22 '20 at 12:34
  • Installing rename on ubuntu/debian `sudo apt-get install rename` RedHat derived distros: `sudo dnf install prename` OSX: `brew install rename` – Attila Fulop Apr 13 '21 at 15:37
3

better use sed for this, something like:

find . -type f -name "*.pkg" |
 sed -e 's/((.*)-[0-9.]*\.pkg)/\1 \2.pkg/g' |
 while read nameA nameB; do
    mv $nameA $nameB;
 done

figuring up the regular expression is left as an exercise (as is dealing with filenames that include spaces)

devios1
  • 33,997
  • 43
  • 149
  • 241
Javier
  • 57,083
  • 7
  • 74
  • 120
1

I had multiple *.txt files to be renamed as .sql in same folder. below worked for me:

for i in \`ls *.txt | awk -F "." '{print $1}'\` ;do mv $i.txt $i.sql; done
Nakilon
  • 32,203
  • 13
  • 95
  • 132
1

This seems to work assuming that

  • everything ends with $pkg
  • your version #'s always start with a "-"

strip off the .pkg, then strip off -..

for x in $(ls); do echo $x $(echo $x | sed 's/\.pkg//g' | sed 's/-.*//g').pkg; done
Steve B.
  • 49,740
  • 11
  • 90
  • 128
  • There are some packages that have a hyphen in their name (see third example). Does this impact your code? – Jeremy L Mar 02 '09 at 15:35
  • 1
    You can run multiple sed expressions using `-e`. E.g. `sed -e 's/\.pkg//g' -e 's/-.*//g'`. – tacotuesday Aug 31 '17 at 23:05
  • [Don't parse `ls` output](https://mywiki.wooledge.org/ParsingLs) and [quote your variables.](/q/10067266) See also http://shellcheck.net/ for diagnosing common problems and antipatterns in shell scripts. – tripleee Mar 04 '18 at 19:30
0

Thank you for this answers. I also had some sort of problem. Moving .nzb.queued files to .nzb files. It had spaces and other cruft in the filenames and this solved my problem:

find . -type f -name "*.nzb.queued" |
sed -ne "s/^\(\(.*\).nzb.queued\)$/mv -v \"\1\" \"\2.nzb\"/p" |
sh

It is based on the answer of Diomidis Spinellis.

The regex creates one group for the whole filename, and one group for the part before .nzb.queued and then creates a shell move command. With the strings quoted. This also avoids creating a loop in shell script because this is already done by sed.

Jerry Jacobs
  • 175
  • 9
  • Should be noted that this is only true for GNU sed. On BSDs sed will not interpret the `-n` option in the same way. – ocodo Mar 11 '18 at 02:15