basically tech

10 Shell stuff: rename multiple files on the command line

Tuesday 24th October, 2006

If you wish to quickly rename multiple files in a directory, a for loop (sometimes combined with other utilities such as sed or tr) is one way to do the job.

The examples in this article include removing spaces from filenames, adding and removing suffixes and prefixes, and changing from uppercase to lowercase.

Here is a full list of the examples I'm going to look at:

remove spaces from file names

I've got some imaginary mp3 files with spaces in the file names. Spaces in filenames are a bad idea. It's inconvenient, spaces are the delimiters between command-line arguments, and for files you're putting out over the Internet, it can cause other problems, so we're going to get rid of these spaces as a matter of principle. (Whose foolish idea was it to put spaces into filenames, anyway?)

$ ls -l 

total 468
-rw-r--r-- 1 rob rob  15015 2006-10-20 11:18 01 - Some Song.mp3
-rw-r--r-- 1 rob rob  17313 2006-10-20 11:18 02 - Another Song.mp3
-rw-r--r-- 1 rob rob  17381 2006-10-20 11:19 03 - Yet Another Song.mp3
-rw-r--r-- 1 rob rob 410075 2006-10-20 11:19 04 - A New Song.mp3
-rw-r--r-- 1 rob rob   7327 2006-10-20 11:19 05 - Some Instrumental Piece.mp3

I'm going to change the spaces to underscores. Since I don't want to mess things up, I'll be cautious to begin with:

$ for FILE in *.mp3 ; do NEWFILE=`echo $FILE | sed 's/ /_/g'` ; echo "$FILE will be renamed as $NEWFILE" ; done 

01 - Some Song.mp3 will be renamed as 01_-_Some_Song.mp3
02 - Another Song.mp3 will be renamed as 02_-_Another_Song.mp3
03 - Yet Another Song.mp3 will be renamed as 03_-_Yet_Another_Song.mp3
04 - A New Song.mp3 will be renamed as 04_-_A_New_Song.mp3
05 - Some Instrumental Piece.mp3 will be renamed as 05_-_Some_Instrumental_Piece.mp3

That looks okay, so I'll try it for real.

$ for FILE in *.mp3 ; do NEWFILE=`echo $FILE | sed 's/ /_/g'` ; mv "$FILE" $NEWFILE ; done
$ ls -l 

total 468
-rw-r--r-- 1 rob rob  15015 2006-10-20 11:18 01_-_Some_Song.mp3
-rw-r--r-- 1 rob rob  17313 2006-10-20 11:18 02_-_Another_Song.mp3
-rw-r--r-- 1 rob rob  17381 2006-10-20 11:19 03_-_Yet_Another_Song.mp3
-rw-r--r-- 1 rob rob 410075 2006-10-20 11:19 04_-_A_New_Song.mp3
-rw-r--r-- 1 rob rob   7327 2006-10-20 11:19 05_-_Some_Instrumental_Piece.mp3

I realise that there is an alternative way, using tr, which is easier (to type :) ). However I would argue that the principle of the above technique (using sed) can be applied to more situations, making it more useful as a template.

For those who are unfamiliar with the command line, I'm going to break this down into it's constituent components. If you are familiar with this sort of thing, feel free to skip down to the other examples.

This is not the place to discuss for loops in great detail. There are loads of resources on the Internet for that sort of thing. When I was starting to learn this stuff, I always found detailed analysis of these things confusing, and that the best way to learn was to actually do it.

for FILE in *.mp3

In this part of the loop, FILE is the variable, and *.mp3 is the argument. The for loop generates a variable for each argument until there are no arguments left. In the example above, the variables generated are all the files which match the regular expression *.mp3, in other words: 

01 - Some Song.mp3
02 - Another Song.mp3
03 - Yet Another Song.mp3
04 - A New Song.mp3
05 - Some Instrumental Piece.mp3

; do

The semicolon is special character which allows you end one command and run another command on the same line. Every for requires a do.

NEWFILE=`echo $FILE | sed 's/ /_/g'`

Now I'm creating a new variable called NEWFILE. sed is used to replace each space in the name of the mp3 file referenced by FILE with an underscore (s/ /_/g), so the variable NEWFILE is the same as FILE, except the value of each variable has every space replaced with an underscore.

The next bit is the command we want to run on each argument:

mv "$FILE" $NEWFILE

Note that I put the loop-generated $FILE variable inside double-quote marks for the mv command. This is required because of those spaces in the original file name. If you don't do that the loop will try to run the following command on each file: 

mv 01 - Some Song.mp3 01_-_Some_Song.mp3

instead of 

mv "01 - Some Song.mp3" 01_-_Some_Song.mp3

; done

Finally, every do requires a done.

Here are some other examples.

alternative way to remove spaces from file names

$ ls -l 

total 468
-rw-r--r-- 1 rob rob  15015 2006-10-20 11:18 01 - Some Song.mp3
-rw-r--r-- 1 rob rob  17313 2006-10-20 11:18 02 - Another Song.mp3
-rw-r--r-- 1 rob rob  17381 2006-10-20 11:19 03 - Yet Another Song.mp3
-rw-r--r-- 1 rob rob 410075 2006-10-20 11:19 04 - A New Song.mp3
-rw-r--r-- 1 rob rob   7327 2006-10-20 11:19 05 - Some Instrumental Piece.mp3

$ for FILE in *.mp3 ; do mv "$FILE" `echo $FILE | tr ' ' '_'` ; done
$ ls -l 

total 468
-rw-r--r-- 1 rob rob  15015 2006-10-20 11:18 01_-_Some_Song.mp3
-rw-r--r-- 1 rob rob  17313 2006-10-20 11:18 02_-_Another_Song.mp3
-rw-r--r-- 1 rob rob  17381 2006-10-20 11:19 03_-_Yet_Another_Song.mp3
-rw-r--r-- 1 rob rob 410075 2006-10-20 11:19 04_-_A_New_Song.mp3
-rw-r--r-- 1 rob rob   7327 2006-10-20 11:19 05_-_Some_Instrumental_Piece.mp3

add a suffix to multiple files

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices

In this example, I wish to add a .txt suffix to each file in this directory.

We don't need sed here:

$ for FILE in * ; do mv $FILE $FILE.txt ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices.txt

add a prefix

Taking the files from the previous example:

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices.txt

I wish to add a maths_ prefix to each file:

$ for FILE in * ; do mv $FILE maths_$FILE ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 maths_algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 maths_calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 maths_equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 maths_geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 maths_matrices.txt

remove a prefix

Sticking with the files we've been using:

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 maths_algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 maths_calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 maths_equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 maths_geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 maths_matrices.txt

I've changed my mind about that maths_ prefix and want to remove it.

$ for FILE in * ; do NEWFILE=`echo $FILE | sed 's/^maths_//'` ; mv $FILE $NEWFILE ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices.txt

The ^ symbol in sed matches the start of a line.

remove a suffix

While I'm at it, I think I'll remove the suffix as well.

$ for FILE in *.txt ; do NEWFILE=`echo $FILE | sed 's/.txt$//'` ; mv $FILE $NEWFILE ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices

The $ symbol in sed matches the end of a line.

uppercase to lowercase

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 ALGEBRA
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 CALCULUS
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 EQUATIONS
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 GEOMETRY
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 MATRICES

$ for FILE in * ; do mv $FILE `echo $FILE | tr '[A-Z]' '[a-z]'` ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices

uppercase to lowercase (suffix only)

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 Algebra.TXT
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 Calculus.TXT
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 Equations.TXT
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 Geometry.TXT
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 Matrices.TXT

$ for FILE in *.TXT ; do NEWFILE=`echo $FILE | sed 's/.TXT$/.txt/'` ; mv $FILE $NEWFILE ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 Algebra.txt
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 Calculus.txt
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 Equations.txt
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 Geometry.txt
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 Matrices.txt

an unlikely situation

Finally, an unlikely situation. I have a bunch of files with multiple spaces and capitalised suffixes. I want the spaces changed to an underscore, except where there are two or more consecutive spaces, where I want just a single underscore. While we're at it, if a space is next to the dot of a prefix, let's just remove it. I also want to make the suffixes lowercase. Hmm.

$ ls -l 

total 20
-rw-r--r-- 1 rob rob  201 2006-10-23 23:44 A file  with many   spaces.TXT
-rw-r--r-- 1 rob rob 1579 2006-10-23 23:44 Another wierdly named  file.TXT
-rw-r--r-- 1 rob rob 1452 2006-10-23 23:44 My mate Arron spels  beter than this.txt
-rw-r--r-- 1 rob rob  924 2006-10-23 23:44 Not really.TXT
-rw-r--r-- 1 rob rob  379 2006-10-23 23:44 Whoo named this .TXT

I'm not going to try to correct the spelling as well!

$ for FILE in * ; do NEWFILE=`echo $FILE | sed -e 's/.TXT$/.txt/' -e 's/[ ]*[ ]/_/g' -e 's/_[.]/./g'` ; mv "$FILE" $NEWFILE ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob  201 2006-10-23 23:44 A_file_with_many_spaces.txt
-rw-r--r-- 1 rob rob 1579 2006-10-23 23:44 Another_wierdly_named_file.txt
-rw-r--r-- 1 rob rob 1452 2006-10-23 23:44 My_mate_Arron_spels_beter_than_this.txt
-rw-r--r-- 1 rob rob  924 2006-10-23 23:44 Not_really.txt
-rw-r--r-- 1 rob rob  379 2006-10-23 23:44 Whoo_named_this.txt

That was somewhat contrived, but it does provide an example of the power of the command line.

Bear in mind that some people pay for this sort of functionality. Mind you, consider their option.

addendum

It occurred to me after I wrote this article that some one might want to emulate the windows option.

Giving all the files the same name, but with a different number seems, well, foolish, but let's do it anyway. Parentheses are also a bad idea in filenames, but this is just an exercise.

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices

$ NUM=0 ; for FILE in * ; do NUM=`expr $NUM + 1` ; mv $FILE foolish\($NUM\) ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 foolish(1)
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 foolish(2)
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 foolish(3)
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 foolish(4)
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 foolish(5)

What might be more useful would be to retain the original name and seperate it from the sequential number with a much more command-line friendly underscore.

$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices

$ NUM=0 ; for FILE in * ; do NUM=`expr $NUM + 1` ; mv $FILE ${FILE}_$NUM ; done
$ ls -l 

total 20
-rw-r--r-- 1 rob rob 733 2006-10-20 13:23 algebra_1
-rw-r--r-- 1 rob rob 194 2006-10-20 13:23 calculus_2
-rw-r--r-- 1 rob rob 117 2006-10-20 13:23 equations_3
-rw-r--r-- 1 rob rob 402 2006-10-20 13:23 geometry_4
-rw-r--r-- 1 rob rob  50 2006-10-20 13:23 matrices_5

Happy renaming. :)

addendum 2

Oct 25 2006: I have added more multiple file renaming techniques in a seperate post. These alternative techniques cover removing and changing suffixes and prefixes.

Home

  command line   tech tips