Five Quick Command Line Tips

January 18th, 2008

These are my tips for the week. Have you any tips to share?

Create many files of random size

$ for i in $(seq 1 1000); do let "size = ($RANDOM/100)"; dd if=/dev/urandom \
of=file-$i count=$size 2>/dev/null;done

Delete files with find, faster

Many people use find to delete files. However, they often to so like this:

find directory conditionals -exec rm {} \;

Which certainly works. However, find has a built-in “-delete” action which is significantly faster.

$ find . -name 'file-*'| wc -l
1000
$ time find . -name 'file-*' -exec rm {} \;

real    0m2.540s
user    0m1.076s
sys     0m1.196s
$ find . -name 'file-*'| wc -l
0

$ find . -name 'file-*'| wc -l
1000
$ time find . -name 'file-*' -delete

real    0m0.118s
user    0m0.008s
sys     0m0.100s
$ find . -name 'file-*'| wc -l
0

Finding more fun

Often those new to the shell do something like this to find the size of files:

$ ls -la | awk '/^\-/ {print $5}'
98816
99328
99840
99840
99840

Or the last modification date:

$ ls -la | awk '/^\-/ {print $6, $7}'
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03

Find offers a much clean interface to this information:

$ find . -maxdepth 1 -type f -printf "%s\n"
98816
99328
99840
99840
99840

$ find . -maxdepth 1 -type f -printf "%TY-%Tm-%Td %TH:%TM\n"
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03
2008-01-17 23:03

Standard input can only be read once

$ echo a > a
$ echo b > b
$ echo c > c

That is, this:

$ cat a b c | cat -
a
b
c

Is the same as:

$ cat a b c | cat - -
a
b
c

Meaning that standard input is “read desctructive.” As Pádraig Brady said recently on the core utils mailing list:

$ echo mouse | cat - -
mouse

Odd characters showing up in your terminal when using Putty

Go to your settings:

Window
Translation
Change “Recieved data assumed to be in which character set” to UTF-8

Update: I changed the first examples when finding last modification date to use awk instead of grep and awk. Hard to change my ways!

Category: Beginners, Good Practice, Shell

10 Responses to “Five Quick Command Line Tips”

thecubic Says:
January 18th, 2008 at 12:27 am
Use stdin/stdout when stdin/stdout isn’t coded:

$ unzip -l - < r.zip
UnZip 5.52 of 28 February 2005, by Info-ZIP. Maintained by C. Spieler. Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]

$ unzip -l /dev/stdin < r.zip
Archive: /dev/stdin
Length Date Time Name
——– —- —- —-

Quickly access a numbered word list:

$ echo file{1..5}
file1 file2 file3 file4 file5
Jeremy Says:
January 18th, 2008 at 9:13 pm
One I seem to use a lot is !$ This is the last string you typed. Best used if looking for something in a directory put you aren’t in that directory

$ ls /var/www/html/
index.html

$ vi !$index.html

Saves you some typing.

Another is cd - which takes you to the last directory you were in.

-Chino
admin Says:
January 18th, 2008 at 10:38 pm
thecubic,

Yes, BASH expansion is often much better than using seq. I often forget BASH will do this automagically!

Jeremy,

I love !$…its saves an unbelievable amount of mouse/keyboard work!

Thanks guys!
eliott Says:
January 24th, 2008 at 2:19 pm
If you use the alternate exec form, find will build longer lists of files to pass to rm, and it is faster than the default exec behavior.

user@box:~/test$ time find . -name “*.txt” -exec rm {} \;

real 0m18.198s
user 0m9.129s
sys 0m9.005s
user@box:~/test$ time find . -name “*.txt” -delete

real 0m0.207s
user 0m0.012s
sys 0m0.196s
user@box:~/test$ time find . -name “*.txt” -exec rm {} \+

real 0m0.250s
user 0m0.012s
sys 0m0.232s

delete is still faster for delete, but for other operations, using the + can speed things up considerably..granted the utility can accept more than one file arg at a time.
ConcernedFellowBlogger Says:
January 24th, 2008 at 5:59 pm
Your copyrighted content has been stolen:

http colon slash slash googit.blogspot.com/2008/01/five-quick-command-line-tips.html
admin Says:
January 24th, 2008 at 6:56 pm
ConcernedFellowBlogger,

Thanks…..
admin Says:
January 24th, 2008 at 7:03 pm
eliott,

Good tip, thanks!
redduck666 Says:
February 23rd, 2008 at 3:21 am
regarding your `Finding more fun’, stat(1) can be used to do the same thing more elegantly. it is not portable, but judging by the rest of your content i’d say you don’t care about that
admin Says:
February 23rd, 2008 at 6:21 pm
redduck666,

Thanks for the tip! The version of stat I had when writing the article did not have the printf option on stat. I am not sure if its new or I just missed it.

Yes, I am not overly concerned about a portability. I generally just write for the modern versions of coreutils and associated commands that come with modern versions of Linux.
Fireside Chats » Blog Archive » Useful Unix Commands Says:
June 29th, 2008 at 11:24 am
[…] Five Quick Command Line Tips […]

BASH Cures Cancer