BASH Cures Cancer

I am making some changes to the moreutils sponge command. Sponge provides a method of prepending which is less specialized than my prepend util. However, it has trouble with large amounts of input.

Regardless, while testing my changes, I want to watch it operate. Normally, you would just do so from a second terminal. That is a pain. kill -0 can be very useful for this. After backgrounding the command, I assign the pid (via the variable $!) to $pid using eval. eval is needed to stop BASH from expanding $! until after the background operation.

After that, I enter a while loop on kill -0 $pid, which will not kill $pid, but will return successfully until $pid has died:

# cat large-file-GB | ./sponge large-file-GB-copy & eval 'pid=$!'; while kill -0 $pid; do sleep 10; ls -lh large-file* /tmp/sponge.*; echo;done
[1] 7937
-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw------- 1 root root 128M 2008-04-09 17:23 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw------- 1 root root 384M 2008-04-09 17:23 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw------- 1 root root 877M 2008-04-09 17:24 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw-r--r-- 1 root root  20M 2008-04-09 17:24 large-file-GB-copy
-rw------- 1 root root 896M 2008-04-09 17:24 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw-r--r-- 1 root root 413M 2008-04-09 17:25 large-file-GB-copy
-rw------- 1 root root 896M 2008-04-09 17:24 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw-r--r-- 1 root root 836M 2008-04-09 17:25 large-file-GB-copy
-rw------- 1 root root 896M 2008-04-09 17:24 /tmp/sponge.JMsBWG

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw-r--r-- 1 root root 920M 2008-04-09 17:25 large-file-GB-copy
[1]+  Done                    cat large-file-GB | ./sponge large-file-GB-copy
ls: cannot access /tmp/sponge.*: No such file or directory

-rw-r--r-- 1 root root 977M 2008-04-09 16:18 large-file-GB
-rw-r--r-- 1 root root 977M 2008-04-09 17:25 large-file-GB-copy
-bash: kill: (7937) - No such process
# md5sum large-file-GB*
b5c667a723a10a3485a33263c4c2b978  large-file-GB
b5c667a723a10a3485a33263c4c2b978  large-file-GB-copy

I am utilizing Google’s project hosting to host software which I create and feel is useful or want to keep track of. I called the project Brock’s Tools. The code that led me to create this project was a command I am calling prepend 1.1. (UPDATE: See this post on sponge as its a better general case tool.)

prepend, prepend’s files or standard input to a file. For example,  you have three files:

$ echo BROCK > a
$ echo DAVID > b
$ echo NOLAND > c

And you want to combine them into one file:

$ echo "My name is:" | prepend - a b c
$ cat c
My name is:
BROCK
DAVID
NOLAND

Or lets say you just want to append a file to itself:

$ cat a
BROCK
$ cat a >> a
cat: a: input file is output file

prepend does this:

$ prepend a
$ cat a
BROCK
BROCK

I come across the a situation where this would be useful quite often. Of course prepend’ing can be done in the shell:

$ { echo "My name is:"; cat a b c; } > tmp && mv -f tmp c
$ cat c
My name is:
BROCK
DAVID
NOLAND

However, that is unsafe and I have lost data that way. I perform this operation most often when dealing with XML. In this example, its trivial to open the file in an editor, but with a large file, its quite nasty to do so:

$ cat something.xml
<entry><blah/><more>stuff 1</more></entry>
<entry><blah/><more>stuff 2</more></entry>
<entry><blah/><more>stuff 3 </more></entry>
<entry><blah/><more>stuff 4</more></entry>
$ echo "</entries>" >> something.xml
$ cat something.xml
<entry><blah/><more>stuff 1</more></entry>
<entry><blah/><more>stuff 2</more></entry>
<entry><blah/><more>stuff 3 </more></entry>
<entry><blah/><more>stuff 4</more></entry>
</entries>
$ echo "<entries>" | prepend - something.xml
$ cat something.xml
<entries>
<entry><blah/><more>stuff 1</more></entry>
<entry><blah/><more>stuff 2</more></entry>
<entry><blah/><more>stuff 3 </more></entry>
<entry><blah/><more>stuff 4</more></entry>
</entries>