Often users of CGI scripts encounter the dreaded “premature end of script headers” error. A quick Google search on that phrase proves this to be true. The cause of this, is often the same as the “bad interpreter” error received on the command line.

A very common cause is where the file was created on a Windows host and then uploaded to a Unix host for execution. (Think of the millions of websites using shared hosting.) The problem here, as no doubt you maybe aware, is that the file contains the dreaded “carriage return” before every newline. This is so common, there is even a standard command dos2unix for converting these files.

However, I had yet to see a reason as to WHY that carriage return after the hash bang causes a problem. I just accepted it as fact. Today, I decided to figure out why and where it failed.

As it turns out, it fails in the kernel. To figure this out, I downloaded the latest kernel (2.6.23.12) to my RHEL 5 host and compiled it with the standard config file that comes with RHEL 5. The configuration file was for a much older kernel, but worked for my purposes. I booted it to make sure I had a working copy of 2.6.23.12. Then I went into the kernel source tree and started modifying stuff!

To be sure, I am no kernel hacker, nor even a C hacker. I had never successfully modified the kernel source, though I haven’t tried in many years. But after three compilations, I was able to figure out where in (2.6.23.12) exec system call was taking place and was able to insert some code to print a custom message. The file I modifed was “fs/exec.c”:

[root@test1 linux-2.6.23.12]# ls -l fs/exec.c
-rw-rw-r– 1 root root 42231 Jan 7 17:44 fs/exec.c

In that file, inside the function search_binary_handler() I inserted the following code:

1139 printk(”{”);
1140 int z = 0;
1141 for(; z < strlen(bprm->buf); z++) {
1142 if(bprm->buf[z] == ‘\r’) {
1143 printk(”(carriage return)”);
1144 } else {
1145 printk(”%c”, bprm->buf[z]);
1146 }
1147 }
1148 printk(”}\n”);

The line numbers may be slightly off as I had inserted and removed code above. After compliation and booting, I was able to give it a shot. I created a sample script with a carriage return after the hash bang interpreter portion.

[root@test1 ~]# echo -e ‘#!/bin/bash\r\n/usr/bin/id’ >id.sh
[root@test1 ~]# chmod +x id.sh

And BAM, it worked!

[root@test1 ~]# ./id.sh
{#!/bin/bash(carriage return)}
-bash: ./id.sh: /bin/bash^M: bad interpreter: No such file or directory

As you can see, the kernel was actually trying to execute “/bin/bash\r” which of course, not a valid file. Here’s a screen shot:

Why do CGI Scripts and Shell Scripts fail when they contain carriage returns

Update: In response to reader requests, I wrote an explanation of my investigation: On the case of carriage returns and kernel exec system calls.

9 Responses to “Why do CGI scripts and shell scripts fail when they contain carriage returns?”

  1. admin Says:

    I must _really_ be a geek. When typing this up, I realized I missed the LSU vs Ohio State game. Oh well, I learned something new and the team I wanted to win, won!

  2. dan Says:

    So, hopefully, someone will submit this change to the linux “powers-to-be” so they will fix the bug in the kernel, *ahem* I mean, modify the feature-set… ;>)

    Dan

  3. Stan Says:

    I’d be interested in the process by which you narrowed it down to the exec.c file. You jumped straight to the catch, but I wanted to see the chase.

  4. admin Says:

    Stan,

    Thats a great idea… I should have done so. I will write it up tonight and post it.

  5. admin Says:

    Also, note that new version of bash try and help you out by adding the ^M to the output of the error if there is a carriage return there. I believe this to be a recent addition.

  6. Stan Says:

    Much obliged, sir. I look forward to reading your next post about this.

  7. On the case of carriage returns and kernel exec function calls Says:

    […] I wrote a post titled “Why do CGI scripts and shell scripts fail when they contain carriage returns?” I got a comment and a few emails saying in the words of Stan “I’d be interested in […]

  8. goll Says:

    great article, but like Stan said, it would be helpful if you showed us how you found that this is the file :)

  9. admin Says:

    Goll,

    Agreed and thanks for the comment! I wrote a second article which explains this:

    On the case of carriage returns and kernel exec system calls

Leave a Reply

If Wordpress eats your comment (shell output, loops, ex..) email the text to me.