Stop Using FTP!

The legendary File Transfer Protocol has served us well over the years, but it's time to put the old horse out to pasture.

One of our technical support guys, through no fault of his own, wasted many hours of his own and other people's time trying to figure out why an innocuous looking shell script would not execute.

The script is executed from within a complex enterprise management application.  Because we could not execute it in a simple terminal command line environment, and because there are several constraints on how the script is named and installed, we did countless iterations to try to isolate what wasn't working.

We did comparison runs on other systems with similar scripts, we injected debug output, we double-checked SELinux file contexts, and we stared a lot.  There was no sign that the management application was even attempting to run the script.  No errors either.  It's like the script wasn't there.

Finally, it occurred to me to just write a new script:

#!/bin/bash
echo hello

We installed the new script with the prescribed procedure, prepped the system, and repeated our test.

hello

It worked. Finally, we were on to something. There was something in the original script that was causing problems. Something invisible.

I ran od -bc on the original script so I could see the binary representation of the file. There it was: ASCII carriage returns before the linefeed characters.

0000000 043 041 057 142 151 156 057 142 141 163 150 015 012 145 143 150
          #   !   /   b   i   n   /   b   a   s   h  \r  \n   e   c   h
0000020 157 040 150 145 154 154 157 015 012
          o       h   e   l   l   o  \r  \n
0000031

"Where did you get the script from?" I asked. I knew what he was going to say.

"One of the developers emailed it to me."

"And how did you transfer it onto the test system?"

"With scp."

"How did the file get onto the host where you transferred it from?"

"Jack copied it there for me. I mailed it to him, he saved the attachment, and then he FTP'd it to the server from his desktop."

Good grief. What a route for a file to take. Jack overheard the conversation and walked over.

"What kind of file transfer program did you use to get this script onto the server, Jack?"

"I used FileZilla on my Windows box."

Memories of writing FTP code back in the mid-90's came flooding back. So did memories of DOS-style line endings. And so did FTP's well-meaning attempts to make life easier for those who transfer files between different operating systems.

I explained to them that FTP is really not a good protocol to be using in a modern Linux world -- especially a world where Microsoft products live. FTP programs can and will alter file contents to be compatible with the receiving OS. Unix systems want lines to end with a single "newline" (linefeed) character, whereas DOS/Windows uses the "carriage return + newline". A holdover from the bygone days of the line printer.

An FTP program can be explicitly told what to do, but casual users often don't bother. The result can be confusing.

FTP probably still has its niche uses. Anonymous FTP servers that don't require users to authenticate with plaintext passwords might be acceptable. But that brings up the issue of running yet another process on an open port that could be exploited. I recommend that you just use scp (on Linux) and something like WinSCP on Windows.

Let's leave FTP to graze in fields of tall grass.