I’ve written before about the extreme usefulness of sshfs for accessing files remotely without having to install server software. It continues to be an important part of my toolbox, as does its cousin CurlFtpFS which – unlike sshfs – I can even use to mount a directory here on my web host. Of course, either becomes even more useful when combined with some easy method of synchronization. You probably have rsync already. You can also use Unison if you need bidirectional synchronization – which you generally will if you’re trying to use a single directory somewhere as a “drop box” to share between multiple machines.
I just found another sshfs trick today. If the connection between your two machines is already secure – e.g. on the same private network or connected via a secure VPN/tunnel – you might want to avoid an extra round of encryption and decryption by using the “-o directport” option to sshfs. This causes sshfs to bypass all of the ssh stuff and just create a simple TCP connection . . . but what should you run at the other end? Here’s where socat (another extra-useful tool) comes in handy. On the server end (assuming a pretty standard Linux setup), you can just run this:
socat TCP4-LISTEN:7777 EXEC:/usr/lib/sftp-server
Then, on the client:
sshfs -o directport=7777 remote:/dir /local/dir
Now you have a completely insecure TCP between the two machines, so you’d really better not do this directly over the internet without some other way of securing things at a lower level. It’s still pretty handy, though, and I couldn’t find a mention anywhere else of how to do it so there it is.
cute trick! I remember sshv1 (not openssh) had a -c none option to use a null cypher.
Just for fun, I tweaked my local copy of sftp-server to set up the port itself, avoiding context switches between itself and socat. It took me about fifteen minutes (less time than it took to download/configure/build the original) and performance is predictably better.
“sftp-server to set up the port itself, avoiding context switches between itself and socat”
For a one-shot server, you might also use
$ netcat -l -p 7777 -e /usr/lib/sftp-server
Because netcat doesn’t seem to fork(), just exec().
For multiple server instances: I figure sftp-server is (x)inetd-compatible.
Netcat is pretty much equivalent to socat, except for having a lot fewer options. Most importantly, it would have to fork, or else the exec would wipe out the netcat image. Thus, it would have exactly the same context-switch behavior as the socat example. Running from inetd works, so long as you have root and you’re running inetd anyway (not all do or want to), but I’m not sure I could recommend it. Leaving connections to an insecure port enabled would be unwise, and having to do a second config-edit/restart to disable it once you’ve made your own connection would be inconvenient. The nice thing about the socat or hacked-server approaches is that they only accept one connection and then the port’s unreachable again.
1. I wasn’t necessarily promoting the inetd setup, just mentioning it for the case when more convenience is needed with multiple sshfs mounts to (different directories) of the same server. I do like single-shot servers, for the reasons you cite.
2a. The netcat example. First, before posting it, I did try it out and checked the PID’s. I used “-c sleep 10″ in place of “-e sftp-server”. The sleep process replaced the netcat process. What’s more, because of -c, there was even a /bin/sh (bash) -c between them. If you pass a single simple command to bash -c, bash executes it as if with the exec builtin. So the bash image replaced the netcat image, and then the sleep image replaced the bash image.
2b. I also tried out the exact sftp-server example. with The netcat image was in fact wiped out by the sftp-server image, but that’s OK because the new image inherits, over exec(), the file descriptors of the original image. (Except for those marked with FD_CLOEXEC.)
This is how the socat example operates:
a) socket(), bind(), listen(), accept()
b) pipe() x 2
c) fork()
d1) child: inherits all fd’s. By way of dup2(), makes its fd0 point to the reading end of one pipe, and makes its fd1 point to the writing end of the other pipe. Closes all fd’s except fd0 and fd1. Then it executes sftp-server, which replaces the socat image running in the child, and inherits fd0 and fd1, both pointing to pipes.
d2) parent: closes unused pipe ends, select()s on the new TCP socket for reading and writing, selects() on one pipe for reading, on the other for writing, copies data.
d3) OS switches between parent (original socat process) and child (original socat child turned to sftp-server) as data passes over the pipes
Netcat example:
a) socket(), bind(), listen(), accept() — same as above
b) dup2(accepted_sock, 0); dup2(accepted_sock, 1); close(server_sock); close(accepted_sock); exec();
c) When the sftp-server image uses its fd0 and fd1, it directly accesses the socket.
I’d also like to use the opportunity to advertise the “*three* levels of a file”. There’s a double indirection between file descriptor and file: file description comes in between.
file descriptor:
- referred to as 0, 1, 2 by the process in appropriate syscalls
- FD_CLOEXEC is a flag on this level
- dup2(oldfd, newfd) makes newfd point at the file description that oldfd points at.
- process private (thread shared)
- disappears if closed
file description:
- kernel structure, not directly accessible for the process
- attributes: file offset, access mode (eg. O_RDWR, O_APPEND), locks, etc
- process shared
- disappears if all file descriptors (across all processes) pointing here are closed
- this is why you can do this:
( /bin/echo one; /bin/echo two ) > file
First echo dies before second echo is forked / executed, we are not in append mode, “two” still appears at the end of regular file “file”. Why? Because the file description survives (due to the shell keeping a reference to it by way of its fd1) the death of the first echo, and so the file offset survives with it. The second echo inherits fd1 from the shell, which refers to the same file description. (Or more exactly, the second echo starts out with an fd1 that points at the same file description where its parent’s fd1 points at.)
file:
- file-description shared
- inode (regular file, socket, char device etc.)
- attributes: type specific (eg. socket options for sockets, file size for regular files)
- disappears if all file descriptions pointing here are closed AND (if regular file) all file system references are removed (link number goes to 0) AND (if regular file) all mmap references are unmapped etc.
( sleep 5; /bin/echo one ) >f &
( sleep 15; /bin/echo two ) >>f &
wait
In this case, two different file descriptions come into play, only the file is common. O_APPEND gets set in the second file description, so it works.
Sorry for this long rant!
Socat disposes over a “nofork” option (in the FORK option group), as well.
Yes, I had overlooked the fact that netcat/socat would have the right file-descriptor state for a fork-less invocation of sftp-server to work. Thanks for pointing that out. The longer explanation of how file descriptors and inodes work might be useful to some other reader too, so thanks for writing that as well.