How to run a command over SSH: quoting, exit codes

Quoting: which shell reads what

Two shells read your command: yours first, and then the one on the server. Quotes decide which shell expands what.

NAME=local
ssh server "echo $NAME"       # local shell expands it, prints: local
ssh server 'echo $HOME $NAME' # server expands it, prints: /home/tester
                              # NAME does not exist on the server

ssh joins its arguments with single spaces and sends one string, so quotes that only your local shell sees are lost:

$ ssh server echo "a  b"
a b
$ ssh server 'echo "a  b"'
a  b

The same applies to pipes. The first command below runs wc on the server, and the second runs it on your machine:

ssh server 'ls /tmp | wc -l'
ssh server ls /tmp | wc -l

To pass a local value that contains spaces or quotes, let bash escape it:

F="a b'c"
ssh server "printf '%s\n' $(printf %q "$F")"

Several lines: a heredoc

Quote the heredoc marker and nothing is expanded locally:

ssh server 'bash -s' <<'REMOTE'
echo "user: $(whoami)"
cd /tmp && pwd
REMOTE

Without quotes around the marker, your local shell expands variables first. Escape the ones meant for the server:

V=abc
ssh server 'bash -s' <<REMOTE
echo "local var: $V, remote: \$HOSTNAME"
REMOTE

To run a local script on the server with arguments:

ssh server 'bash -s -- one two' < script.sh

Exit codes

Command	Exit code
`ssh server 'exit 3'`	3
`ssh server 'false; true'`	0, because only the last command counts
`ssh server 'set -e; false; echo no'`	1, and it stops at `false`
`ssh server nosuchcmd`	127, command not found on the server
`ssh -p 1 server true`	255, SSH could not connect

A remote command can also exit with 255 by itself, so 255 only suggests an SSH failure. Standard error stays separate from standard output, so 2>/dev/null and > file work as usual on your side.

Why ssh in a loop stops after one host

ssh reads standard input and sends it to the server, so it eats the rest of the list. This loop ran only once in our test, with three lines in the file:

while read -r h; do ssh "$h" uptime; done < hosts

-n tells ssh not to read standard input. This one ran three times:

while read -r h; do ssh -n "$h" uptime; done < hosts

Before you loop over production servers, read how to avoid mistakes on production servers.

Sending data through standard input

Without -n, standard input is useful. Remote commands can read it:

printf 'hello\n' | ssh server 'tr a-z A-Z'
tar -cf - folder | ssh server 'tar -C /tmp -xf -'

Commands that need a terminal

With a command, ssh does not allocate a terminal, so tty prints not a tty and sudo cannot ask for a password:

sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper

Add -t for interactive programs:

ssh -t server sudo systemctl restart nginx

If your own standard input is not a terminal, for example in a script, -t prints Pseudo-terminal will not be allocated because stdin is not a terminal. Use -tt to force it, and see that error's guide.

Long jobs and background commands

A background job that still holds the output open keeps ssh waiting. The first command below took 2 seconds in our test, and the second returned at once:

ssh server 'sleep 2 &'
ssh server 'nohup sleep 2 >/dev/null 2>&1 &'

For real work that must outlive the connection, use tmux on the server. ssh -f server 'command' only puts your local ssh in the background after login.

A command over SSH does not run in a login shell, so files such as ~/.profile are not read. In our test shopt login_shell printed off and PATH was /usr/local/bin:/usr/bin:/bin:/usr/games. If a command works by hand but reports command not found over SSH, use its full path.

Questions

How do I run sudo over SSH without the terminal error?

Add -t, as in ssh -t server sudo systemctl restart nginx, so sudo gets a terminal to ask for the password. In scripts use -tt or configure sudo for that command.

Why does a command work by hand but not over SSH?

A command over SSH runs in a non-login shell, so ~/.profile is not read and PATH is shorter. Use the full path of the program.

How do I run several commands in one SSH call?

Put them in one quoted string, such as ssh server 'cd /tmp && ls', or send a script with ssh server 'bash -s' < script.sh.

Sources

ssh(1), OpenBSD manual

From the blog

All guides