SSH agent forwarding and screen

ssh-agent-forwarding-screen.md

SSH agent forwarding and screen

When connecting to a remote server via SSH it is often convenient to use SSH agent forwarding so that you don't need a separate keypair on that server for connecting to further servers.

This is enabled by adding the

ForwardAgent yes

option to any of your Host entries in ~/.ssh/config (or alternatively with the -A option). Don't set this option in a wildcard Host * section since any user on the remote server that can bypass file permissions can now als use keys loaded in your SSH agent. So only use this with hosts you trust.

The problem with screen

Unfortunately, this doesn't work as-is with GNU screen. On every new SSH connection, agent forwarding is setup via a socket specified in the SSH_AUTH_SOCK environment variable (usually somewhere in /tmp). So the socket location will be different on each connection. However, your typical screen session will live over several SSH connections and the shells in your screen session won't know where to find the current socket (their environments are not updated).

Fixing agent forwarding with screen

A simple fix is to symlink to the current socket from a fixed location on each new connection and have SSH look for the socket in that fixed location (specified by the SSH_AUTH_SOCK environment variable). We'll use ~/.ssh/ssh_auth_sock for the symlink location.

To have SSH within a screen session use the symlink, add the following line to ~/.screenrc:

setenv SSH_AUTH_SOCK $HOME/.ssh/ssh_auth_sock

To update the symlink we'll use the ~/.ssh/rc file which is executed by SSH on each connection. This can be any executable file, so something like the following script will do:

if test "$SSH_AUTH_SOCK" ; then
    ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
fi

Unfortunately, this will break X11 forwarding because SSH runs xauth on each connection, except when there is a ~/.ssh/rc file. We can fix this by running xauth from our ~/.ssh/rc as suggested in the sshd(8) manual page.

This is our complete ~/.ssh/rc file:

#!/bin/bash

# Fix SSH auth socket location so agent forwarding works with screen.
if test "$SSH_AUTH_SOCK" ; then
    ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
fi

# Taken from the sshd(8) manpage.
if read proto cookie && [ -n "$DISPLAY" ]; then
        if [ `echo $DISPLAY | cut -c1-10` = 'localhost:' ]; then
                # X11UseLocalhost=yes
                echo add unix:`echo $DISPLAY |
                    cut -c11-` $proto $cookie
        else
                # X11UseLocalhost=no
                echo add $DISPLAY $proto $cookie
        fi | xauth -q -
fi

Credits go to this blog post: Managing SSH Sockets in GNU Screen

An alternative to using a ~/.ssh/rc file is simply to use the following inside ~/.bashrc:

if [[ -S "$SSH_AUTH_SOCK" && ! -h "$SSH_AUTH_SOCK" ]]; then
    ln -sf "$SSH_AUTH_SOCK" ~/.ssh/ssh_auth_sock;
fi
export SSH_AUTH_SOCK=~/.ssh/ssh_auth_sock;

This checks if a socket exists, and it's not a symlink already. This does the same as everything above, as long as ForwardAgent yes or ssh -A is used.

Credit goes to http://superuser.com/a/424588/237155

One other refinement is to test for the presence of SSH_TTY in the environment — I frequently do non-interactive ssh commands and without the following check it will overwrite the link to a non-existing agent socket:

if [[ -n "$SSH_TTY" && -S "$SSH_AUTH_SOCK" ]]; then
    ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
fi

@drd I may be mistaken, but my simplistic tests would indicate that while $SSH_TTY is not set for non-interactives, neither is $SSH_AUTH_SOCK. e.g.: with ForwardAgent Yes I get the following:

 ** root@foobar ** ~ ** Wed Feb 18 16:52:26
# ssh [email protected] 'pwd; echo $SSH_TTY; echo $SSH_AUTH_SOCK; echo $SSH_CONNECTION'
/usr/home/user


204.128.227.42 58307 204.128.227.38 22

I only seek understanding; but this would indicate to me that in the case of a non-interactive ssh command, the '-S' socket test on $SSH_AUTH_SOCK would fail and, as such, not require the additional $SSH_TTY test?

What am I missing?
btw: both boxen are FreeBSD. The FBSD core committers change some of the ssh defaults to enhance security which might explain the differences if Linux exhibits differently?

Thanks for the post! This gist helped me get this working right:

https://github.com/bluefeet/rassh

The issue is when you have old ssh sockets that have not expired yet.
I use following in my .bashrc

export SSH_AUTH_SOCK=$(find /tmp -maxdepth 2 -type s -name "agent*" -user $USER -printf '%T@ %p\n' 2>/dev/null |sort -n|tail -1|cut -d' ' -f2)

And then i just reread my .bashrc in screen windows to apply working ssh socket.

Bless you.

@bruno- you think it's possible to make a tmux plugin to do this?

@kshcherban Have you or anyone figured out a way to automate this process and have it work when you reattach to an existing screen session, without having to manually run it?

I don't think screen allows you to run a command upon reattaching.

The best solution I came up with is alias git='. ~/.bashrc; git' but that seems wasteful and doesn't work if I use composer instead of git first, for example.

See alternative https://chrisdown.name/2013/08/02/fixing-stale-ssh-sockets-in-tmux.html.

I've been using http://samrowe.com/wordpress/ssh-agent-and-gnu-screen/ to solve some of this but it fails (after re-attach) if I then ssh to another computer and from that computer be expected to then ssh to yet another.

ssh to computer 1
start screen
ssh to computer 2 (confirm can ssh to computer 3 via forwarding; exit back to computer 2)
detach
log off computer 1
ssh to computer 1
re-attach screen

expect to be on computer 2

ssh to computer 3 (fails)

Will this method fair better?

Hybrid solution for TMUX based on @kshcherban comment:

.bashrc

# Launch SSH agent if not running
if ! ps aux |grep $(whoami) |grep ssh-agent |grep -v grep >/dev/null; then ssh-agent ; fi

# Link the latest ssh-agent socket
ln -sf $(find /tmp -maxdepth 2 -type s -name "agent*" -user $USER -printf '%T@ %p\n' 2>/dev/null |sort -n|tail -1|cut -d' ' -f2) ~/.ssh/ssh_auth_sock

export SSH_AUTH_SOCK=~/.ssh/ssh_auth_sock

.tmux.conf

set -g update-environment -r
setenv -g SSH_AUTH_SOCK $HOME/.ssh/ssh_auth_sock

In some cases, like me just goes vim ssh-agent.plugin.zsh to understand how its works, realized that @matfra 👍 with actually really cool example, code isn't working, here is my snippet, improving by ensuring this is really running ssh-agent, not any ^*.ssh-agent*.$ 😉

.bashrc:

if for pid in $(ps auxwww | grep ssh-agent | grep -v grep | awk '{print $2}'); do sudo ls -l /proc/$pid/exe | grep \/usr\/bin\/ssh-agent >/dev/null; done; then echo nope >/dev/null; else ssh-agent >/dev/null; fi

• if your ssh-agent path is different, this would not work
• this supress output, so beware

But this is becoming ugly, i'd like to no one will use this solution, check this out for more elegant way to solve problem:
found here: https://gist.github.com/duijf/baeeec8197cdf81448370cf33dc0e708

function start_agent {
  echo "Initialising new SSH agent..."
  /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
  echo Succeeded
  chmod 600 "${SSH_ENV}"
  . "${SSH_ENV}" > /dev/null
  /usr/bin/ssh-add
}

if [ -f "${SSH_ENV}" ]; then
  . "${SSH_ENV}" > /dev/null
  ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
    start_agent;
  }
else
  start_agent
fi

atm idk how to pair this with tmux, someone?

Or adopt this awesomeness: https://github.com/wwalker/ssh-find-agent

What about just this ?

export SSH_AUTH_SOCK="$HOME/.ssh/agent.sock"
pgrep -fx "ssh-agent -a $SSH_AUTH_SOCK" || ssh-agent -a "$SSH_AUTH_SOCK"

I prefer to avoid configuring GNU/screen or TMUX (etc) and keep everything purely in ~/.ssh/:

Create ~/.ssh/rc:

#!/bin/bash

# Fix SSH auth socket location so agent forwarding works within tmux
if test "$SSH_AUTH_SOCK" ; then
  ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
fi

Add following to ~/.ssh/config so it no longer relies on $SSH_AUTH_SOCK, which goes stale in detached terminals:

Host *
  IdentityAgent ~/.ssh/ssh_auth_sock

I have created a wrapper for screen as a shell-function. The symlink gets updated when you run screen, not at the login:

function screen {
    if [ -S "$SSH_AUTH_SOCK" ] && [ ! -h "$SSH_AUTH_SOCK" ]; then
        ln -sf "$SSH_AUTH_SOCK" ~/.ssh/ssh_auth_sock
    fi
    SSH_AUTH_SOCK=~/.ssh/ssh_auth_sock command screen "$@"
}

Add following to ~/.ssh/config so it no longer relies on $SSH_AUTH_SOCK, which goes stale in detached terminals:

Host *
  IdentityAgent ~/.ssh/ssh_auth_sock

Works like a charm, thank you!

I have created a wrapper for screen as a shell-function. The symlink gets updated when you run screen, not at the login:

Looks like it would not work, if you have opened two screen sessions from two different ssh connections and closed the second one, because symlink will be overwritten but the socket will be invalid.

An alternative to using a ~/.ssh/rc file is simply to use the following inside ~/.bashrc:

if [[ -S "$SSH_AUTH_SOCK" && ! -h "$SSH_AUTH_SOCK" ]]; then
    ln -sf "$SSH_AUTH_SOCK" ~/.ssh/ssh_auth_sock;
fi
export SSH_AUTH_SOCK=~/.ssh/ssh_auth_sock;

This checks if a socket exists, and it's not a symlink already. This does the same as everything above, as long as ForwardAgent yes or ssh -A is used.

Credit goes to http://superuser.com/a/424588/237155

This is not really an alternative because it would only update the symlink when a new bash session has been opened. In instances where we want to continue where we left off in an existing pane, .bashrc would never be sourced and as a consequence the symlink would not be updated. The only sure-fire way to update the symlink is to do it when a new SSH session is created as per original gist.

For my future self: put this in .bashrc. Solves the problem:

# Make SSH vs screen/tmux happy. Features:
#  * keep just one agent running (checks agent PID stored in ~/.ssh/agent-environment)
#  * have all sessions share the same socket (using a symlink at ~/.ssh/ssh_auth_sock)
#  * call "ressh" from shell at any time to resync, for stale / reattached sessions
#
# Adapted from https://gist.github.com/martijnvermaat/8070533
# Best intro: https://smallstep.com/blog/ssh-agent-explained/

export SSH_ENV="$HOME/.ssh/agent-environment"

function start_agent {
    echo "Initialising new SSH agent"
    /usr/bin/ssh-agent | grep 'SSH_AGENT_PID' > "${SSH_ENV}"
    export SSH_AUTH_SOCK="$HOME/.ssh/ssh_auth_sock";
    echo "export SSH_AUTH_SOCK=$SSH_AUTH_SOCK" >> "${SSH_ENV}"
    source "${SSH_ENV}" > /dev/null;
}

if [ -f "${SSH_ENV}" ]; then
    source "${SSH_ENV}" > /dev/null;
    ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
        start_agent;
    }
else
    start_agent;
fi

function ressh {
    export SSH_AUTH_SOCK="$HOME/.ssh/ssh_auth_sock";
    ln -sf $(find /tmp -maxdepth 2 -type s -name "agent*" -user $USER -printf '%T@ %p\n' 2>/dev/null |sort -n|tail -1|cut -d' ' -f2) $SSH_AUTH_SOCK
    /usr/bin/ssh-add;
}
ressh;

In the following scenario I was left with a broken symlink:

Open connection in A via SSH.

Open then close connection B via SSH. This way we are left with a symlink pointing to the now missing connection B socket.

The following change to ~/.ssh/rc takes this scenario into account by not changing the socket if it's already valid. This has the drawback that connection B looses conectivity if connection A is closed. But I prefer it this way.

 #!/bin/bash
 if test "$SSH_AUTH_SOCK" && [ ! -e ~/.ssh/ssh_auth_sock ]; then
     ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
 fi

I'm using this to get ssh authentication in a remote desktop session by simply opening an shh connection too.

The persistent issue here is: what to do when existing connections close or when there are multiple agent connections open. Without a dedicated (user) daemon on the system to handle this moving target optimally for all cases, we get to pick some default simple strategy for ourselves.

My strategy is that a new connection with an agent gets priority and replaces any existing agent. This is accomplished simply by two small additions to the end of .bashrc:

# if $SSH_AUTH_SOCK is a socket, replace .ssh/auth.sock with a new symlink pointing to it
[ -S "$SSH_AUTH_SOCK" ] && ln -snf "$SSH_AUTH_SOCK" ~/.ssh/auth.sock

# if running in screen or tmux, set $SSH_AUTH_SOCK to the shared auth.sock location, regardless if there is an existing auth sock.
case "$TERM" in
    screen*|tmux*)
        export SSH_AUTH_SOCK=~/.ssh/auth.sock
        ;;
esac