The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers
«
»

5 days after the exploit publication of snowcra5h/CVE-2023-38408: Remote Code Execution in OpenSSH’s forwarded ssh-agent

Posted by jpluimers on 2023/07/26

TL;DR is at the bottom (;

5 days ago this exploit development got published: [Wayback/Archive] snowcra5h/CVE-2023-38408: CVE-2023-38408 Remote Code Execution in OpenSSH’s forwarded ssh-agent.

It is about [Wayback/Archive] NVD – CVE-2023-38408 which there at NIST isn’t rated (yet?), neither at [Wayback/Archive] CVE-2023-38408 : The PKCS#11 feature in ssh-agent in OpenSSH before 9.3p2 has an insufficiently trustworthy search path, leading to remot.

However at [Wayback/Archive] CVE-2023-38408- Red Hat Customer Portal it scores 7.3 and [Wayback/Archive] CVE-2023-38408 | SUSE it did get a rating of 7.5, so since I mainly use OpenSuSE I wondered what to do as the CVE is formulated densely at [Wayback/Archive] www.qualys.com/2023/07/19/cve-2023-38408/rce-openssh-forwarded-ssh-agent.txt: it mentions Alice, but no Bob or Mallory (see Alice and Bob – Wikipedia).

Luckily, others readly already did the fine reading and emphasised the important bits, especially at [Wayback/Archive] RCE Vulnerability in OpenSSH’s SSH-Agent Forwarding: CVE-2023-38408 (note that instead of Alex, they actually mean Alice)

“A system administrator (Alice) runs SSH-agent on her local workstation, connects to a remote server with ssh, and enables SSH-agent forwarding with the -A or ForwardAgent option, thus making her SSH-agent (which is running on her local workstation) reachable from the remote server.”

According to researchers from Qualys, a remote attacker who has control of the host, which Alex has connected to, can load (dlopen()) and immediately unload (dlclose()) any shared library in /usr/lib* on Alice’s workstation (via her forwarded SSH-agent if it is compiled with ENABLE_PKCS11, which is the default).

The vulnerability lies in how SSH-agent handles forwarded shared libraries. When SSH-agent is compiled with ENABLE_PKCS11 (the default configuration), it forwards shared libraries from the user’s local workstation to the remote server. These libraries are loaded (dlopen()) and immediately unloaded (dlclose()) on the user’s workstation. The problem arises because certain shared libraries have side effects when loaded and unloaded, which can be exploited by an attacker who gains access to the remote server where SSH-agent is forwarded to.

Mitigations for the SSH-Agent Forwarding RCE Vulnerability

Given the potential risks associated with forwarding, users and system administrators should exercise caution when enabling this feature. While it offers convenience, it also increases the attack surface.Here are some steps to mitigate the risk:
  1. Minimize Agent Forwarding: Limit the use of forwarding to only when necessary. Avoid enabling forwarding for every SSH session and only use it on trusted remote servers.
  2. Use Jump Hosts: Consider using the -J option in SSH to specify a jump host. This approach allows you to access remote servers through an intermediary host, reducing the exposure of your SSH-agent.
  3. Disable PKCS#11 Support: If possible, compile the SSH-agent without ENABLE_PKCS11 to prevent forwarding shared libraries and reduce the potential attack surface.
  4. Keep Software Updated: Regularly update your operating system and SSH software to ensure you have the latest security patches: OpenSSH 9.3p2 or later

In addition, I learned they have an interesting [Wayback/Archive] External Attack Surface – SOCRadar LABS, definitely worth trying that out!

The best diagram of an almost identical situation I could find is at this post which only got published 3 weeks ago: [Wayback/Archive] Understanding SSH Agent and SSH Agent Hijacking: A Real-Life Scenario | by Nidal Mahmud | Jul, 2023 | Medium

…

consider deploying public keys directly, keeping your systems and SSH software updated, and limiting the time keys are loaded into the agent.

In addition, leveraging the principle of least privilege helps — each key should only have the access it needs, thereby limiting potential damage. Regular monitoring and logging of SSH access can help detect suspicious activities and prevent an SSH agent hijacking attempt before it wreaks havoc.

Definitely recommended reading that whole post.

Mitigation

Of course the best way is to upgrade to [Wayback/Archive] www.openssh.com/txt/release-9.3p2, but not all systems have that version available for installation yet and some that have cannot be upgraded at will.

So for me it was interesting to know that the actual vulnerability is on the local (client) side, depends on a compromised remote (server) side and the usage of ssh-agent on the local side.

Since the client side never knows if a server side is compromise, the next best from upgrade is to disable or minimise ssh-agent forwarding::

  • avoid the ssh -A switch in [Wayback/Archive] ssh -A: OpenSSH remote login client | openssh-clients Commands | Man Pages | ManKier

    -A

    Enables forwarding of connections from an authentication agent such as ssh-agent(1). This can also be specified on a per-host basis in a configuration file.

    Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s UNIX-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent. A safer alternative may be to use a jump host (see -J).

  • avoid enabling ForwardAgent in [Wayback/Archive] ssh_config: OpenSSH client configuration file | openssh-clients File Formats | Man Pages | ManKier

    ForwardAgent

    Specifies whether the connection to the authentication agent (if any) will be forwarded to the remote machine. The argument may be yes, no (the default), an explicit path to an agent socket or the name of an environment variable (beginning with ‘$’) in which to find the path.

    Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.

What you see here is that agent forwarding is both discouraged at the client side and server side.

Disabling this on the remote side (even if you can trust it: remember Mallory can only abuse the ssh-agent vulnerability when controlling the remote) is not possible as you can see in [Wayback/Archive] sshd_config: OpenSSH daemon configuration file | openssh-server File Formats | Man Pages | ManKier

AllowAgentForwarding

Specifies whether ssh-agent(1) forwarding is permitted. The default is yes. Note that disabling agent forwarding does not improve security unless users are also denied shell access, as they can always install their own forwarders.

no-agent-forwarding

Forbids authentication agent forwarding when this key is used for authentication.

…

restrict

Enable all restrictions, i.e. disable port, agent and X11 forwarding, as well as disabling PTY allocation and execution of ~/.ssh/rc. If any future restriction capabilities are added to authorized_keys files, they will be included in this set.

This means you need to raise ssh client usage awareness among the users in your organisation to favour Jump Host over agent-forwarding.

ssh Jump Hosts

Both AgentForwarding and Jump Host were new for me (as I always had fresh sets of identities for each local or intermediate use case), so here are some links I might learn a thing or two from:

  • [Wayback/Archive] ssh -J: OpenSSH remote login client | openssh-clients Commands | Man Pages | ManKier

    -J destination

    Connect to the target host by first making a ssh connection to the jump host described by destination and then establishing a TCP forwarding to the ultimate destination from there. Multiple jump hops may be specified separated by comma characters. This is a shortcut to specify a ProxyJump configuration directive. Note that configuration directives supplied on the command-line generally apply to the destination host and not any specified jump hosts. Use ~/.ssh/config to specify configuration for jump hosts.

Some links to skip (with reasons why):

Don’t use, but know about Agent Forwarding

Some links so I at least know more about Agent Forwarding (and why users love it despite they should be discouraged from using it):

Exploit

The exploit repository has these files:

  1. [Wayback/Archive] CVE-2023-38408/README.md at main · snowcra5h/CVE-2023-38408
  2. [Wayback/Archive] CVE-2023-38408/gpl-3.0.txt at main · snowcra5h/CVE-2023-38408
  3. [Wayback/A] CVE-2023-38408/step1.sh at main · snowcra5h/CVE-2023-38408
    ./step1.sh /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_jammy_main_binary-amd64_Packages
./step1.sh /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_jammy_universe_binary-amd64_Packages
  4. [Wayback/Archive] CVE-2023-38408/step2.c at main · snowcra5h/CVE-2023-38408
    ./step2
  5. [Wayback/Archive] CVE-2023-38408/step3.sh at main · snowcra5h/CVE-2023-38408
    ./step3.sh /tmp/step2*
  6. [Wayback/Archive] CVE-2023-38408/step4.c at main · snowcra5h/CVE-2023-38408
    ./step4 /tmp/step3*
  7. [Wayback/Archive] CVE-2023-38408/step5.c at main · snowcra5h/CVE-2023-38408
    XSTACK=/tmp/step3*?/xstack.maps ./step5 /tmp/step4*?/?*?/logger
  8. [Wayback/Archive] CVE-2023-38408/step6.c at main · snowcra5h/CVE-2023-38408
    for i in /tmp/step5*?/?*?/logger; do TRIES=16 ./step6 "$i"; done

Further reading

Queries

TL;DR

  1. There is a vulnerability in the local ssh-agent because the remote server can load arbitrary dynamic libraries on it and have their load/unload logic cause trouble.
  2. Despite ssh-agentforwarging is discouraged, users still use it because it is convenient for them.
  3. Forcing client and server systems to upgrade to the latest OpenSSH releases is really important but not always possible. Still try this as much as possible as it is the best solution.
  4. There is no way to fully disable ssh-agent forwarding on the local client with the ssh -A option or ForwardAgent config setting,  and allowagentforwardingor  settings on the remote server: perform as many discouragement steps as possible, as then only the very persistent users will be able to shoot themselves in the foot.
  5. This means that unpatched systems will stay vulnerable, as coercing users into not using ssh-agent forwarding is hard: they are stubborn humans.

This again means you need to raise ssh client usage awareness among the users in your organisation to favour Jump Host over agent-forwarding.

–jeroen

This entry was posted on 2023/07/26 at 19:08 and is filed under *nix, *nix-tools, bash, bash, Communications Development, Development, Internet protocol suite, OpenSSH, Power User, PowerShell, Scripting, Security, Software Development, SSH. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

«
»