Win32 OpenSSH cross domain authentication via public key

[robertstrom]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest version
• Search the existing issues.

Steps to reproduce

I have a multi-domain environment. There is currently no trust between these domains.

The server that I am trying to connect to is running:

Microsoft Windows Server 2019 Standard
OpenSSH_for_Windows_9.5p2, LibreSSL 3.8.2

I have copied the public key over to the administrators_authorized_keys file using the documented method on this page - https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement - except that there needs to be some modification to make the copying of the SSH key work. I have found that this part needs to be modified:

Original:

$remotePowershell = "powershell Add-Content -Force -Path $env:ProgramData\ssh\administrators_authorized_keys -Value '''$authorizedKey''';icacls.exe ""$env:ProgramData\ssh\administrators_authorized_keys"" /inheritance:r /grant ""Administrators:F"" /grant ""SYSTEM:F"""

Modified:

$remotePowershell = "powershell Add-Content -Force -Path $env:ProgramData\ssh\administrators_authorized_keys -Value '$authorizedKey';icacls.exe ""$env:ProgramData\ssh\administrators_authorized_keys"" /inheritance:r /grant ""Administrators:F"" /grant ""SYSTEM:F"""

Only one set of single quotes around the $authorizedKey variable. Without doing this the administrators_authorized_keys file ends up having single quotes around the public key. I have used this modification successfully in the domain that my workstation is associated with so I know that it generally works. I can also just copy over the SSH public key.

Here's the scenario:

• My workstation is an EntraAD joined machine (not hybrid joined) but there is an association with the main domain
• I can use the method described above to copy over my public SSH key to a server and I can then connect to it without having to use a password
• My SSH public key is for my standard domain user account but it is placed in the administrators_authorized_keys file and I can log in without using a password using adminaccount@servername.fqdn
• Now I try to connect to a machine in the QA domain (there are no domain trusts between these domains) with the same SSH public key in the QA domain server administrators_authorized_keys file and I am prompted for a password
• That is not the behavior that I was expecting. I figured that an SSH key would validate me regardless

I find very little to no documentation on cross domain SSH authentication via public key.

I have also setup the SSH access to Azure Arc-enabled servers successfully in the main domain and also configured a server in the QA domain (both servers are in the same Azure Tenant and the same Azure Arc RG).

The command below works for the main domain but not for the QA domain. I am prompted for the users password when trying to connect to the QA domain server

az ssh arc -g <resource group>  --n <ARC server name> --local-user <user name>

• Is cross domain SSH public key authentication supported?
• If so, how do I make it work?
• Is there documentation pertaining to cross domain SSH public key authentication?
• When will SSH authentication with Microsoft Entra ID be available for WIndows?
• When SSH authentication with Microsoft Entra ID is available for Windows will it work in the cross domain scenario that I am describing?

Expected behavior

Be able to use an SSH key universally across any machine where the public key has been deployed

Actual behavior

I'm prompted for the password for the users password when attempting to login to the QA domain server regardless of whether I am using a standard SSH or an Azure Arc ssh connection method.

Error details

Environment data

This QA machine that is currently failing is currently configured to present the standard CMD shell upon SSH login. I have configured the other domain machine that is working to default to PowerShell 7 as the default shell.


Name                           Value
----                           -----
PSVersion                      7.5.3
PSEdition                      Core
GitCommitId                    7.5.3
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

and

Name                           Value
----                           -----
PSVersion                      5.1.17763.8146
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.8146
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version

OpenSSH_for_Windows_9.5p2, LibreSSL 3.8.2

Visuals

No response

[matsmcp]

I do not do cloud so I can't help you with regards with Entra.

However I do have a number of local domains, none of them have trusts.
Do I use the same SSH key to log on to a number of them - Yes

I do use a modified SSH config so we do not use administrators_authorized_keys but I place my public key in my authorized_keys file on servers in different domains (or stand alone machines) I can log on to them.

The https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement page seems to be out of date.
AuthorizedKeysCommand and AuthorizedKeysCommandUser do work in later versions of OpenSSH on Windows (Introduced in 8.6) I have verified that I can store the key in AD and use them in SSH - You can even do this with smart cards

The most common error I see with key based auth is that the permissions on the keys are not acceptable for OpenSSH.

I think the best step to troubleshot this further is to collect a debug log from SSHD from the affected server. in the sshd_config

LogLevel DEBUG3
SyslogFacility LOCAL0

[robertstrom]

@matsmcp - Thanks so much for the information!

I was already starting to go down the path of using the individual users authorized_keys files anyway since I have found some really disturbing behavior when using the administrators_authorized_keys file.

So, very glad to know that cross domain is working for you when using the individual authorized_keys files.

It's also excellent news that these two new sshd_config configuration arguments are now supported! I will start playing around with those too.

Can you elaborate any further on how / where you are storing the keys in AD and what that portion of the sshd_config file looks like?

Other questions that I just thought of. Is it possible to store multiple keys for a specific user? Let's say that I access systems from multiple systems and have a different key on each of those systems. What about using SSH key passwords. Do you set a password on the key if it is stored in AD?

Thanks again!

[matsmcp]

@matsmcp - Thanks so much for the information!

Can you elaborate any further on how / where you are storing the keys in AD and what that portion of the sshd_config file looks like?

Yes. In my test/Dev environment the user class in AD was extended with an extra attribute sshPublicKeys. It a multivalue stirng value making it possible to have multiple keys.

In SSHD_config
PubkeyAuthentication yes
AuthorizedKeysCommand C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive -File somescript.ps1 %u
AuthorizedKeysCommandUser system

It makes SSHD run "somescript.ps1" with the %u (username) as a parameter. That script in turn connects to the AD, reads and returns the value(s) of the attribute used to store the sshkeys. Since it's running with system it has access to AD through the machine account. The script has to be written to suit your needs

The Password on a SSH key is set on the private part of the key and that is still stored on your computer, not in AD so if you have set a password when you created the key (good practice) is still will be password protected

@tgauth
Do you know anyone who can fix the https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement article?

[robertstrom]

@matsmcp - Excellent, that is exactly the information that I found on storing and using SSH keys in Active Directory.

I found these two articles that pull the information together

Retrieve SSH public key from Active Directory for SSH authentication - but this one is missing the last necessary step in the schema upgrade.

I found the information I needed for the last step in this post - Pushing SSH keys from Okta to AD - in the Modifying the Active Directory Schema section.

I was able to get it all working in my lab last night.

It would be really excellent if Microsoft would make this capability known and document how to do it.