When you upgrade from Solaris 11.3 to Solaris 11.4 there are a lot of changes where SSH is one of them. Where Solaris 11.3 defaults to use SunSSH, Solaris 11.4 changed this to OpenSSH.
In the environment I work with, some hardening was configured to disallow certain unsafe Ciphers and MACs. Therefor the directives were configured as:
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
MACs hmac-sha1,hmac-sha1-96,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96
Those values reflect the manual page for sshd_config on FIPS-140:
Ciphers
Specifies the ciphers allowed for protocol version 2. Cipher order-
ing on the server side is not relevant. Multiple ciphers must be
comma separated.
Valid ciphers are: aes128-ctr, aes192-ctr, aes256-ctr, aes128-cbc,
aes192-cbc, aes256-cbc, arcfour, arcfour128, arcfour256, 3des-cbc,
and blowfish-cbc.
The default cipher list is:
aes128-ctr,aes192-ctr,aes256-ctr,arcfour128,
arcfour256,arcfour
Using CBC modes on the server side is not recommended due to poten-
tial security issues in connection with the SSH protocol version 2.
When configuring sshd to run OpenSSL in FIPS-140 mode, the default
cipher list is: aes128-cbc, aes192-cbc, aes256-cbc. Specifying a
non-FIPS approved cipher will return an error.
For the case of ssh with FIPS-140 enabled logging into a non-
FIPS-140 sshd, the supported and approved FIPS ciphers must be
explicitly specified in sshd_config using "Ciphers" for this sce-
nario.
MACs
Specifies the available MAC (message authentication code) algo-
rithms. The MAC algorithm is used in protocol version 2 for data
integrity protection. Multiple algorithms must be comma-separated.
The default is hmac-md5,hmac-sha1,hmac-sha1-96,hmac-md5-96, hmac-
sha2-256, hmac-sha2-256-96, hmac-sha2-512, and hmac-sha2-512-96.
When configuring sshd to run OpenSSL in FIPS-140 mode, the default
is hmac-sha1, hmac-sha1-96, hmac-sha2-256, hmac-sha2-256-96, hmac-
sha2-512, and hmac-sha2-512-96. Specifying a non-FIPS approved MAC
algorithm will return an error.
As said earlier Solaris 11.4 uses OpenSSH and the Ciphers and MACs are just a bit different in this version. The supported Ciphers and MACs are:
The supported ciphers are:
3des-cbc
aes128-cbc
aes192-cbc
aes256-cbc
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
blowfish-cbc
chacha20-poly1305@openssh.com
The default is:
chacha20-poly1305@openssh.com,
aes128-ctr,aes192-ctr,aes256-ctr,
aes128-gcm@openssh.com,aes256-gcm@openssh.com
The following ciphers are FIPS-140 approved and are supported in
FIPS-140 mode:
aes128-ctr,aes192-ctr,aes256-ctr,
aes128-gcm@openssh.com,aes256-gcm@openssh.com,
aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
The supported MACs are:
hmac-ripemd160
hmac-sha1
hmac-sha1-96
hmac-sha2-256
hmac-sha2-512
umac-64@openssh.com
umac-128@openssh.com
hmac-ripemd160-etm@openssh.com
hmac-sha1-etm@openssh.com
hmac-sha1-96-etm@openssh.com
hmac-sha2-256-etm@openssh.com
hmac-sha2-512-etm@openssh.com
umac-64-etm@openssh.com
umac-128-etm@openssh.com
The default is:
umac-64-etm@openssh.com,umac-128-etm@openssh.com,
hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,
hmac-sha1-etm@openssh.com,
umac-64@openssh.com,umac-128@openssh.com,
hmac-sha2-256,hmac-sha2-512,hmac-sha1
The following MACs are FIPS-140 approved and are supported in
FIPS-140 mode:
hmac-sha2-256-etm@openssh.com,
hmac-sha2-512-etm@openssh.com,
hmac-sha2-256,hmac-sha2-512,
hmac-sha1-etm@openssh.com,
hmac-sha1-96-etm@openssh.com
hmac-sha1,hmac-sha1-96
It seems that OpenSSH does not knows about the two MACs:
hmac-sha2-256-96
hmac-sha2-512-96
If you use one of those MACs specifically /etc/ssh/sshd_config and you upgrade from Solaris 11.3 to Solaris 11.4 you will be unable to login using SSH because the SSH service will be in maintenance mode:
tail /var/svc/log/network-ssh:default.log
[ 2018 Oct 1 14:53:18 Executing start method ("/lib/svc/method/sshd start"). ]
/etc/ssh/sshd_config line 15: Bad SSH2 mac spec 'hmac-sha1,hmac-sha1-96,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96'.
[ 2018 Oct 1 14:53:19 Method "start" exited with status 255. ]
The resolution is to login to the console, remove both MACs (or comment out the whole MACs directive) and clear the service:
svcadm clear ssh