SaltConfig and Identity manager integration

Posted on by Nithin Titta

SaltConfig must be running version 8.5 and must be deployed via LCM.

If vRA is running on self-signed/local-CA/LCM-CA certificates the saltstack UI will not load and you will see similar symptoms:

Specifically, a blank page when logging on to salt UI with account/info api returning 500

Logs: 

less /var/log/raas/raas
Traceback (most recent call last):
File "requests/adapters.py", line 449, in send
File "urllib3/connectionpool.py", line 756, in urlopen
File "urllib3/util/retry.py", line 574, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='automation.ntitta.lab', port=443): Max retries exceeded with url: /csp/gateway/am/api/auth/discovery?username=service_type&state=aHR0cHM6Ly9zYWx0eS5udGl0dGEubGFiL2lkZW50aXR5L2FwaS9jb3JlL2F1dGhuL2NzcA%3D%3D&redirect_uri=https%3A%2F%2Fsalty.ntitta.lab%2Fidentity%2Fapi%2Fcore%2Fauthn%2Fcsp&client_id=ssc-HLwywt0h3Y (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "tornado/web.py", line 1680, in _execute
File "raas/utils/rest.py", line 153, in prepare
File "raas/utils/rest.py", line 481, in prepare
File "pop/contract.py", line 170, in __call__
File "/var/lib/raas/unpack/_MEIb1NPIC/raas/mods/vra/params.py", line 250, in get_login_url
verify=validate_ssl)
File "requests/api.py", line 76, in get
File "requests/api.py", line 61, in request
File "requests/sessions.py", line 542, in request
File "raven/breadcrumbs.py", line 341, in send
File "requests/sessions.py", line 655, in send
File "requests/adapters.py", line 514, in send
requests.exceptions.SSLError: HTTPSConnectionPool(host='automation.ntitta.lab', port=443): Max retries exceeded with url: /csp/gateway/am/api/auth/discovery?username=service_type&state=aHR0cHM6Ly9zYWx0eS5udGl0dGEubGFiL2lkZW50aXR5L2FwaS9jb3JlL2F1dGhuL2NzcA%3D%3D&redirect_uri=https%3A%2F%2Fsalty.ntitta.lab%2Fidentity%2Fapi%2Fcore%2Fauthn%2Fcsp&client_id=ssc-HLwywt0h3Y (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))
2021-08-23 04:29:16,906 [tornado.access                                                    ][ERROR   :2250][Webserver:59844] 500 POST /rpc (127.0.0.1) 1697.46ms

To resolve this, grab the root certificate of vRA and import this over to the saltstack appliance root store:

Grab root certificate:

Cli method: 

root@salty [ ~ ]# openssl s_client -showcerts -connect automation.ntitta.lab:443
CONNECTED(00000003)
depth=1 CN = vRealize Suite Lifecycle Manager Locker CA, O = VMware, C = IN
verify error:num=19:self signed certificate in certificate chain
---
Certificate chain
 0 s:/CN=automation.ntitta.lab/OU=labs/O=GSS/L=BLR/ST=KA/C=IN
   i:/CN=vRealize Suite Lifecycle Manager Locker CA/O=VMware/C=IN
-----BEGIN CERTIFICATE-----
MIID7jCCAtagAwIBAgIGAXmkBtDxMA0GCSqGSIb3DQEBCwUAMFMxMzAxBgNVBAMM
KnZSZWFsaXplIFN1aXRlIExpZmVjeWNsZSBNYW5hZ2VyIExvY2tlciBDQTEPMA0G
A1UECgwGVk13YXJlMQswCQYDVQQGEwJJTjAeFw0yMTA1MjUxNDU2MjBaFw0yMzA1
MjUxNDU2MjBaMGUxHjAcBgNVBAMMFWF1dG9tYXRpb24ubnRpdHRhLmxhYjENMAsG
A1UECwwEbGFiczEMMAoGA1UECgwDR1NTMQwwCgYDVQQHDANCTFIxCzAJBgNVBAgM
AktBMQswCQYDVQQGEwJJTjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJ+p/UsPFJp3WESJfUNlPWAUtYOUQ9cK5lZXBrEK79dtOwzJ8noUyKndO8i5wumC
tNJP8U3RjKbqu75UZH3LiwoHTOEkqhWufrn8gL7tQjtiQ0iAp2pP6ikxH2bXNAwF
Dh9/2CMjLhSN5mb7V5ehu4rP3/Niu19nT5iA1XMER3qR2tsRweV++78vrYFsKDS9
ePa+eGvMNrVaXvbYN75KnLEKbpkHGPg9P10zLbP/lPIskEGfgBMjS7JKOPxZZKX1
GczW/2sFq9OOr4bW6teWG3gt319N+ReNlUxnrxMDkKcWrml8EbeQMp4RmmtXX5Z4
JeVEATMS7O2CeoEN5E/rFFUCAwEAAaOBtTCBsjAdBgNVHQ4EFgQUz/pxN1bN/GxO
cQ/hcQCgBSdRqaUwHwYDVR0jBBgwFoAUYOI4DbX97wdcZa/pWivAMvnnDekwMAYD
VR0RBCkwJ4IXKi5hdXRvbWF0aW9uLm50aXR0YS5sYWKCDCoubnRpdHRhLmxhYjAO
BgNVHQ8BAf8EBAMCBaAwIAYDVR0lAQH/BBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMB
MAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQELBQADggEBAA2KntXAyrY6DHho8FQc
R2GrHVCCWG3ugyPEq7S7tAabMIeSVhbPWsDaVLro5PlldK9FAUhinbxEwShIJfVP
+X1WOBUxwTQ7anfiagonMNotGtow/7f+fnHGO4Mfyk+ICo+jOp5DTDHGRmF8aYsP
5YGkOdpAb8SuT/pNerZie5WKx/3ZuUwsEDTqF3CYdqWQZSuDIlWRetECZAaq50hJ
c6kD/D1+cq2pmN/DI/U9RAfsvexkhdZaMbHdrlGzNb4biSvJ8HjJMH4uNLUN+Nyf
2MON41QKRRuzQn+ahq7X/K2BbxJTQUZGwbC+0CA6M79dQ1eVQui4d5GXmjutqFIo
Xwo=
-----END CERTIFICATE-----
 1 s:/CN=vRealize Suite Lifecycle Manager Locker CA/O=VMware/C=IN
   i:/CN=vRealize Suite Lifecycle Manager Locker CA/O=VMware/C=IN
-----BEGIN CERTIFICATE-----
MIIDiTCCAnGgAwIBAgIGAXmEbtiqMA0GCSqGSIb3DQEBCwUAMFMxMzAxBgNVBAMM
KnZSZWFsaXplIFN1aXRlIExpZmVjeWNsZSBNYW5hZ2VyIExvY2tlciBDQTEPMA0G
A1UECgwGVk13YXJlMQswCQYDVQQGEwJJTjAeFw0yMTA1MTkxMTQyMDdaFw0zMTA1
MTcxMTQyMDdaMFMxMzAxBgNVBAMMKnZSZWFsaXplIFN1aXRlIExpZmVjeWNsZSBN
YW5hZ2VyIExvY2tlciBDQTEPMA0GA1UECgwGVk13YXJlMQswCQYDVQQGEwJJTjCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK6S4ESddCC7BAl4MACpAeAm
1JBaw72NgeSOruS/ljpd1MyDd/AJjpIpdie2M0cweyGDaJ4+/C549lxQe0NAFsgh
62BG87klbhzvYja6aNKvE+b1EKNMPllFoWiCKJIxZOvTS2FnXjXZFZKMw5e+hf2R
JgPEww+KsHBqcWL3YODmD6NvBRCpY2rVrxUjqh00ouo7EC6EHzZoJSMoSwcEgIGz
pclYSPuEzdbNFKVtEQGrdt94xlAk04mrqP2O6E7Fd5EwrOw/+dsFt70qS0aEj9bQ
nk7GeRXhJynXxlEpgChCDEXQ3MWvLIRwOuMBxQq/W4B/ZzvQVzFwmh3S8UkPTosC
AwEAAaNjMGEwHQYDVR0OBBYEFGDiOA21/e8HXGWv6VorwDL55w3pMB8GA1UdIwQY
MBaAFGDiOA21/e8HXGWv6VorwDL55w3pMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0P
AQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IBAQBqAjCBd+EL6koGogxd72Dickdm
ecK60ghLTNJ2wEKvDICqss/FopeuEVhc8q/vyjJuirbVwJ1iqKuyvANm1niym85i
fjyP6XaJ0brikMPyx+TSNma/WiDoMXdDviUuYZo4tBJC2DUPJ/0KDI7ysAsMTB0R
8Q7Lc3GlJS65AFRNIxkpHI7tBPp2W8tZQlVBe7PEcWMzWRjWZAvwDGfnNvUtX4iY
bHEVWSzpoVQUk1hcylecYeMSCzBGw/efuWayIFoSf7ZXFe0TAEOJySwkzGJB9n78
4Rq0ydikMT4EFHP5G/iFI2zsx2vZGNsAHCw7XSVFydqb/ekm/9T7waqt3fW4
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=automation.ntitta.lab/OU=labs/O=GSS/L=BLR/ST=KA/C=IN
issuer=/CN=vRealize Suite Lifecycle Manager Locker CA/O=VMware/C=IN
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2528 bytes and written 393 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: B06BE4668E5CCE713F1C1547F0917CC901F143CB13D06ED7A111784AAD10B2F6
    Session-ID-ctx:
    Master-Key: 75E8109DD84E2DD064088B44779C4E7FEDA8BE91693C5FC2A51D3F90B177F5C92B7AB638148ADF612EBEFDA30930DED4
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - b9 54 91 b7 60 d4 18 d2-4b 72 55 db 78 e4 91 10   .T..`...KrU.x...
    0010 - 1f 97 a0 35 31 16 21 db-8c 49 bf 4a a1 b4 59 ff   ...51.!..I.J..Y.
    0020 - 07 22 1b cc 20 d5 52 7a-52 84 17 86 b3 2a 7a ee   .".. .RzR....*z.
    0030 - 14 c3 9b 9f 8f 24 a7 a1-76 4d a2 4f bb d7 5a 21   .....$..vM.O..Z!
    0040 - c9 a6 d0 be 3b 57 4a 4e-cd cc 9f a6 12 45 09 b5   ....;WJN.....E..
    0050 - ca c4 c9 57 f5 ac 17 04-94 cb d0 0a 77 17 ac b8   ...W........w...
    0060 - 8a b2 39 f1 78 70 37 6d-d0 bf f1 73 14 63 e8 86   ..9.xp7m...s.c..
    0070 - 17 27 80 c1 3e fe 54 cf-                          .'..>.T.

    Start Time: 1629788388
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)

From the above example,
Certificate chain 0 s:/CN=automation.ntitta.lab/OU=labs/O=GSS/L=BLR/ST=KA/C=IN <—-this is my vRA cert
i:/CN=vRealize Suite Lifecycle Manager Locker CA/O=VMware/C=IN <—-This is the root cert (Generated via LCM)

Create a new cert file with the contents of the root certificate.

cat root.crt
-----BEGIN CERTIFICATE-----
MIIDiTCCAnGgAwIBAgIGAXmEbtiqMA0GCSqGSIb3DQEBCwUAMFMxMzAxBgNVBAMM
KnZSZWFsaXplIFN1aXRlIExpZmVjeWNsZSBNYW5hZ2VyIExvY2tlciBDQTEPMA0G
A1UECgwGVk13YXJlMQswCQYDVQQGEwJJTjAeFw0yMTA1MTkxMTQyMDdaFw0zMTA1
MTcxMTQyMDdaMFMxMzAxBgNVBAMMKnZSZWFsaXplIFN1aXRlIExpZmVjeWNsZSBN
YW5hZ2VyIExvY2tlciBDQTEPMA0GA1UECgwGVk13YXJlMQswCQYDVQQGEwJJTjCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK6S4ESddCC7BAl4MACpAeAm
1JBaw72NgeSOruS/ljpd1MyDd/AJjpIpdie2M0cweyGDaJ4+/C549lxQe0NAFsgh
62BG87klbhzvYja6aNKvE+b1EKNMPllFoWiCKJIxZOvTS2FnXjXZFZKMw5e+hf2R
JgPEww+KsHBqcWL3YODmD6NvBRCpY2rVrxUjqh00ouo7EC6EHzZoJSMoSwcEgIGz
pclYSPuEzdbNFKVtEQGrdt94xlAk04mrqP2O6E7Fd5EwrOw/+dsFt70qS0aEj9bQ
nk7GeRXhJynXxlEpgChCDEXQ3MWvLIRwOuMBxQq/W4B/ZzvQVzFwmh3S8UkPTosC
AwEAAaNjMGEwHQYDVR0OBBYEFGDiOA21/e8HXGWv6VorwDL55w3pMB8GA1UdIwQY
MBaAFGDiOA21/e8HXGWv6VorwDL55w3pMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0P
AQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IBAQBqAjCBd+EL6koGogxd72Dickdm
ecK60ghLTNJ2wEKvDICqss/FopeuEVhc8q/vyjJuirbVwJ1iqKuyvANm1niym85i
fjyP6XaJ0brikMPyx+TSNma/WiDoMXdDviUuYZo4tBJC2DUPJ/0KDI7ysAsMTB0R
8Q7Lc3GlJS65AFRNIxkpHI7tBPp2W8tZQlVBe7PEcWMzWRjWZAvwDGfnNvUtX4iY
bHEVWSzpoVQUk1hcylecYeMSCzBGw/efuWayIFoSf7ZXFe0TAEOJySwkzGJB9n78
4Rq0ydikMT4EFHP5G/iFI2zsx2vZGNsAHCw7XSVFydqb/ekm/9T7waqt3fW4
-----END CERTIFICATE-----

Backup existing certificate store: 

cp  /etc/pki/tls/certs/ca-bundle.crt   ~/

Copy the lcm certificate to the certificate store:

cat root.crt >> /etc/pki/tls/certs/ca-bundle.crt

add the below to raas.service, /usr/lib/systemd/system/raas.service

Environment=REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt

Example: 

root@salty [ ~ ]# cat /usr/lib/systemd/system/raas.service
[Unit]
Description=The SaltStack Enterprise API Server
After=network.target

[Service]
Type=simple
User=raas
Group=raas
# to be able to bind port < 1024
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
PermissionsStartOnly=true
ExecStartPre=/bin/sh -c 'systemctl set-environment FIPS_MODE=$(/opt/vmware/bin/ovfenv -q --key fips-mode)'
ExecStartPre=/bin/sh -c 'systemctl set-environment NODE_TYPE=$(/opt/vmware/bin/ovfenv -q --key node-type)'
Environment=REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
ExecStart=/usr/bin/raas
TimeoutStopSec=90

[Install]
WantedBy=multi-user.target

Restart salt service: 

systemctl daemon-reload
systemctl restart raas && tail -f /var/log/raas/raas

Upon restart, the above command should start to tail the raas logs, ensure that we no longer see the certificate-related messages.

NSLCD based auth on Linux machine/Usage meter 4.4 LDAP

Posted on by Nithin Titta

Below is the sample configuration to get UM-Ad intigation via LDAP (replace the contents of /ec/nslcd.conf with what is appropriate in your environment.

root@photon-machine [ /etc ]# cat /etc/nslcd.conf


uid nslcd
gid ldap

uri ldap://ntitta.lab
base dc=ntitta,dc=lab
binddn CN=service,CN=Users,DC=ntitta,DC=lab
bindpw P@ssw0rd

pagesize 1000
filter passwd (&(objectClass=user)(!(objectClass=computer))(uidNumber=*)(unixHomeDirectory=*))
map    passwd uid              sAMAccountName
map    passwd homeDirectory    unixHomeDirectory
map    passwd gecos            displayName
filter shadow (&(objectClass=user)(!(objectClass=computer))(uidNumber=*)(unixHomeDirectory=*))
map    shadow uid              sAMAccountName
map    shadow shadowLastChange pwdLastSet
filter group  (objectClass=group)

cat /etc/nsswitch.conf


hosts: files resolve dns
networks: files

protocols: files
services: files
ethers: files
rpc: files


passwd: files ldap
group: files ldap
shadow: files ldap

cat /etc/pam.d/system-auth

# Begin /etc/pam.d/system-auth
auth    required    pam_env.so
auth    required    pam_tally2.so onerr=fail deny=3 unlock_time=900 root_unlock_time=900 file=/var/log/tallylog
auth    sufficient  pam_ldap.so
auth    required    pam_unix.so
auth    optional    pam_faildelay.so delay=4000000

cat /etc/pam.d/system-account

# Begin /etc/pam.d/system-account
account    sufficient  pam_ldap.so
account    required    pam_tally2.so file=/var/log/tallylog
account    required    pam_unix.so
# End /etc/pam.d/system-account

cat /etc/pam.d/system-password

# Begin /etc/pam.d/system-password
password    sufficient  pam_ldap.so try_first_pass
password    requisite   pam_cracklib.so     minlen=10 minclass=4 difok=4 maxsequence=0 retry=3 enforce_for_root
password    requisite   pam_pwhistory.so    retry=3 remember=5 enforce_for_root
password    required    pam_unix.so         sha512 shadow use_authtok
# End /etc/pam.d/system-password

cat /etc/pam.d/system-session

# Begin /etc/pam.d/system-session
session   required    pam_unix.so
session   required    pam_limits.so
session   optional    pam_motd.so
session   optional    pam_lastlog.so showfailed
session   optional    pam_systemd.so
session   optional    pam_ldap.so
# End /etc/pam.d/system-session

cat /etc/pam.d/vmware-um-pam

auth       sufficient /lib64/security/pam_ldap.so
auth       required   /lib64/security/pam_unix_auth.so
account    sufficient /lib64/security/pam_ldap.so
account    required   /lib64/security/pam_unix_acct.so

aside from this, NSLCD also needs some pre-requisites for the user that is used to log in, This is described here: https://www.server-world.info/en/note?os=Windows_Server_2019&p=active_directory&f=12
basically the below attributes must exist on ad for the user in question: 

uid           sAMAccountName
uidNumber     objectSid:<yourValue>
gidNumber     primaryGroupID
homeDirectory "/home/$sAMAccountName"
gecos         displayName
loginShell    "/bin/bash"
gidNumber      primaryGroupID

Note:

  • uid must be uniq for a user and should not be associated with any existing user in UM appliance, see getent passwd for the full list of used uids
  • gid must exist in the UM appliance before configuring AD. Use it to control what kind of privileges you want to give to the user. In most case use the usgemeger gid 1002

Note: Making changes to /etc/pam.d generally needs a reboot to take effect. Once the above config are in place, reboot the appliance and try login in
Note: If you move the original config files and create new files, please ensure the permissions of the files are corrected.

If configured correctly and when the user is attempted to log in on the UI, you should see the same domain user mapped in getent passwd

root@photon-machine [ /etc ]# getent passwd


root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/dev/null:/bin/false
daemon:x:6:6:Daemon User:/dev/null:/bin/false
messagebus:x:18:18:D-Bus Message Daemon User:/var/run/dbus:/bin/false
systemd-bus-proxy:x:72:72:systemd Bus Proxy:/:/bin/false
systemd-journal-gateway:x:73:73:systemd Journal Gateway:/:/bin/false
systemd-journal-remote:x:74:74:systemd Journal Remote:/:/bin/false
systemd-journal-upload:x:75:75:systemd Journal Upload:/:/bin/false
systemd-network:x:76:76:systemd Network Management:/:/bin/false
systemd-resolve:x:77:77:systemd Resolver:/:/bin/false
systemd-timesync:x:78:78:systemd Time Synchronization:/:/bin/false
nobody:x:65534:65533:Unprivileged User:/dev/null:/bin/false
sshd:x:50:50:sshd PrivSep:/var/lib/sshd:/bin/false
named:x:999:999::/var/lib/bind:/bin/false
polkitd:x:27:1000:PolicyKit Daemon Owner:/etc/polkit-1:/bin/false
nslcd:x:998:998:nslcd ldap user:/:/usr/sbin/nologin
ntp:x:87:87:Network Time Protocol:/var/lib/ntp:/bin/false
usagemeter:x:1000:1002::/home/usagemeter:/bin/bash
umauditor:x:1001:1003::/home/umauditor:/bin/bash
test:*:5000:1002:test:/home/test:/bin/bash                         <---------this is the domain user