SSL errors between Netapp Filer and RedHat Directory Server

As I mentioned in my last post, we’ve been working on an LDAP SSL configuration for our Netapp filers at work. Most of the work was already done and I had to coordinate with our storage team to test the changes. Unfortunately, while most of the work was already done, when we would enable SSL, LDAP lookups would just fail for seemingly no reason. It worked fine when SSL was disabled, albeit unencrypted. Running ldapsearch also worked (after tweaking of OpenLDAP’s client configuration) over SSL and unencrypted. So, it seems, the problem was with the filer.

We decided to run some wireshark analysis of the SSL handshakes, and those turned up that the filer was rejecting the server’s cert with the error “Bad Certificate (42)”. Leading us back to concluding, at first, that it was a problem on the server.

Unfortunately, there wasn’t much help on Google about the issue, and neither RedHat nor Netapp had a solution. But, eventually one was found. I’ll detail everything below, but the ultimate problem was that while the Netapp was rejecting the certificate sent from the server, it was because the Netapp didn’t trust the Root CA certificate which had signed the intermediate CA certificates, which were used to sign the server’s CA. Even after using keymgr to install the correct root CA, we had to go one step further than that by installing the root CA via secureadmin.

For a bit of background, the LDAP server is RedHat Directory Server running on RHEL6. The RHDS passes through AD lookups for passwords so that Windows users can login to Linux. We’re currently running the Netapps with NIS lookups and wanted to phase out NIS.

The original RHDS config was set to allow but not require SSL, and provided access for anonymous binds in addition to simple binds. We opted to go with simple binds on the filer.

netapp*> options ldap
ldap.base                   dc=example,dc=com
ldap.enable                 on
ldap.minimum_bind_level     simple                   (Base DN from any ldapsearch command you might run)
[-snip ldap.nssmap.* as we used defaults here-]
[-snip ldap.passwd-]
ldap.port                   636
ldap.ssl.enable             on
ldap.timeout                20
[-snip ldap.usermap.* as we used defaults here-]

As I mentioned above, the LDAP Server’s certificate is signed by a trusted CA, not self-signed. So in order to get this to work, we needed the netapp to trust the Root CA.

[linux-host $] mount netapp:/vol/vol0 /mnt
[linux-host $] cat /etc/openldap/cacerts/root-ca.crt /etc/openldap/cacerts/server-cert.crt > /mnt/etc/cert-bundle.crt
netapp*> priv set advanced
netapp*> options ldap.enable off
netapp*> options ldap.ssl.enable off
netapp*> options ssl off
netapp*> secureadmin disable ssl
netapp*> secureadminaddcert /etc/cert-bundle.crt
netapp*> secureadmin enable ssl
netapp*> secureadmin status
netapp*> keymgr list cert
netapp*> keymgr list root
netapp*> keymgr install root /etc/cert-bundle.crt
netapp*> options ssl on
netapp*> options ldap.ssl.enable on
netapp*> options ldap.enable on
netapp*> getXXbyYY getpwbyname_r (unix username)

Now, in order to get the Root CA, I actually had to export it from the firefox install on the server, though I found out later on that the CA (in our case Verisign) does provide their Root CA. I had originally assumed they only provided their intermediate CA’s. And yes, in order for this to work, you have to use the ROOT Root CA… The one used to sign the Intermediate CA’s. It’ll be self signed (issuer and subject lines will be identical).

In addition to all of the above for the filer side, you also must configure the RHDS side. To do that, launch the redhat-idm-console, open up the Directory Server, and click Manage Certificates.

In the Server Certs tab, install the server’s certificate which was signed by the CA (Verisign). In the CA Certs tab, install the Intermediate and Root CA certificates. Then save all changes and restart RHDS.

Also, as a side note, in order to make 100% certain that the right certificates were being passed around, I copied the RHDS certificate DBs to other places on the disk where it was used, such as /etc/dirsrv/admin-serv and /etc/openldap/certs and /etc/pki/nssdb, and I also modified the trust levels for all 3 Root CA’s to be valid for client and server security.

All in all, this has been an interesting learning experience. I hope it helps you sometime. If it doesn’t or if you have any questions, please feel free to post them in the Disqus comments so that I can have a look and offer assistance.