Key And Trust Store on Unix-like OS

lock drawing

NB: This is a repost on this blog of a post made on nixers.net

Let’s have a discussion about all the kinds of trust stores found on Unix-like operating systems.
For those not in the know, trust stores are places where the operating sytems generally, or the specific software, stores private and public keys (asymmetric), trusted CAs, and symmetric keys (decryption keys).

There’s a lot to cover on this topic, I thought of writing an article about this because I couldn’t find anything online that covered it in a generic manner. That’s what gets me writing anyway.

Let’s tackle some of the stuffs regarding TLS PKI (Public Key Infrastructure).

Mozilla maintains a list of trusted CAs in a certificate store that a lot of Unix-like operating system fetch through the package manager and deploy at /etc/ssl/certs.

This location is accessed system wide by a lot of utilities to check the trusted certificates. It sort of has become standard, though as you’ll see in a bit it’s not really.

You also may find a symbolic link there pointing to /etc/ca-certificates/extracted/. This all points to the same thing, there’s even /usr/share/ca-certificates or /usr/lib/mozilla/certificates, /usr/lib64/mozilla/certificates, ~/.mozilla/certificates.

Openssl also stores/read certificate from that location /etc/ssl, that’s where you’ll find openssl.cnf for example. In this directory you can choose to store your private keys associated with certificates you’ve generated yourself in /etc/ssl/private. For obvious reasons, this directory should only be owned by root.

But there’s a catch here, openssl can be compiled with the nss library, which will have its own list of trusted CAs built-in though usually through /usr/lib/libnssckbi.so which, again, has the list of maintained trusted CAs by Mozilla (Mozilla’s Network Security Services).

The Chrome browser also uses nss so you might ask where the trust exclusions are stored when added. They are in $HOME/.pki/nssdb or /etc/pki/nssdb globally in an sqlite3 db.

Firefox also uses an sqlite3 database to store its exclusions. However, it’s not in the .pki directory but right within its profile directory: $HOME/.mozilla/firefox/<profilename>.default/cert_override.txt. Add to this that it has two (or maybe more) sqlite3 dbs in there which are basically copy of the nss trusted certs that are found globally on the system.

Now what about programming languages that want to access the internet in a secure manner through TLS PKI.

Most of them rely on the trusted stores mentioned previously, namely nss or /etc/ssl. However, some don’t.

I’m aware of one well known example with the Java language. It stores its trust store in the $JAVA_HOME/jre/lib/security/cacerts which is a java keystore. The password to this keystore is “changeit”. Java has a concept of security provider, and they are listed in order of preference in the java.security file. Hopefully you can find one of the provider relying on the nss.cfg, and so we have less redundancy within our system.

Let’s also put a hint here about certificate revocation. Sometimes, in specific cases, you can’t always rely on your OS packages to update your trusted CAs and you’ll need a daemon to check CRLs and OCSPs for all the trusted certs you got.

One example is: dirmngr(1)

Now there are two other common places that I’ll tackle too.

Gnupg trust store and ssh trust store. Those are in $HOME/.gnupg and $HOME/.ssh respectively.

Those directories both contains trusted certificates and your private/public pairs.

Let’s mention that almost all things in the crypto world uses a format called ASN.1 with DER encoding or not. GPG, X509, SSH all have it this way with some different formatting in some places.

You can have a look at those here:

And here’s a useful link:

https://wiki.gentoo.org/wiki/Certificates

Outro

So nixers, what do you have to say about trust stores on Unix-like OS. Anything to add to what I’ve mentioned. There’s a lot I’ve deliberately left out. Maybe talk about interfacing with keystores on a hardware security module through pkcs#11, like a yubikey, that could be used for OTP. Maybe we can talk about all the utilities that can be used to manipulate, create, and display in a human readable format the certificates, public/private pairs, and more (openssl, keytool, certutil, etc..). We can also talk about building your own PKI. We can talk about what specific language do to handle cryptographic keystores, what you like about it. Or maybe simply share a link you’ve found useful. Or maybe we can talk about package management and how maintainers should sign their packages. Or just express your opinion about anything.

We could go into secret management, PAM, crypto protocols and different libraries, and MAC (Mandatory access control, think SELinux and others) as a whole but that would be too big for a single thread. If you want to do that we can open a new one. Let’s attack trust and key stores in this thread.

There are also authentication agent, secret manager, password agent, polkit, etc. And all the issues about how to hold them in memory.

What’s your take on trust and key stores?

NB: A more in-depth article can be found here

Attributions:

Unknown. The author of the picture is not given, but it is possible that it is Josef Pokorný, the author of the text of the encyclopedia entry. [Public domain]

If you want to have a more in depth discussion I'm always available by email or irc. We can discuss and argue about what you like and dislike, about new ideas to consider, opinions, etc..
If you don't feel like "having a discussion" or are intimidated by emails then you can simply say something small in the comment sections below and/or share it with your friends.