Generate self signed SSL certificates for local development

Hi all,

Nowadays it's quite common to have a site or web page with TLS enabled, or even communicate between services using TLS to make the communication secure, and there are a lot of options to generate those certificates (even free with Let's encrypt), but what happens when you are developing your own service or site and you want to check these secure communication in your local machine? Here is where self signed certificates come to the rescue.

Depending on your needs you will need to generate one or more certificates for your local environment. On the most simple example, you will need just one certificate, in which case it's enough to generate a self signed certificate and trust that certificate on your browser, but in more complex scenarios you will need more than one certificate and trust them all not just on your browser, but also during the communication between systems, so for this cases is good to have a root certificate and one or more intermediate certificates so you can trust directly the root certificate or the intermediate certificates that the service or the browser uses.

In this post we are going to review how to generate one or more certificates from a self signed root and intermediate certificates and how to add them to your local environment so you don't see any errors in your browser or when two services communicate using TLS.

What is TLS?

You might be wondering, what the hell is TLS and why should I use it? Well, TLS (Transport Layer Security) and its predecessor SSL (Secure Sockets Layer) are protocols for establishing authenticated and encrypted links between networked computers, so it allows a secure communication between two programs (i.e., your browser and a website) by encrypting that communication, making sure that no one even if it's listening to your communications, can know what the two programs are talking about.

Although the SSL protocol was deprecated with the release of TLS 1.0 in 1999, it is still common to refer to these related technologies as “SSL” or “SSL/TLS.” The most current version is TLS 1.3, defined in RFC 8446 (August 2018), and it can use different encryption algorithms (ones more secure than others) to cipher the messages passed between the two programs.

Since it's not the subject of this post, I'm not going to give details on the theoretical concepts of TLS, encryption, algorithms and so on, since you have a lot of resources (books, videos, posts, ...) that explains in deep (and much better than me) all this theory, but you can start learning more about this in the Wikipedia.

Generate a self signed root certificate

So lets start by creating a self signed root certificate for you local environment. A root certificate is a certificate issued by a trusted Certificate Authority (CA), which is usually a trusted entity that certifies that you are who you say you are, so any software that trust that Certificate Authority can validate against it that your certificate is valid and that you are not impersonating anyone. This validation is done by signing the root certificate with a private key owned by the CA and, on the certificate consumer side, by validating the signature with the public key associated with the private key. To learn more about PKI (Public Key Infrastructure) you can check again Wikipedia.

Since the root certificate is for your local development and (I hope) you trust yourself, you can generate a self signed root certificate that we are going to use to generate other certificates. To generate the root certificate we are going to install OpenSSL, a well known free software project to work with TLS, which in a Debian based GNU/Linux distribution you can do by running: $ apt-get install openssl


Once installed OpenSSL we are going to create a directory structure to store both the private keys and public certificates that we are going to generate, and we are going to create the initial files to be able to generate the certificates: $ mkdir -p tls/certs
$ mkdir -p tls/private
$ echo "01" > tls/serial
$ chmod 0644 tls/serial
$ touch tls/index.txt
$ chmod 0644 tls/index.txt


Once we have the basic structure we want to generate a configuration file for OpenSSL so we can use it when we generate the certificates. You have an example of the configuration file at `/etc/ssl/openssl.cnf`, so you can copy that file and change the `dir` variable to indicate the directory we have just created. After that we can finally create the root private key by running the following commands: $ echo "superCoolPassword" > tls/pwd
$ openssl enc -aes256 -pbkdf2 -salt -in tls/pwd -out tls/pwd.enc
$ chmod 0644 tls/pwd.enc
$ rm -f tls/pwd
$ openssl genrsa -des3 -passout file:tls/pwd.enc -out tls/private/cakey.pem 4096
$ openssl rsa -noout -text -in tls/private/cakey.pem -passin file:tls/pwd.enc


As you see, with the first command we are storing a password used later to generate the private key. That password can be whatever you want, but if your imagination is as good as mine you can use `pwgen` to generate the password.

Since we don't want that password to be in clear text, we are going to cipher the file with OpenSSL using the algorithm AES 256 (which is good enough for a local environment) and we're going to write that ciphered content to the pwd.enc file, which will be the input for the following OpenSSL commands. This command will ask for an encryption password, which can be use to decipher back the password.

With the third command we have finally generated the (encrypted) private key for the root certificate, which it's placed in the `tls/private/cakey.pem` directory. Finally, the last command just check that the private key is valid, so you can skip it if you don't want to do that check.

After generating the private key we want to generate the public certificate that belongs to that private key: $ openssl req -new -x509 -days 3650 -passin file:tls/pwd.enc \
    -config tls/openssl.cnf \
    -extensions v3_ca \
    -key tls/private/cakey.pem \
    -subj "/C=ES/ST=Andalucia/L=Cordoba/O=Ritho/CN=Root certificate" \
    -out tls/certs/cacert.pem
$ openssl x509 -in tls/certs/cacert.pem -out tls/certs/cacert.pem -outform PEM
$ openssl x509 -noout -text -in tls/certs/cacert.pem


The first command is creating the public certificate, which will use the `subj` parameter as the certificate information. As you see, we are using the file `pwd.enc` we had generated before with the encrypted password so we can decrypt the private key to generate the public certificate.

If you want to use this certificate directly in your project remember to put in the `CN` (Common Name) the domain of your development environment, so you can avoid problems with the DNS verification of the certificate, but, since we will generate more certificates from this certificate, we won't set the actual domain in the common name.

After generating the certificate, the second command will change its to PEM, which is quite common to use in web servers, and the third command will check that the certificate generated is valid. Once the commands are executed you can find the certificate at `tls/certs/cacert.pem`.

Generate an intermediate certificate

Once we have the root private key and public certificate we can generate one or more intermediate certificates to use it to generate the final certificates. Technically there's no need to generate an intermediate certificate, but it's a common good practice, mainly because you can generate as many intermediate certificates as you want (i.e., one for each domain or one for each subsystem), so if one of the intermediate certificates is compromised you don't need to generate all the certificates again, just the ones depending on that intermediate certificate. Also, when renewing a certificate (root or intermediate), you need to renew too all the certificates that depends on that certificate, so it's quite common to have a long expiration period for the root certificate (5 or 10 years, for example) and a shorter expiration period for the intermediate certificate (2 or 3 years, for example).

To generate an intermediate certificate we will begin, as before, creating the file and directory structure for the certificate: $ mkdir -p tls/intermediate/certs
$ mkdir -p tls/intermediate/csr
$ mkdir -p tls/intermediate/private
$ echo "01" > tls/intermediate/serial
$ chmod 0644 tls/intermediate/serial
$ echo "01" > tls/intermediate/crlnumber
$ chmod 0644 tls/intermediate/crlnumber
$ touch tls/intermediate/index.txt
$ chmod 0644 tls/intermediate/index.txt
$ cp tls/openssl.cnf tls/intermediate/openssl.cnf


As you see, the main difference with the previous certificate is that we have added a directory and a file to store the certificate signing request (or csr), which is the proper mechanism to request a new certificate based on a previous certificate. For the openssl configuration file, we are going to reuse the one we used for the root certificate but changing the following values:

• dir: We will set it to the new directory we have created for the certificate.
• certificate: We will change it to `$dir/certs/intermediate.cacert.pem`, which will be the filename of the public intermediate certificate.
• private_key: We will change it to `$dir/private/intermediate.cakey.pem`, which will be the filename of the intermediate private key.
• policy: We will set it to `policy_anything`, which will apply the proper policy for the certificate subject.

After creating the proper configurations, we can proceed to generate the private key, csr and certificate for the intermediate CA: $ openssl genrsa -des3 -passout file:tls/pwd.enc -out tls/intermediate/private/intermediate.cakey.pem 4096
$ openssl req -new -sha256 -config tls/intermediate/openssl.cnf \
    -passin file:tls/pwd.enc \
    -key tls/intermediate/private/intermediate.cakey.pem \
    -subj "/C=ES/ST=Andalucia/L=Cordoba/O=Ritho/CN=Intermediate certificate" \
    -out tls/intermediate/csr/intermediate.csr.pem
$ openssl ca -config tls/intermediate/openssl.cnf -extensions v3_intermediate_ca -days 2650 -notext -batch \
    -passin file:tls/pwd.enc \
    -keyfile tls/private/cakey.pem \
    -cert tls/certs/cacert.pem \
    -in tls/intermediate/csr/intermediate.csr.pem \
    -out tls/intermediate/certs/intermediate.cacert.pem


The first command will generate the private key for the intermediate certificate using the same key as the root certificate to encrypt the private key. You can choose to use a new password to encrypt the private key, in which case you will use the same commands of the previous section to generate the ciphered password, but, for the sake of simplicity (and because we're just generating the certificates for the local environment), we're just using the same password.

The second command will generate the certificate signing request (csr) for the intermediate certificate. As you can gather, it's basically a certificate that we are requesting to sign with the root private key, so it will have the certificate subject information (including the common name, CN, of the intermediate certificate) using the openssl configuration file we have changed for the intermediate certificate.

Finally, with the third command we are generating the intermediate public certificate, valid for 2650 days and signed by the root private key (that's why we're passing the root certificate and private key and the csr file). Once executed, you can find the intermediate public certificate in `tls/intermediate/certs/intermediate.cacert.pem`, which you can use later on to generate the final certificates. It worth notice that we are generating a CA certificate, as indicated in the command, which means that the certificate will be use to validate other certificates, not to validate a specific domain.

Sometimes it's useful to have a chained certificate file, for example, to import in the trusted certificates of the browser or your system. You can generate a chain certificate by running the following commands: $ cat tls/intermediate/certs/intermediate.cacert.pem \
tls/certs/cacert.pem > tls/intermediate/certs/ca-chain-bundle.cert.pem
$ openssl verify -CAfile tls/certs/cacert.pem tls/intermediate/certs/ca-chain-bundle.cert.pem


The first command will generate a new file with the chain certificate (i.e., the root and the intermediate certificates in the same file), and the second command will check that the chain certificate is correct.

Generate a domain certificate

Once we have the root and the intermediate certificates, we can finally generate a certificate for a specific domain using the intermediate certificate as a CA. As before, the first step is creating the directory structure and the proper configurations: $ mkdir -p tls/ritho/certs
$ mkdir -p tls/ritho/csr
$ mkdir -p tls/ritho/private
$ echo "01" > tls/ritho/serial
$ chmod 0644 tls/ritho/serial
$ echo "01" > tls/ritho/crlnumber
$ chmod 0644 tls/ritho/crlnumber
$ touch tls/ritho/index.txt
$ chmod 0644 tls/ritho/index.txt
$ cp tls/openssl.cnf tls/ritho/openssl.cnf


For the openssl configurations file we are going to change the following values:

• dir: We will set it to the new directory we have created for the certificate.
• certificate: We will change it to `$dir/certs/ritho.cacert.pem`, which will be the filename of the public certificate.
• private_key: We will change it to `$dir/private/ritho.cakey.pem`, which will be the filename of the private key.
• policy: We will set it to `policy_anything`, which will apply the proper policy for the certificate subject.
• nsComment: We will change it to "Ritho host certificate"
• DNS.1: You will set this domain to your local domain, in my case `karpov.ritho.net`, which is the hostname of my machine.
• DNS.2: You will set this domain to your wildcard local domain, in my case `*.karpov.ritho.net`.

After that, you can create the private key and the public certificate for your domain: $ openssl genrsa -passout file:tls/pwd.enc -des3 -out tls/ritho/private/ritho.key 4096
$ openssl rsa -passin file:tls/pwd.enc -in tls/ritho/private/ritho.key -text > tls/ritho/private/ritho-key.pem
$ openssl rsa -in tls/ritho/private/ritho-key.pem -out tls/ritho/private/ritho-key.pem
$ openssl req -new -sha512 -config tls/ritho/openssl.cnf \
    -extensions v3_req \
    -passin file:tls/pwd.enc \
    -key tls/ritho/private/ritho.key \
    -subj "/C=ES/ST=Andalucia/L=Cordoba/O=Ritho/CN=karpov.ritho.net" \
    -out tls/ritho/csr/ritho.csr
$ openssl x509 -req -sha512 -days 3650 \
    -extfile tls/ritho/openssl.cnf \
    -extensions host_cert \
    -passin file:tls/pwd.enc \
    -in tls/ritho/csr/ritho.csr \
    -CA tls/intermediate/certs/intermediate.cacert.pem \
    -CAkey tls/intermediate/private/intermediate.cakey.pem \
    -CAcreateserial \
    -out tls/ritho/certs/ritho.cert.pem
$ cat tls/ritho/certs/ritho.cert.pem \
    tls/intermediate/certs/intermediate.cacert.pem \
    tls/certs/cacert.pem > tls/ritho/certs/chain.cert.pem
$ openssl verify -CAfile tls/intermediate/certs/ca-chain-bundle.cert.pem tls/ritho/certs/chain.cert.pem


The first command, as the previous steps, generates the private certificate for the new domain using the same encrypted password as the root and intermediate certificates. After that, the second and third commands generate a PEM file for the private key, since the web servers work better with PEM files than with encrypted keys.

After that we generate the CSR file and the public certificate as we did with the intermediate certificate, but using `karpov.ritho.net` in the Common Name (CN), and, after that, we generate the public certificate for the domain. The main difference with the intermediate certificate is that, instead of creating a CA certificate, we are creating a x509 certificate using the intermediate private key and certificate as the authority to make sure that the certificate is valid.

Finally, in the last two commands we are generating the chain certificate including the root, the intermediate and the final public certificates and we are validating it with `openssl verify`.

After all this steps, you finally have a certificate for your local environment. You can use the certificate `tls/ritho/certs/ritho.cert.pem` by itself or, more likely, the chain certificate to have TLS on your site or service locally.

How to trust the certificate in your local environment?

Now that you have your certificates generated, it's a good idea to include the root and the intermediate certificates in the proper places to indicate they can be trusted.

There are a couple of places you probably want to put those certificates to make easier your life, in the nss database of your browser and where the certificates are stored in your OS for other services/programs to use.

You can add the certificates to the trusted CA of your browser using the graphical interface, but to make it funnier lets do it with a command line, so you can add it to a script or a build target of your project, and for that we're going to use the tool `certutil`. In a debian basically GNU/Linux distribution you can find the tool installing `libnss3-tools`: $ apt-get install libnss3-tools


After installing the package, you can add the certificate to any nss database, which is what the main browsers use to store the CA certificates they trust. Before adding the certificate, you need to locate the file where the database is located, for example, in chrome/brave it's usually located in ~/.pki/nssdb, and for Librewolf it's located in your profile subdirectory under ~/.librewolf. Once located the database file you can add the certificates by running the commands: $ certutil -A -n "Root certificate" -t "TC,TC,TC" -i "tls/certs/cacert.pem" -d dbPath
$ certutil -A -n "Intermediate certificate" -t "TC,TC,TC" -i "tls/intermediate/certs/intermediate.cacert.pem" -d dbPath


Where `dbPath` is the path of the nss database of your browser. The parameter `-t` of certutil indicates the trust arguments, where `T` means trusted CA for client authentication and `C` means Trusted CA. Once you are done with the CA certificates you can remove them from the database using the following commands: $ certutil -D -n "Root certificate" -d dbPath
$ certutil -D -n "Intermediate certificate" -d dbPath


If you want to know more about `certutil` you can run `man certutil` to check the documentation.

Finally, to add the root and intermediate certificates to your system you need to install the package `ca-certificates` and run the following commands: $ sudo mkdir -p /usr/local/share/ca-certificates
$ sudo cp tls/certs/cacert.pem /usr/local/share/ca-certificates/my-root-certificate.pem
$ sudo cp tls/intermediate/certs/intermediate.cacert.pem /usr/local/share/ca-certificates/my-intermediate-certificate.pem
$ sudo update-ca-certificates


When you are done with the root and the intermediate certificates remember to remove them also from your system: $ sudo rm -f /usr/local/share/ca-certificates/my-root-certificate.pem
$ sudo rm -f /usr/local/share/ca-certificates/my-intermediate-certificate.pem
$ sudo rm -f /etc/ssl/certs/my-root-certificate.pem
$ sudo rm -f /etc/ssl/certs/my-intermediate-certificate.pem
$ sudo update-ca-certificates


Conclusions

As you have seen, create your own self signed certificates for your local development is not difficult, but it can be a bit tedious, so I'd suggest you integrate it in your development process by creating a script to generate the certificates and add them to the proper places (so it can be trusted) or by putting it in your Makefile (or the tool you use to build your system). In the future I'll probably publish the needed shell scripts in a repository to create the certificates automatically (in which case I'll update this post), but in the meantime I hope this helps you to understand a bit better how the TLS certificates work and how you can integrate them in your local development.

Happy Hacking!