Kerberos – A ‘Hello World’ example on CentOS/RHEL 7

Kerberos is a network authentication protocol that’s designed to allow machines to securely authenticate one another over a public network. In a typical Kerberos setup, there is a single Kerberos server and lots of kerberos clients. The Kerberos server is often referred to as the KDC server, where KDC is short for Key Distribution Center. Before you can implement kerberos, there are 2 key requirements that needs to be met:

• all boxes must have fully qualified domain names
• time needs to be accurately synced across all boxes

The best way to understand how kerberos works is to go through a working example. You can follow this example with this Kerberos vagrant project. In this vagrant project we have the following scenario:

            +------------------------------------------------+
            |                                                |
            |                Kerberos Server                 |
            |                kdc.sherc.sg-host.com               |
            |                (192.168.10.100)                |
            |                                                |
            +-------+--------------------------------+-------+
                    |                                |
                    |                                |
                    |                                |
                    v                                v
+----------------------------+               +----------------------------+
|                            |               |                            |
|  krb client 1              |     ssh       |  krb client 2              |
|  krb-client1.sherc.sg-host.com | <-----------> |  krb-client1.sherc.sg-host.com |
|  (192.168.10.101)          |               |  (192.168.10.102)          |
|                            |               |                            |
+----------------------------+               +----------------------------+

The goal is that we want to ssh from krb-client-1 to krb-client-2 (and vice versa) using Kerberos authentication instead of entering a password, or use ssh keys. So the first thing to do is to create the above setup. With vagrant this is easy, all you do is a ‘vagrant up’. But if you were to do this manually then it would mean creating the Kerberos Server, and then the Kerberos Clients and then make sure that they can all ping each other.

Kerberos Server Setup

In the RHCE exam, you don’t need to know how to setup a Kerberos server, but it is still useful to know how to do it.

We can break down the installation process into 4 stages:

1. Setup valid fqdns
2. Setup Kerberos Server
3. Register trusted entities to the Kerberos Database
4. Configure firewalld to accept kerberos related traffic
5. Locally test our setup – configure ssh

Setup the Kerberos Server

To start with, we need to mimic valid fqdns by inserting the following entries into the /etc/hosts file:

192.168.10.100 kdc.sherc.sg-host.com
192.168.10.101 krb-client1.sherc.sg-host.com
192.168.10.102 krb-client2.sherc.sg-host.com

To start with we need to install the necessary rpms:

$ yum install krb5-server krb5-workstation pam_krb5

krb5-server – this is the main kerberos server side software.

krb5-workstation – this is the keberos client side software. It’s optional to install it on the kerberos server, but is useful for Kerberos troubleshooting purposes.

pam_krb5 – this is the pam krb5 module. ssh does it’s authentication via pam, hence we need to make pam krb5 aware.

Next we need to update the following file:

[root@kdc ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 EXAMPLE.COM = {
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

Basically we need to specify a realm:

[root@kdc ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 CODINGBEE.NET = {
  master_key_type = aes256-cts
  default_principle_flags = +preauth
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

Note, you have to match the case, i.e. you need to write it in upper case.

Note, we also uncommented master_key_type and inserted the ‘default_principle_flags = +preauth’ line. That is optional, and is done so to add a bit more security hardening. But it also breaks backward compatibility to kerberos 4 and older.

Then we do the same thing to the following small file:

[root@kdc ~]# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@EXAMPLE.COM	*

Which means it ends up looking like this:

[root@kdc ~]# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@CODINGBEE.NET	*

Next we edit the following file:

[root@kdc ~]# cat /etc/krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
# default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
# EXAMPLE.COM = {
#  kdc = kerberos.example.com
#  admin_server = kerberos.example.com
# }

[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM

Here we need to activate this commented parts, and once again replace the generic information, i.e. it should look like:

[root@kdc ~]#
[root@kdc ~]# cat /etc/krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = CODINGBEE.NET
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
  CODINGBEE.NET = {
   kdc = kdc.sherc.sg-host.com
   admin_server = kdc.sherc.sg-host.com
}

[domain_realm]
 .sherc.sg-host.com = CODINGBEE.NET
 sherc.sg-host.com = CODINGBEE.NET

Now as part of our hello-world ssh test, we need to create a test user account, so in our example we’ll create the following:

[root@kdc ~]# useradd krbtest

This user actually needs to be created on the client boxes only. However we’re creating it on the kerberos server so that we can do some sanity testing locally.

Now we’re ready to create the Kerberos server’s internal database, to do this run:

[root@kdc ~]# kdb5_util create -s -r CODINGBEE.NET

This command can take several minutes. You need to choose a password when prompted.

Once that’s done, you need to enable and start the following deamons:

$ systemctl enable krb5kdc
$ systemctl enable kadmin
$ systemctl start krb5kdc
$ systemctl start kadmin

Configure firewalld to accept Kerberos related traffic

In terms of firewalld, you just need to add the kerberos and kadmin services:

$ systemctl start firewalld.service
$ firewall-cmd --add-service=kerberos --permanent
$ firewall-cmd --add-service=kadmin --permanent
$ systemctl restart firewalld.service
$ systemctl enable firewalld.service

Register trusted entities to the Kerberos Database

Next we need to start populating the Kerberos Database with entities that makes up the realm. We do this by running the kadmin.local which starts an interactive shell:

[root@kdc ~]# kadmin.local
Authenticating as principal root/admin@CODINGBEE.NET with password.
kadmin.local:

The first entity to add should be the root level kerberos entity, which is done by running the following in the kadmin.local session:

kadmin.local: addprinc root/admin

Then choose+confirm a password when prompted. Make a note of this password because it is need when (remotely) adding any additional entities in the future. Also notice how we used the ‘root/’ notation. You can think of this as meaning that we’re referring to a table name called ‘root’ inside the internal kdb database.

Now we need to register the krbtest user account to the kerberos database, so that it becomes trusted.

kadmin.local:  addprinc krbtest

Then choose+confirm a password for this account when prompted. Notice this time we didn’t use ‘{table-name}/’ notation. I think that’s because when not specified it uses the ‘user/’ as the default.

Then add the kerberos server itself as a trusted object into the kerberos database:

kadmin.local:  addprinc -randkey host/kdc.sherc.sg-host.com

Here, we’re adding a new entry to the ‘host’ table. Next get the keytab file:

kadmin.local:  ktadd host/kdc.sherc.sg-host.com

The ktadd essentially creates the following file in the background:

$ file /etc/krb5.keytab

Kerberos works using symmetric encryption. This file stores the keys for encryption/decryption.

Quick tip: to find out what other commands you can run in this interactive shell, run:

kadmin.local:  ?

Finally do a quick check of what’s in the database:

kadmin.local:  listprincs

Finally quit out of the kadmin interactive session:

kadmin.local:  quit

Locally test our setup – configure ssh

Next we’ll configure ssh so that it start’s relying on kerberos to do the authentication. This is optional, since in our scenario, we don’t need to ssh from any of the client boxes to the Kerberos server. But we are going to do this for early sanity checking purposes only.

First we’ll configure the ssh daemon side config file /etc/ssh/sshd_config. All you need to do is ensure the following line is present:

GSSAPIAuthentication yes

Then restart the sshd deamon:

[root@kdc ~]# systemctl restart sshd

Now the sshd deamon will attempt to authenticate using kerberos. However we still need to configure the ssh client side to initiate an kerberos based authentication. To do this we need edit /etc/ssh/ssh_config. Basically the following lines needs to be present:

    GSSAPIAuthentication yes
    GSSAPIDelegateCredentials yes

These lines are likely to already be present but commented out. If so then need to uncomment them and make sure they are both set to yes.

Finally we need to apply the following setting:

[root@kdc ~]# authconfig --enablekrb5  --update

Now we are ready to do a quick sanity check to test our setup.

First switch to the test user:

[root@kdc ~]# su - krbtest

Now if we try to ssh into itself, we get a password prompt:

[krbtest@kdc ~]$ ssh kdc.sherc.sg-host.com
The authenticity of host 'kdc.sherc.sg-host.com (127.0.1.1)' can't be established.
ECDSA key fingerprint is SHA256:wkwlYMdcolcOB+zOiw7cMsAb+RKTn1Tim01WJGrZ21Y.
ECDSA key fingerprint is MD5:7c:f6:cc:1a:85:f0:fa:78:02:37:61:27:f3:fb:cc:9e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'kdc.sherc.sg-host.com' (ECDSA) to the list of known hosts.
krbtest@kdc.sherc.sg-host.com's password:

So far we have failed to ssh in without the password. That’s because we don’t have an existing token to use. So we need to generate a new token. Before we generate this token, lets first check that there are no active tokens:

[krbtest@kdc ~]$ klist
klist: Credentials cache keyring 'persistent:1001:1001' not found

To get a token, we first need to initialise kerberos:

[krbtest@kdc ~]$ kinit
Password for krbtest@CODINGBEE.NET:  {enter password that you chose when registering krbtest in the krb database}

If you entered the correct password, then you don’t get any reponse. So to check if this has worked run the klist command again, it should now look something like this:

[krbtest@kdc ~]$ klist
Ticket cache: KEYRING:persistent:1001:1001
Default principal: krbtest@CODINGBEE.NET

Valid starting       Expires              Service principal
02/25/2018 19:25:26  02/26/2018 19:25:26  krbtgt/CODINGBEE.NET@CODINGBEE.NET

Now if we try again, we it should connect without any password prompt:

[krbtest@kdc ~]$ ssh kdc.sherc.sg-host.com
Last login: Sun Feb 25 19:21:24 2018
[krbtest@kdc ~]$

Success!

Kerberos Client Setup

As part of our example scenario we have 2 client boxes, so we need to repeat the following steps on both boxes in order to get our scenario working.

To start with, we need to mimic valid fqdns by inserting the following entries into the /etc/hosts file:

192.168.10.100 kdc.sherc.sg-host.com
192.168.10.101 krb-client1.sherc.sg-host.com
192.168.10.102 krb-client2.sherc.sg-host.com

Next we need to install the necessary rpms:

$ yum install krb5-workstation pam_krb5

Then edit the /etc/krb5.conf. It needs to look identical to the /etc/krb5.conf file that’s in the kerberos server. So simply copy and pasting will do the trick.

Now run the following command:

[root@krb-client1 ~]# kadmin
Authenticating as principal root/admin@CODINGBEE.NET with password.
Password for root/admin@CODINGBEE.NET:

Here you’ll get prompted to enter the root kerberos password. This is the password you chose when setting up the kerberos server. After that you will get taken to the remote version of the kadmin.local session. Next let’s see what entities are registered on the central kerberos database:

kadmin:  list_principals
K/M@CODINGBEE.NET
host/kdc.sherc.sg-host.com@CODINGBEE.NET
kadmin/admin@CODINGBEE.NET
kadmin/changepw@CODINGBEE.NET
kadmin/kdc.sherc.sg-host.com@CODINGBEE.NET
kiprop/kdc.sherc.sg-host.com@CODINGBEE.NET
krbtest@CODINGBEE.NET
krbtgt/CODINGBEE.NET@CODINGBEE.NET
root/admin@CODINGBEE.NET

At the moment our box isn’t registered so let’s register it:

kadmin:  addprinc -randkey host/krb-client1.sherc.sg-host.com

This command essentially generates a long random string that’s going to be stored on the kerberos database. This is symmetric encryption key, so we need to install a local copy of this key to. We create this local key by generating the keytab file:

kadmin:  ktadd host/krb-client1.sherc.sg-host.com

Now if we list principles:

kadmin:  list_principals
K/M@CODINGBEE.NET
host/kdc.sherc.sg-host.com@CODINGBEE.NET
host/krb-client1.sherc.sg-host.com@CODINGBEE.NET
kadmin/admin@CODINGBEE.NET
kadmin/changepw@CODINGBEE.NET
kadmin/kdc.sherc.sg-host.com@CODINGBEE.NET
kiprop/kdc.sherc.sg-host.com@CODINGBEE.NET
krbtest@CODINGBEE.NET
krbtgt/CODINGBEE.NET@CODINGBEE.NET
root/admin@CODINGBEE.NET
kadmin:

Now we can see our recently added entry. Now we exit out:

kadmin:  quit

Next we configure ssh:

First we’ll configure the ssh daemon side config file /etc/ssh/sshd_config. All you need to do is ensure the following line is present:

GSSAPIAuthentication yes

Then restart the sshd deamon:

[root@kdc ~]# systemctl restart sshd

Now the sshd deamon will attempt to authenticate using kerberos. However we still need to configure the ssh client side to initiate an kerberos based authentication. To do this we need edit /etc/ssh/ssh_config. Basically the following lines needs to be present:

    GSSAPIAuthentication yes
    GSSAPIDelegateCredentials yes

These lines are likely to already be present but commented out. If so then need to uncomment them and make sure they are both set to yes.

Finally we need to apply the following setting:

[root@kdc ~]# authconfig --enablekrb5  --update

Now you need to repeat the same steps for the krb-client2.sherc.sg-host.com box. After that you will be ready to

Test the Kerberos setup

Now we are ready to put our setup to the test. To do this test, log into one of the client boxes. In my case I’ll use. krb-client2.sherc.sg-host.com

We first need to create a test user account, so in our example we’ll create the following:

[root@krb-client1 ~]# useradd krbtest

First switch to the test user:

[root@kdc ~]# su - krbtest

[root@kdc ~]# su - krbtest

Now if we try to ssh into itself, we get a password prompt:

[krbtest@krb-client1 ~]$ ssh krb-client2.sherc.sg-host.com
The authenticity of host 'krb-client2.sherc.sg-host.com (192.168.10.102)' can't be established.
ECDSA key fingerprint is SHA256:wkwlYMdcolcOB+zOiw7cMsAb+RKTn1Tim01WJGrZ21Y.
ECDSA key fingerprint is MD5:7c:f6:cc:1a:85:f0:fa:78:02:37:61:27:f3:fb:cc:9e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'krb-client2.sherc.sg-host.com,192.168.10.102' (ECDSA) to the list of known hosts.
krbtest@krb-client2.sherc.sg-host.com's password:

So far we have failed to ssh in without the password. That’s because we don’t have an existing token to use, which you can check like this:

[krbtest@krb-client1 ~]$ klist
klist: Credentials cache keyring 'persistent:1001:1001' not found

To get a token, we first need to initialise kerberos:

[krbtest@kdc ~]$ kinit
Password for krbtest@CODINGBEE.NET:  {enter password that you chose when registering krbtest in the krb database}

If you entered the correct password, then you don’t get any reponse. So to check if this has worked run the klist command again, it should now look something like this:

[krbtest@kdc ~]$ klist
Ticket cache: KEYRING:persistent:1001:1001
Default principal: krbtest@CODINGBEE.NET

Valid starting       Expires              Service principal
02/25/2018 19:25:26  02/26/2018 19:25:26  krbtgt/CODINGBEE.NET@CODINGBEE.NET

This time we have a token, but not it expires after exactly 24 hours.

Now if we try again, we it should connect without any password prompt:

[krbtest@krb-client1 root]$ ssh krb-client2.sherc.sg-host.com
Last login: Sun Feb 25 17:17:20 2018 from 192.168.10.101
[krbtest@krb-client2 ~]$

Success!

Note: if skipped doing ‘kinit’, and instead ran the ssh command directly, then you will get a password prompt. If you then entered the password that you set during the principal adding of the krbtest user, then it will login in successfully and and create the 24 hour token for you.

Recommended reading

http://www.roguelynn.com/words/explain-like-im-5-kerberos/

[post-content post_name=rhsca-quiz]

In our quiz we’ll call our kerberos server as kdc.sherc.sg-host.com and our kerberos client as krb-client1.sherc.sg-host.com.