Understanding the /etc/passwd and /etc/shadow files

Now that we know how to create (and delete) user accounts on our server, we are well on our way to being able to manage our users. But where exactly is this information stored? We know that users store their personal files in /home, but is there some kind of database somewhere that keeps track of which user accounts are on our system? Actually, user account information is stored in two special text files: /etc/passwd and /etc/shadow.

You can display the contents of each of those two files with the following commands. Take note that any user can look at the contents of /etc/passwd, while only root has access to /etc/shadow:

# cat /etc/passwd
# cat /etc/shadow

Go ahead and take a look at these two files (just don't make any changes), and I will help you understand them. First, let's go over the /etc/passwd file. What follows is example output from this file on my test server. For brevity, I limited the output to the last seven lines:

testuser:x:1000:1000::/home/testuser:0
testuser2:x:1006:1006:Test user,,,:/home/testuser2:/bin/bash
myuser:x:1002:1002::/home/myuser:
myuser2:x:1003:1003::/home/myuser2:
jdoe:x:1004:1004::/home/jdoe:
bsmith:x:1005:1005::/home/bsmith:/bin/bash
jdoe2:x:1007:1007::/home/jdoe2:

Each line within this file contains information for each user account on the system. Entries are split into columns, separated by a colon ":". The username is in the first column, so you can see that I've created users testuser, testuser2, myuser, and so on. The next column on each is simply an "x". I'll go over what that means a bit later. For now, let's skip to the third and fourth columns, which reference the UID and GID respectively. On a Linux system, user accounts and groups are actually referenced by their IDs. While it's easier for you and I to manage users by their names, usernames and group names are nothing more than a label placed on the UID and GID in order to help us identify with them easier. For example, it may be frustrating to try to remember that jdoe is UID 1004 on our server each time we want to manage this account. Managing it by referring to the account as jdoe is easier for us humans, since we don't remember numbers as well as we do names. But to Linux, each time we reference user jdoe, we're actually just referencing UID 1004. When a user is created, the system (by default) automatically assigns the next available UID to the account.

In my case, the UID of each user is the same as their GID. This is just a coincidence on my system and isn't always that way in practice. While I'll discuss creating groups later in this chapter, understand that creating groups works in a similar way as creating users, in the sense that the group is assigned the next available GID in much the same way as new user accounts are assigned the next available UID. When you create a user, the user's primary group is the same as their username (unless you request otherwise). For example, when I created jdoe, the system also automatically created a jdoe group as well. This is what you're actually seeing here – the UID for the user, as well as the GID for the user's primary group. Again, we'll get to groups in more detail later.

You probably also noticed that the /etc/passwd file on your system contains entries for many more users than the ones we've created ourselves. This is perfectly normal, as Linux uses user accounts for various processes and things that run in the background. You'll likely never interact with the default accounts at all, though you may someday create your own system user for a process to run as. For example, perhaps you'd create a data processor account for an automated data processing script to run under.

Anyway, back to our /etc/passwd file. The fifth column is designated for user info, most commonly the first and last name. In my example, the fifth field is blank for most except for testuser2. I created testuser2 with the adduser command, which asked me for the first and last name, which I entered as Test user. Here is that line again:

testuser2:x:1006:1006:Test user,,,:/home/testuser2:/bin/bash

In the sixth column, the home directory for our user is shown. In the case of testuser2, it's set as /home/testuser2. Finally, we designate the user's shell as /bin/bash. This refers to the default shell the user will use, which always defaults to /bin/bash. If we wanted the user to use a different shell, we can clarify that here (though shells other than /bin/bash are beyond the scope of this book). If we wanted, we could change the user's shell to something invalid to prevent them from logging in at all. This is useful for when a security issue requires us to disable an account quickly, or if we're in the mood to pull a prank on someone.

With that out of the way, let's take a look at the /etc/shadow file. We can use cat to display the contents as any other text file, but unlike /etc/passwd, we need root privileges in order to view it. So, go ahead and display the contents of this file, and I'll walk you through it:

# cat /etc/shadow

For comparison, the last few lines of my /etc/shadow file are as follows:

myuser2:$6$maFOiNL.:16809:0:99999:7:::
jdoe:$6$TPxx8Z.:16809:0:99999:7:::
bsmith:$6$KoShUY.:16809:0:99999:7:::
testuser3:$6$QAGTNqR:16809:0:99999:7::

Note

I've shortened the second column by quite a few characters in order to save space on this page. The second column in your output will appear much longer.

First, we have the username in the first column—no surprises there. Note that the output is not showing the UID for each user in this file. The system knows which username matches to which UID based on the /etc/passwd file, so there's no need to repeat that here. In the second column, we have what appears to be random gobbledygook. Actually, that's the most important part of this entire file. That's the actual hash for the user's password. If you recall, in the /etc/passwd file, each user listing had an "x" for the second column, and I mentioned I would explain it later. What the "x" refers to is the fact that the user's password is encrypted and simply not stored in /etc/passwd but is instead stored in /etc/shadow. After all, the /etc/passwd file is viewable by everyone, so it would compromise security quite a bit if anyone could just open up the file and see what everyone's password hashes are. In the days of old, you could actually store a user's password in /etc/passwd, but it's literally never done that way anymore. Whenever you create a user account on a modern Linux system, the user's password is encrypted (an "x" is placed in the second column of /etc/passwd for the user), and the actual password hash is stored in the second column of /etc/shadow to keep it away from prying eyes. Hopefully now the relationship between these two files has become apparent.

Remember earlier I mentioned that the root user account is locked out by default? Well, let's actually see that in action. Execute the following command to see root's entry in /etc/shadow:

# cat /etc/shadow | grep root

On my system, I get the following output:

root:!:16402:0:99999:7:::

You should notice right away that the root user account doesn't have a password hash at all. Instead, there's an exclamation point where the password hash would've been. In practice, placing an exclamation point here is one way to lock out an account, though the standard practice is to use an "x" in this field instead. But either way, we can still switch to the root account (which I'll show you how to do later on in this chapter). The restriction is that we can't directly log in as root from the shell or over the network. We have to log in to the system as a normal user account first.

With the discussion of password hashes out of the way, there are a few more fields within /etc/shadow entries that we should probably understand. Here's an example line from the file on my system to save you the trouble of flipping back:

jdoe:$6$TPxx8Z.:16809:0:99999:7:::

Continuing on with the third column, we can see the number of days since the UNIX Epoch that the password has last been changed. For those that don't know, the UNIX Epoch is January 1st, 1970. Therefore, we can read that column as the password having last been changed 16,809 days since the UNIX Epoch.

Note

Personally, I like to use the following command to show more easily when the password was last changed:

# passwd -S <username>

By executing this command (it must be run as root), you can view information about any account on your system. The third column of this command's output gives you the actual date of the last password change for the user.

The fourth column tells us how many days are required to pass before the user will be able to change their password again. In the case of my previous sample, testuser can change the password anytime because the minimum number of days is set to 0. We'll talk about how to set the minimum number of days later on in this chapter, but I'll give you a brief explanation of what this refers to. At first, it may seem silly to require a user to wait a certain number of days to be able to change his or her password. However, never underestimate the laziness of your users. It's quite common for a user, when required to change his or her password, to change their password several times to satisfy history requirements, to then just change it back to what it was originally. By setting a minimum number of days, you're forcing a waiting period in between password changes, making it less convenient for your users to cycle back through to their original password.

The fifth column, as you can probably guess, is the maximum number of days that can pass between password changes. If you require your users to change their passwords every certain number of days, you'll see that in this column. By default, this is set to 99,999 days. It probably stands to reason that you'll move on to bigger and better things by the time that elapses, so it may as well be infinite.

Continuing with the sixth column, we have the number of days that will elapse before the user is warned that they will soon be required to change their password. In the seventh column, we set how many days can pass after the password expires, in which case the account will be disabled. In our example, this is not set. Finally, with the eighth column, we can see the number of days since the UNIX Epoch that will elapse before the account is disabled (in our case, there's nothing here, so there is no disabled day set). We'll go over setting these fields later, but for now hopefully you understand the contents of the /etc/shadow file better.