Administering the System

Managing Users and Groups

Understanding Users and Groups

Linux manages security control with Linux's discretionary access control (DAC).

Groups are also part of DAC. When User is created a group is given called the default group.

The user's process can have only one designated current group at a time.

Configuring User Accounts

/etc/default/useradd, /etc/login.defs, /etc/skel, user input -> User account creation process -> /home/userid, /etc/passwd, /etc/shadow, /etc/group

The /etc/login.defs File

Contains directives for use in various shadow password suite commands.

Shadow password suite: useradd, userdel, passwd commands

Two types of accounts:

User account: Also called normal account. UID in the UID_MIN and UID_MAX range.
System account: Used by daemons or special users like root. UID in the SYS_UID_MIN and SYS_UID_MAX range.

The /etc/default/useradd File

Configuration file that directs the process of creating accounts.

To view contents:

cat /etc/default/useradd
sudo useradd -D

Using useradd with -D we can also change directives.

The /etc/skel/ Directory

Contains files that are copied into the home directory of the newly created user

The /etc/passwd File

Contains account information in a single line.

If shell is /sbin/nologin or /bin/false that means that the user cannot login. To tweak the message shown by nologin we can create the /etc/nologin.txt but the false one just kicks you off.

Passwords are stored in the /etc/shadow file instead since is more permission restricted.

If we find passwords in /etc/passwd we can migrate them using pwconv.

The /etc/shadow File

<1>:<2>:<3>:<4>:<5>:<6>:<7>:<8>:<9>

1 -> username 2 -> Salted or hashed password. A !! or ! indicates that password has not been set A ! or * indicates the account can't use the password to login A ! in front of a password indicates the account has been locked 3 -> Date of last password change in Unix Epch time (days) format. 4 -> Number of days after a password is changed until the password may be changed again. 5 -> Number of days until a password change is required. This is the password's expiration date. 6 -> Number of days a warning is issued to the user prior to a password's expiration (field 5) 7 -> Number of days aftr a password has expired (field 5) and has not been changed until the account will be deactivated 8 -> Date of account's expiration in Unix Epoch time (days) format. 9 -> Called the special flag. It is a field for a special future use.

The first field is the only one shared with /etc/passwd file.

The Account Creation Process

The useradd looks at the /etc/login.defs and /etc/default/useradd.

Example user creation:

sudo useradd -md /home/Yasin -s /bin/bash Yasin

short	long
-c	--comment
-d	--home or --home-dir
-D	--defaults
-e	--expiredate
-f	--inactive
-g	--gid
-G	--groups
-m	--create-home
-M	--no-create-home
-s	--shell
-U	--uid
-r	--system

Debian distros promote use of adduser instead of useradd. Config file in /etc/adduser.conf

To view records in /etc/passwd and /etc/shadow we can employ getent.

getent passwd Yasin
sudo getent shadow Yasin #here we need sudo since shadow is more restricted

To modify directives from /etc/default/useradd we can use useradd -D followed by the directive we want to change.

useradd -D -s /bin/bash

We can create a password using the crypt utility and then add it when the account is created via the -p option. This is bad practice.

Maintaining Passwords

passwd utility is our best friend here.

With no arguments we can change our password. To change it for a certain user we pass the username as an argument.

short	long
-d	--delete
-e	--expire
-i	--inactive
-l	--lock
-n	--minimum
-S	--status
-u	--unlock
-w	--warning or --warndays
-x	--maximum or --maxdays

-S shows the status

Example output: Yasin PS 2018-10-01 0 99999 7 -1 (Password set, SHA512 crypt.) -1 indicates never

Format: username state last_password_change_date minimum maximum warning inactive

We can also use chage to list status or change account password's settings:

With no options it prompts an interactive program to change password settings. With -l we can do the same as the -S option in passwd but in a more human-readably manner.

Modifying Accounts

We can employ usermod to modify accounts.

short	long
-c	--comment
-d	--home
-e	--expiredate
-f	--inactive
-g	--gid
-G	--groups
-l	--login
-L	--lock
-s	--shell
-u	--uid
-U	--unlock

Deleting Accounts

We can employ the userdel utility.

Most common option is -r that will delete the account's home directory tree and any files within it.

Configuring Groups

If no default group is designated to a user on creation, a group with the same name as the user name is created. To view the account's group memberships:

getent passwd yasin # Returns the passwd value which contains the GID
sudo groups yasin # Shows group membership
getent group yasin # returns the GID as yasin:x:GID
grep GID /etc/group # We use the GID with the grep command

For group management we employ groupadd. Like useradd, a more friendly command called addgroup in Debian distros is recomended.

sudo groupadd -g 1042 Test # Here with the option -g we can specify the GID
getent group Test # This will return Test:x:1042 since we specified this GID
grep Test /etc/group # Here we will get the same output as above

The fields in the /etc/group are as follows: <group_name>:<passwd>:<GID>:<group_members>.

Like with users, passwords are stored as hashed values in /etc/gshadow.

To add an account to a group:

sudo usermod -aG Test Yasin # Here the a option is to preserve the other groups and G option is to add to the specified group.

Using -g we can modify the GID and using -n we can modify the name.

For group deletion we can employ groupdel.

Managing Email

Understanding Email

Linux mail system follows the Unix aproach. The mailing system is divided in several components:

Mail delivery agent (MDA): Delivers messages to a ocal user's inbox.
Mail transfer agent (MTA): Sends incoming emails to a MDA or local user's inbox. If outbound messages are for a remote system then MTA establishes a communication link with another MTA.
Mail user agent (MUA): An interface to read stored in mailboxes.

Choosing Email Software

Three popular MTA packages in the Linux world: Sendmail (complex config file), Postfix (Modular) and Exim (Flexible)

Working with Email

Sending and Receiving Email

binmail -> reference to the mail program that resides on /bin/mail or /usr/bin/mail

binmail reads messages stored in the /var/sool/mail directory but we an pass another location.

binmail is no longer included in Linux distros by default

Ubuntu: bsd-mailx CentOS: mailx

common options:

-s subject
-cc recipient
-bc recipient
-v

mail -s "TEST" yasin

To send the message just press Ctrl+D.

On Postfix non-lowercase users may not be delivered. Check /var/log/maillog or /var/log/mail.log to find undeliverable emails.

To read emails we can type mail and an interactive console will show up and the mails will be displayed with the index unmber.

Options:

N: We can read a mail using its number
q: to quit
d N: We can delete a mail using its number

Using -f option we can pass the nondefault location where mail is stored.

Using -u we can read another user's mail, if its on the default location. If not then we can use the -f option if we have enough permissions.

Checking the Email Queue

To check the email queue:

mailq
sendmail -bp
# Format -> Queue ID - Size - Arrival Time - Sender/Recipient

We can find mail files stucked in the queue dirs that are stored somewhere in /var/spool:

find /var/spool -name QueueID

We can employ rm to delete the file.

Redirecting Email

We can redirect emails using aliases. This is useful for security reasons or when a recipient have a complex username.

Two steps:

Add the alias to the /etc/aliases file
Run newaliases command to update the aliases database /etc/aliases.db

Format of the alias records: ALIAS-NAME: RECIPIENT1[, RECIPIENT2]

Example:

nano /etc/aliases
# hostmaster: yasin, root
newaliases

Now we will recieve the mails sent to hostmaster on yasin and root.

For email forwarding we can employ the .forward file in the home directory of the username we wish to forward messages from.

We do that in two steps:

Create the .forward file in the $HOME directory
We give the 644 permissions to the file

We can then add the user we want to forward messages to:

echo yasin > .forward

To stop the forwarding we can just delete the file.

Emulating Commands

Postfix suports Sendmail comands. For example:

mailq
sendmail -bp
newaliases
sendmail -I (does the same as newaliases)

Using Log and Journal Files

Examining the syslog protocol

syslog protocol facility values

code	keyword
0	kern
1	user
2	mail
3	daemon
4	auth
5	syslog
6	lpr
7	news
8	uucp
9	cron
10	authpriv
11	ftp
12	ntp
13	security
14	conosle
15	solaris-cron
16 - 23	local0-local7

syslog protocol severity values

code	keyword
0	emerg
1	alert
2	crit
3	err
4	warning
5	notice
6	info
7	debug

Viewing the History of Linux logging

sysklogd: Original syslog application with two programs, syslogd to monitor the system and apps for events and klogd to monitor the kernel for events.
syslogd-ng: Added advanced features such as message filtering and the ability to send messages to a remote host.
rsyslog: r stands for rocket-fast. The rsyslogd quickly become the standard logging package for many linux distros
systemd-journald: Part of the systemd application. It does not follow the syslog protocol.

Logging Basics Using rsyslogd

Utilizes all of the features of the original syslog protocol.

Configuring rsyslogd

Config file stored in /etc/rsyslogd.conf or /etc/rsyslog.d/ depending on the distro.

Format of a rule in the config file: facility.priority action

facility is the syslog facility keyword and the priority uses the severity keyword. The severity includes the specified severity and the higher ones (lower numbers).

To log only a specific severity we can employ an equal sign: kern.=crit

We can also use wildcard characters for the facility or priority: *.emerg

The action entry defines what syslog should do with the recieved syslog message:

Forward to a regular file
Pipe the message to an application
Display the message on a terminal or the system console
Send the message to a remote host
Send the message to a list of users
Send the message to all logged-in users

We can specify more than one facility using commas: auth,authpriv.*

We can also specify which events we do not want to handle: *.*;auth, authpriv.none

Using a minus sign on the filename tells rsyslogd to not sync the file after each write: -/var/log/kern.log

We can send the message to a user or users using omusrmsg: :omusrmsg:*

Sending Log Messages to a Log Server

Using the same format rule as before in the /etc/rsyslogd.conf we can add an entry specifying the facility.priority the same way as before but using a remote server as the action.

The remote server's format is: TCP|UDP[(z#)]HOST:[PORT#]

For example: *.* @@(z9)test.com:6555

UDP is @ and TCP is @@. The compression rate with z goes from 1 to 9. If IPv6 is used, we need to enclose it with brackets.

After that modification we need to reload the service.

Rotating Log Files

logrotate utility helps us manage logfiles.

Config file: /etc/logrotate.conf

In the config file find global directives and then specific file directives that override the global directives:

weekly
rotate 4
dateext # Use current system date. If not then numbers are used exmpl.log.1

include /etc/logrotate.d

/var/log/btmp {
    missing ok
    monthly
    create 0600 root utmp
    rotate 1
}

We can also include files from /etc/logrotate each file having a service specific name.

directive	description
hourly
daily
weekly n	n indicates day of the week stating from 0. 7 indicates every 7 days
monthly
size n	nothing is KB then we have K, M and G
rotate n	if equal to 0 files are deleted instead of rotated
dateformat format-string
missingok	Do not issue error message if log file is missing
notifempty	If log file empty, do not rotate it

We can check the logritate status file in /var/lib/logrotate. In ubuntu the file is status and in CentOS logrotate.status.

Making Log Entries

To log events we can employ logger:

logger [-isd] [-f file] [-p priority] [-t tag] [-u socket] [message]

-i specifies the PID
-s send event message to the standard error output
-u socket
-d advanced option for network
-f file that contains the message
-p priority
-t tag

Finding Event Messages

Good way to troubleshoot: tail -f logfile

Journaling with systemd-journald

Configuring systemd-journald

Config file: /etc/systemd/journald.conf

directive	description	default
Storage=	auto,persistent,volatile or none	auto
Compress=	yes or no	yes
ForwardToSyslog=	yes or no	yes
ForwardToWall=	yes or no	yes
MaxFileSec=	number followed by month, week or day. 0 to turn feature off	1month
RuntimeKeepFree=	number followed by K, M or G (amount that needs to be kept free when employing volatile storage)	15% of current space
RuntimeMaxFileSize=	number followed by K, M or G (max amount of space journal files can consume if storage is volatile)
RuntimeMaxUse=	number followed by K, M or G (max amount that can be used on volatile storage)	10% of current space
SystemKeepFree=	number followed by K, M or G (same as with runtime but for persistent storage)	15% of current space
SystemMaxFileSize=	number followed by K, M or G (same as with runtime but for persistent storage)
SystemMaxUse=	number followed by K, M or G number followed by K, M or G (same as with runtime but for persistent storage)	10% of current space

For the Storage directive options:

auto: Event messages saved in /var/log/journal dir. If this dir doesn't exist it stores event messages in the temporary /run/log/journal which is deleted when the system shuts down.
persistent: Automatically create /var/log/journal if it doesn't exist and save event messages there.
volatile: Forces systemd-journald to store event messages only in /run/log/journal directory.
none: Event messages are discarded.

Looking at Journal Files

System journal files are saved as system.journal user journal files are saved as user-UID.journal.

Archived journal are saved as user-UID|system@chars.journal

On some systems we can employ the journalctl --rotate-command

Layering Your Logging

We can have both systemd-journald and a syslog protocol app such as rsyslog running and working together.

Two methods:

Journal Client Method: syslog protocol program acts as a journal client. Tipically configured by default. We can check for imuxsock and/or imjournal modules in the /etc/rsyslog.conf file.
Forward to Syslog Method: Employs the /run/systemd/journal/syslog as a socket where syslog protocol program can read. We need to set the ForwardToSyslog directive to yes on the /etc/systemd/journal.conf file. For this changes to take efect we must restart the service systemctl restart systemd-journald.

Making the Journal Persistent

Setting the Storage directive to persistent will create the /var/log/journal dir.

Viewing Journal Entries

To view entries we can employ the journalctl program. By default employs the less pager, we can avoid that using --no-pager.

journalctl [OPTIONS...] [MATCHES...]

Options:

short	long
-a	--all
-e	--pager-end
-k	--dmesg (only kernel entries)
-n number	--lines=number
-r	--reverse
-S date	--since=date (Can be YYYY-MM-DD:HH:MM:SS) if time specification is left then 00:00:00 is asumed. Can also employ yesterday, today, tomorrow and now.
-U date	--until=date
-u unit or pattern	--unit=unit or pattern

Matches:

match	description
field	match a specific field in the journal
OBJECT_PID=pid
PRIORITY=value	keyword(number): emerg(0), alert(1), crit(2), err(3), warning(4), notice(5), info(6), debug(7)
_HOSTNAME=host
_SYSTEMD_UNIT	nothing is KB then we have K, M and G
_TRANSPORT	if equal to 0 files are deleted instead of rotated
_UDEV_SYSNAME=dev
_UID=userid	Do not issue error message if log file is missing

Example:

sudo journalctl --since=today _SYSTEM_UNIT=ssh.service

To follow the journal like we do with tail -f we can employ -f or --follow.

Maintaining the Journal

We can check the systemd-journald disk usage and remove archived journal files based on time and size:

journalctl --disk-usage
journalctl --vacuum-size=10K # N(K,M,G,T)
journalctl --vacuum-time=10months # N(s,min,h,days,months,weeks,years)

If we want to back up journal files we first sync the entries from the queue into the file and then we copy the files:

journalctl --sync

Viewing Different Journal Files

We can point to another directory using -D or --directory option.

If wthe dir we are pointing to has a diferent name than system.journal or UID.journal we can employ the --file option with the pattern that can be the full name of the file or file globbing.

We can also merge several journal files with the -m or --merge option. This option doesn't save the output in a new file, it just outputs the merged content.

To send the journals to a centralized journal host system we can do so via systemd-journal-remote.

To view all the various journal files from the centralized host we will have to employ the previously shown merge option.

Making Journal Entries

We can employ systemd-cat tool.

To do so we need to pipe the program's STDOUT into the utility: command | systemd-cat.

If we are using syslog protocol program as a journal client we can use the logger utility.

Maintaining the System Time

Understanding Linux Time Concepts

Local time is also called wall time.

Coordinated Universal Time (UTC) is the same across all countries.

Two types of time clocks:

Software based
Hardware based. This one is called real-time clock also. It uses the system battery (CMOS battery) when the machine is powered down

Viewing and Setting Time

Using the hwclock Utility

Primarily used for the hardware clocks.

Short Option	Long option
	--localtime
-r	--show
-s	--hctosys
-u	--utc
-w	--systohc

Using the date Utility

Software clock primarly uses the date command or the timedatectl utility.

Syntax: date [-u|--utc|--universal] [MMDDhhmm[[CC]YY][.ss]]

Without options it shows the date.

If we are specifying localtime there is no need to use switches but for UTC we can employ -u and so on.

We may not be able to set the time using date if the system is using NTP or SNTP.

To check that:

systemctl status ntpd
systemctl status chronyd
systemctl status systemd-timesyncd

if any of those are running then the system is employing NTP or SNTP.

Using timedatectl Utility

Without options shows the local time, time zone, and more information.

To set the system time:

timedatectl set-time "YYYY-MM-DD HH:MM:SS"

If automatic time sync is enabled we will get an error. To disable it:

timedatectl set-ntp 0

We can also flip-flop the clock between using the UTC standard and the localtime with set-local-rtc Bool being 1 for localtime and 0 for UTC.

Understanding the Network Time Protocol

Read pages 408-410.

Using the NTP Daemon

Configuring the NTP daemon

ntpd is the daemon name.

Config file: /etc/ntp.conf

The iburst directive helps to speed up the initial time sync.

Port used: 123

insane time: This happened on older Linux systems when the system's time was more than 17 minutes different than real time and NTP servers would not talk to your system because of it.

On newer Linux Systems this does not happen. When it happened back then the fix was to change manually the time and then starting ntpd.

We can employ the ntpdate utility for this case.

ntpdate 0.pool.ntp.org

Then we start the service:

systemctl start ntpd
systemctl enable ntpd

After that we wait 10-15min and chck the status:

ntpstat

Managing the NTP Service

To view server and polling information

ntpq -p

Using the chrony Daemon

Its better than ntpd in a lot of things. The daemon name is chronyd

Configuring the chrony Daemon

Config file: /etc/chrony.conf or /etc/chrony/chrony.conf

The config file contains several items like the NTP servers to use.

pool directive: designates a pool server directive: single time server

maxresources directive is used to designate the maximum number of time servers from the designated source.