gnu-linux.org

#+title: GNU/Linux
#+date: 2023-08-24

* System
** System organization
Operating system usually segregates virtual memory into *user space* and *kernel space*. Primarily, this separation serves to provide memory protection and hardware protection from malicious or errant software behaviour.

#+begin_src

  +-------------------------------------------------------------------------+
  |       |User        |    GUI, Servers, Shell, GIMP, Blender, Mozilla     |
  |       |applications|                                                    |
  |       |-----------------------------------------------------------------|
  |User   |System      |init     |System    |Window    |Graphics:|Other     |
  |mode   |components  |daemon:  |daemons:  |manager:  |         |libraries:|
  |       |            |systemd  |polkitd   |X11       |AMD      |GTK       |
  |       |-----------------------------------------------------------------|
  |       |C standard  |   fopen, execv, malloc,... (up to 2000 sub)        |
  |       |library     |   glibc aims to be fast, musl aims to be lightw    |
  |-------------------------------------------------------------------------|
  |       |            |   stat, splice, dup, read, open, ioctl, write      |
  |       |            |   mmap, close, exit,... (about 380 system calls)   |
  |       |            |   System Call Interface (SCI), aims to be POSIX    |
  |       |            |----------------------------------------------------|
  |Kernel |  Linux     |Process   |IPC      |Memory    |Virtual  |Networking|
  |mode   |  kernel    |scheduling|subsystem|management|files    |subsystem |
  |       |            |subsystem |         |subsystem |subsystem|          |
  |       |            |----------------------------------------------------|
  |       |            | Other components: ALSA, DRI, evdev, klibc, LVM     |
  |       |            | device mapper, Linux Network Scheduler, Netfilter  |
  |       |            | Linux Security Modules: SELinux, TOMOYO, AppArmor  |
  |-------------------------------------------------------------------------|
  |    Hardware (CPU, Main Memory, Data Storage Devices, Network Ports)     |
  +-------------------------------------------------------------------------+

#+end_src

** User
*** User
Is an entity that can run processes and own files. Users exist primarily to support permissions and boundaries. Every user-space process has a user owner, and processes are said to run as the owner. A user may terminate or modify the behavior of its own processes (within certain limits), but it cannot interfere with other users’ processes.

The *root* user is an exception to the preceding rules because root may terminate and alter another user’s processes and access any file on the local system. For this reason, root is known as the superuser.

*Groups* are sets of users. The primary purpose of groups is to allow a user to share file access to other members of a group.

*** User space
Is the main memory that the kernel allocates for user processes. If a process makes a mistake and crashes, the consequences are limited and can be cleaned up by the kernel.

Because a process is simply a state (or image) in memory, user space also refers to the memory for the entire collection of running processes.

The *bottom level* of user space tends to consist of small components that perform single, uncomplicated tasks (Network configuration, Communication Bus, Diagnostic Logging,..). The *middle level* has larger components such as mail, print, and database services. Finally, components at the *top level* perform complicated tasks that the user often controls directly (Web Browser, User Interface, ...).

*** User space starts in roughly this order
1. ~init~.
2. Essential low-level services, such as ~udevd~ and ~syslogd~.
3. Network configuration.
4. Mid- and high-level services (cron, printing, and so on).
5. Login prompts, GUIs, and high-level applications, such as web servers.

*** Packaging systems
**** Brief
You may know some software like Firefox, GIMP, etc, but what they really are is just lots and lots of files that have been compiled into one. The people (or sometimes a single person) that write this software are known as upstream providers, they compile their code and write up how to get it installed.

These upstream providers work on getting new software out and update existing software. When they are ready to release it to the world, they send their package to package maintainers who handle getting this piece of software in the hands of the users. These package maintainers review, manage, and distribute this software in the form of packages. So instead of downloading packages/software one by one from random sites (like on Windows), there is something better -- called package repositories in Debian.

Repositories are just a central storage location for packages. Many online repositories hold lots of packages. Your machine doesn't know where to look for these repositories unless you explicitly tell it where to look. Debian already comes with pre-approved sources to get packages from and this is how it installs all the base packages you see on your system (if a user did a net-install). On a Debian system, this sources file is the ~/etc/apt/sources.list~ file. Your machine will know to look there and check for any source repositories you added.

**** Package Files
The basic unit of software in a packaging system is the *package file*. A package file is a compressed collection of files that comprise the software package. A package may consist of numerous programs and data files that support the programs. In addition to the files to be installed, the package file also includes metadata about the package, such as a text description of the package and its contents. Additionally, many packages contain pre- and post-installation scripts that perform configuration tasks before and after the package installation.

Package files are created by a person known as a *package maintainer*, often (but not always) an employee of the distribution vendor. The package maintainer gets the software in source code form from the *upstream provider* (the author of the program), compiles it, and creates the package metadata and any necessary installation scripts. Often, the package maintainer will apply modifications to the original source code to improve the program's integration with the other parts of the Linux distribution.

**** Repositories
While some software projects choose to perform their own packaging and distribution, most packages today are created by the distribution vendors and interested third parties.  Packages are made available to the users of a distribution in central repositories that may contain many thousands of packages, each specially built and maintained for the distribution.

A distribution may maintain several different repositories for different stages of the software development life cycle. For example, testing, development and also have related third-party repositories.

**** Dependencies
Programs are seldom “standalone”; rather they rely on the presence of other software components to get their work done. Common activities, such as input/output for example, are handled by routines shared by many programs. These routines are stored in what are called *shared libraries*, which provide essential services to more than one program. If a package requires a shared resource such as a shared library, it is said to have a *dependency*. Modern package management systems all provide some method of *dependency resolution* to ensure that when a package is installed, all of its dependencies are installed, too.

**** High and Low-level Package Tools
Package management systems usually consist of two types of tools.

- Low-level tools which handle tasks such as installing and removing package files, like ~dpkg~.
- High-level tools that perform metadata searching and dependency resolution, like ~apt~.

~apt~ provides a high-level command line interface for debian's package management system, with all the necessary tools in one place. ~apt~ aims to provide a simple, efficient way of handling packages for end users.

**** Debian packages
***** Brief
All packages that are included in the official Debian distribution are free according to the Debian Free Software Guidelines. This assures free use and redistribution of the packages and their complete source code. The official Debian distribution is what is contained in the /main/ section of the Debian archive.

As a service to our users, we also provide packages in separate sections that cannot be included in the /main/ distribution due to either a restrictive license or legal issues. They include:

- contrib :: packages in this area are freely licensed by the copyright holder but depend on other software that is not free.
- non-free :: packages in this area have some onerous license condition restricting use or redistribution of the software.
- non-free-firmware :: this section is restricted to device firmware. While packages in this area basically have the same underlying license restrictions as those in non-free, a special exception has been made for them: even if not being DFSG-free, packages from non-free-firmware are allowed to be included in the official installation images.

*Note* that same packages might appear in several distributions, but with different version numbers.

***** Package lists
Package lists in the:
- stable distribution :: this is the latest official release of the Debian distribution. This is stable and well tested software, which changes only if major security or usability fixes are incorporated. The current "stable" distribution is version 12, codenamed /bookworm/. It was initially released as version 12.0 on June 10th, 2023.
- testing distribution :: this area contains packages that are intended to become part of the next stable distribution. There are strict criteria a package in unstable must obey before it can be added to testing. Note that "testing" does not get the timely security updates from the security team. The main advantage of using this distribution is that it has more recent versions of software. The current "testing" distribution is /trixie/.
- unstable distribution ::  this area contains the most recent packages in Debian. Once a package has met our criterion for stability and quality of packaging, it will be included in testing. "unstable" is also not supported by the security team. The "unstable" distribution is always called /sid/.

Corollaries to this in the commercial world are /Development/, /Testing/, and /Production/.

***** Unstable
Debian Unstable is not strictly a release, but rather a rolling development version of the Debian distribution containing the latest packages that have been introduced into Debian.

While other release code names progress in time from being "testing" to being "stable", Sid is forever doomed to being unstable. Sid will always be the unstable branch. When the current "testing" repository becomes mature and is released, "testing" becomes the latest "stable" release. From there, a new "testing" repository will be created with the next planned code name, and packages will continue to trickle down from Sid into "testing" just as before.

Sid is where packages go after they've been uploaded by their maintainer, and cleared for release by the FTP master. When packages have met certain criteria, they are automatically moved from Sid to the current "testing" repository. The "Unstable" repository is updated every 6 hours.

Sid exclusively gets security updates through its package maintainers. The Debian Security Team only maintains security updates for the current "stable" release.

***** Unstable installation
Ways to get set up on Sid:

1. Use the current "stable" installer to install a minimal stable system (*recommended*).
   - Change your apt sources to point to "unstable".
   - Run ~apt update~ and ~apt full-upgrade~.

2. Use the current "testing" installer to install a minimal (next) stable system.
   - Change your apt sources to point to "unstable".
   - Run ~apt update~ and ~apt full-upgrade~.

3. Use the Unstable "mini.iso" image.
   - Download the "mini.iso" for your CPU architecture located here: https://d-i.debian.org/daily-images/ under */daily/netboot/
   - During the installation choose "Advanced options" -> "Expert install".
   - In the step "Choose a mirror of the Debian archive" choose version "sid - unstable"

If you are already using a stable system use the method described above for upgrading to unstable changing your apt sources from "stable" (or the codename for the current stable) to "unstable".

***** sources.list
****** Format
The entries in this file normally follow this format:

#+begin_src sh

  deb http://site.example.com/debian distribution comp1 comp2 comp3
  deb-src http://site.example.com/debian distribution comp1 comp2 comp3

#+end_src

****** Archive type
The first word on each line, ~deb~ or ~deb-src~, indicates the type of archive. *Deb* indicates that the archive contains binary packages (deb), the pre-compiled packages that we normally use. *Deb-src* indicates source packages, which are the original program sources plus the Debian control file (.dsc) and the diff.gz containing the changes needed for packaging the program.

****** Repository URL
The next entry on the line is a URL to the repository that you want to download the packages from.

****** Distribution
The 'distribution' can be either the release code name / alias ( stretch, buster, bullseye, bookworm, sid) or the release class (oldoldstable, oldstable, stable, testing, unstable) respectively.

****** Component

- main :: consists of DFSG-compliant packages, which do not rely on software outside this area to operate. These are the only packages considered part of the Debian distribution.

- contrib :: packages contain DFSG-compliant software, but have dependencies not in main (possibly packaged for Debian in non-free).

- non-free :: contains software that does not comply with the DFSG.

****** Examples sources.list
******* Debian Stable
Below is an example of a sources.list for Debian 12/Bookworm (stable) released 10th June 2023.

#+begin_src sh

  deb http://deb.debian.org/debian bookworm main non-free-firmware
  deb-src http://deb.debian.org/debian bookworm main non-free-firmware

  deb http://deb.debian.org/debian-security/ bookworm-security main non-free-firmware
  deb-src http://deb.debian.org/debian-security/ bookworm-security main non-free-firmware

  deb http://deb.debian.org/debian bookworm-updates main non-free-firmware
  deb-src http://deb.debian.org/debian bookworm-updates main non-free-firmware

#+end_src

If you need the contrib, non-free and non-free-firmware components, add contrib non-free non-free-firmware after main. For example, for Debian 12/Bookworm:

#+begin_src sh

  deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
  deb-src http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware

  deb http://deb.debian.org/debian-security/ bookworm-security main contrib non-free non-free-firmware
  deb-src http://deb.debian.org/debian-security/ bookworm-security main contrib non-free non-free-firmware

  deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware
  deb-src http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware

#+end_src

******* Backports
If you need the Backports, contrib, and non-free components, add bookworm-backports lines. For example, for Debian 12/Bookworm:

#+begin_src sh

  # Append the following line to the bottom of the file:
  deb http://deb.debian.org/debian bookworm-backports main contrib non-free non-free-firmware
  deb-src http://deb.debian.org/debian bookworm-backports main contrib non-free non-free-firmware

  # Now that you have added the repository, update APT's cache to include
  # the backports in the list of available packages:
  apt update

#+end_src

******* Debian Unstable
If you need the Debian Unstable, then change the distribution to unstable. Note that Updates and Security is not used since Unstable is a rolling development version consisting of the latest packages. That is, unstable main will be the only entry in sources.list.

#+begin_src sh

  deb http://deb.debian.org/debian/ unstable main
  deb-src http://deb.debian.org/debian/ unstable main

#+end_src

******* Mirror
Brazil is the closest mirror, this example is for debian unstable.

#+begin_src sh

  deb http://ftp.br.debian.org/debian/ unstable main
  deb-src http://ftp.br.debian.org/debian/ unstable main

  # or
  deb ftp.br.debian.org/debian/ unstable main
  deb-src ftp.br.debian.org/debian/ unstable main

#+end_src

***** Debian Mirrors
Debian is distributed (aka mirrored) on hundreds of servers worldwide, all offering the same content. This way we can provide better access to our archive.

You can find the following content on our mirror servers:

- Debian Packages (debian/) :: the Debian package pool: this includes the vast majority of .deb packages, installation material, and the sources.
- CD Images (debian-cd/) :: the repository of CD images: Jigdo files and ISO image files.
  See the list of Debian mirrors that include the debian-cd/ archive.
- Old Releases (debian-archive/) :: the archive of older Debian versions, released in the past.

*** Compressing
Data compression is the process of removing *redundancy* from data.

Compression algorithms fall into two general categories.

- Lossless :: Lossless compression preserves all the data contained in the original.  This means that when a file is restored from a compressed version, the restored file is exactly the same as the original, uncompressed version.
- Lossy :: Lossy compression, on the other hand, removes data as the compression is performed to allow more compression to be applied. When a lossy file is restored, it does not match the original version; rather, it is a close approximation. Exam- ples of lossy compression are JPEG (for images) and MP3 (for music).

A common file-management task often used in conjunction with compression is *archiving*. Archiving is the process of gathering up many files and bundling them together into a single large file.

*** Cron job management
Any jobs under the purview of the system administrator should be in [[ /etc]], since they are configuration files. If you have a root cron job for daily, weekly, or monthly runs, put them in [[ /etc/cron.daily][/etc/cron.{daily,weekly,monthly}]]. These are invoked from [[ /etc/crontab]], and will run in alphabetic order, which serializes them.

On the other hand, if you have a cron job that (a) needs to run as a special user, or (b) needs to run at a special time or frequency, you can use either [[ /etc/crontab]], or, better yet, [[ /etc/cron.d][/etc/cron.d/whatever]]. These particular files also have an extra field that allows you to stipulate the user account under which the cron job runs.

In either case, you just edit the files and cron will notice them automatically. There is no need to run a special command.

*** Apt
**** Backports
***** Brief
Debian Backports provides new packages with new features on supported Debian stable releases.

*Note* (debian the beginners handbook): … what do you mean exactly by “backports” ? Nothing to do, in fact, with the “backdoors” used to spy on computers running proprietary systems… The backport is a mechanism allowing an application currently hold in the Debian development repositories, to be *ported back* to the “stable” version.

As a matter of Backports policy, packages in the stable-backports suite are taken from Debian testing; however, in rare cases such as security updates, packages are taken from unstable. Backports Policy permits this exclusively when such packages will be part of the next Debian stable release. In any case, the new packages are recompiled for use on Debian stable. Backports are expected to make use of the versions of libraries available in Debian stable whenever possible; however, when required, the backporting of additional dependencies is permitted.

Backports cannot be tested as extensively as Debian stable, and are thus supported on a best-effort basis; there is a risk of incompatibilities with other components in Debian stable, so backports should be used with care! That said, Debian Backports Policy does not allow backports of libraries that would break all dependent packages in stable (eg: new Qt 5.x releases), and by virtue of this, Debian Backports are considered generally safe when used as intended on an individual package basis.

The coinstallability of all available backports is not tested, and it is strongly recommended to opt-into the use of specific backported packages on an as-needed basis.

***** Add Backports to sources.list

1. Add a new file ~/etc/apt/sources.list.d/debian-backports.sources~:

#+begin_src sh

  Types: deb deb-src
  URIs: http://deb.debian.org/debian
  Suites: trixie-backports
  Components: main
  Enabled: yes
  Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

#+end_src

2. Run  ~apt update~ or ~apt-get update~

***** Installing a Package from Backports
All backports are deactivated by default so that the normal operation of a stable installation will not be compromised with potentially disruptive changes (such as incompatible configuration schema). Release files make this possible, and all backported packages are pinned to priority 100 via ~ButAutomaticUpgrades: yes~. To install something from backports run one of:

#+begin_src sh

  apt install <package>/trixie-backports
  apt-get install <package>/trixie-backports

#+end_src

or, if you also want missing dependencies to be installed from trixie-backports too

#+begin_src sh

  apt install -t trixie-backports <package>
  apt-get install -t trixie-backports <package>

#+end_src

*** Setting up your system to use e-mail
**** Brief
There are three main functions that make up an e-mail system. First there is the Mail User Agent (MUA) which is the program a user actually uses to compose and read mails. Then there is the Mail Transfer Agent (MTA) that takes care of transferring messages from one computer to another. And last there is the Mail Delivery Agent (MDA) that takes care of delivering incoming mail to the user's inbox.

These three functions can be performed by separate programs, but they can also be combined in one or two programs. It is also possible to have different programs handle these functions for different types of mail.

On Linux and Unix systems ~mutt~ is historically a very popular MUA. Like most traditional Linux programs it is text based. It is often used in combination with ~exim~ or ~sendmail~ as MTA and ~procmail~ as MDA.

With the increasing popularity of graphical desktop systems, the use of graphical e-mail programs like GNOME's ~evolution~, KDE's ~kmail~ or Mozilla's ~thunderbird~ has becoming more popular. These programs combine the function of a MUA, MTA and MDA, but can — and often are — also be used in combination with the traditional Linux tools.

**** Default e-mail configuration
Even if you are planning to use a graphical mail program, it would be useful, to have a traditional MTA/MDA installed and correctly set up on your Debian GNU/Linux system. Reason is that various utilities running on the system can send important notices by e-mail to inform the system administrator of (potential) problems or changes.

For this you can install ~exim4~ and ~mutt~ with ~apt install exim4 mutt~. ~exim4~ is a combination MTA/MDA that is relatively small but very flexible. By default it will be configured to only handle e-mail local to the system itself and e-mails addressed to the system administrator (root account) will be delivered to the regular user account created during the installation.

When system e-mails are delivered they are added to a file in [[ /var/mail/$username]]. The e-mails can be read using ~mutt~.

**** Sending e-mails outside the system
As mentioned earlier, the installed Debian system is only set up to handle e-mail local to the system, not for sending mail to others nor for receiving mail from others.

If you intend to use a graphical mail program and use a mail server of your Internet Service Provider (ISP) or your company, there is not really any need to configure ~exim4~ for handling external e-mail. Just configure your favorite graphical mail program to use the correct servers to send and receive e-mail (how is outside the scope of this manual).

However, in that case you may need to configure individual utilities to correctly send e-mails. One such utility is ~reportbug~, a program that facilitates submitting bug reports against Debian packages. By default it expects to be able to use ~exim4~ to submit bug reports.

To correctly set up ~reportbug~ to use an external mail server, please run the command ~reportbug --configure~ and answer “no” to the question if an MTA is available. You will then be asked for the SMTP server to be used for submitting bug reports.

**** Configuring the exim4 mail transport agent
If you would like your system to also handle external e-mail, you will need to reconfigure the ~exim4~ package or another alternative.

#+begin_src sh

  sudo dpkg-reconfigure exim4-config

#+end_src

After entering that command (as root), you will be asked if you want split the configuration into small files. If you are unsure, select the default option.

Next you will be presented with several common mail scenarios. Choose the one that most closely resembles your needs.

- internet site :: your system is connected to a network and your mail is sent and received directly using SMTP. On the following screens you will be asked a few basic questions, like your machine's mail name, or a list of domains for which you accept or relay mail.

- mail sent by smarthost :: in this scenario your outgoing mail is forwarded to another machine, called a “smarthost”, which takes care of sending the message on to its destination. The smarthost also usually stores incoming mail addressed to your computer, so you don't need to be permanently online. That also means you have to download your mail from the smarthost via programs like fetchmail.

  In a lot of cases the smarthost will be your ISP's mail server, which makes this option very suitable for dial-up users. It can also be a company mail server, or even another system on your own network.

- mail sent by smarthost; no local mail :: this option is basically the same as the previous one except that the system will not be set up to handle mail for a local e-mail domain. Mail on the system itself (e.g. for the system administrator) will still be handled.

- local delivery only :: this is the option your system is configured for by default.

- no configuration at this time :: choose this if you are absolutely convinced you know what you are doing. This will leave you with an unconfigured mail system — until you configure it, you won't be able to send or receive any mail and you may miss some important messages from your system utilities.

If none of these scenarios suits your needs, or if you need a finer grained setup, you will need to edit configuration files under the ~/etc/exim4~ directory after the installation is complete. More information about exim4 may be found under ~/usr/share/doc/exim4~; the file README.Debian.gz has further details about configuring exim4 and explains where to find additional documentation.

Note that sending mail directly to the Internet when you don't have an official domain name, can result in your mail being rejected because of anti-spam measures on receiving servers. Using your ISP's mail server is preferred. If you still do want to send out mail directly, you may want to use a different e-mail address than is generated by default. If you use exim4 as your MTA, this is possible by adding an entry in ~/etc/email-addresses~.

**** Example of exim4 configuration on Debian
1. First install ~exim4~ and ~mutt~. ~exim4~ is a combination MTA/MDA that is relatively small but very flexible. By default it will be configured to only handle e-mail local to the system itself and e-mails addressed to the system administrator (root account) will be delivered to the regular user account created during the installation.

#+begin_src sh

  sudo apt install exim4 mutt

#+end_src

2. Configuring the exim4 mail transport agent
https://wiki.debian.org/Exim4Gmail or/and https://sites.google.com/view/chewkeanho/guides/linux-os/exim4/setup-gmail-send-only

Initial configuration of Exim4 is invoked with:

#+begin_src sh

  sudo dpkg-reconfigure exim4-config

#+end_src

- Choose =mail sent by smarthost; received via SMTP or fetchmail=
- Set to =localhost= for "System mail name:".
- Set to =127.0.0.1 ; ::1= for "IP-addresses to listen on for incoming SMTP connections" to refuse external connections.
- Leave as empty for "Other destinations for which mail is accepted:".
- Leave as empty for "Machines to relay mail for:".
- Set to =smtp.gmail.com::587= for "IP address or host name of the outgoing smarthost:".
- Choose =NO= for "Hide local mail name in outgoing mail?".
- Choose =NO= for "Keep number of DNS-queries minimal (Dial-on-Demand)?".
- Choose =mbox format in /var/mail/= for "Delivery method for local mail".
- Choose =YES= for "Split configuration into small files?".

Then you will be needing *.gmail.com for your ~passwd.client~ file

3. Getting app-specific password from google account
You can obtain an "App-Specific Password" from your Google Account for local Exim4 authentication. The guide is here: https://sites.google.com/view/chewkeanho/guides/google/get-user-app-specific-password or/and https://dev.to/florenceokoli/smtp-mailutils-how-to-send-your-mails-via-the-linux-terminal-1m7o

- To get app specific password, you need to head over to https://myaccount.google.com/security section when logged in.
- Scroll down the 2-step verification page and click on the App passwords.
- Type a name for your app and a password will be generated automatically.
- Make sure to copy (without spaces) and save this password as we will need it later.

*Note*: keep in mind that 2-step verification must be enabled.

4. The authentication information for the external SMTP service is stored in ~/etc/exim4/passwd.client~. Run

#+begin_src sh

  sudo editor /etc/exim4/passwd.client

#+end_src

and add the following line:

#+begin_src sh

  smtp.gmail.com:SMTPAccountName:y0uRpaSsw0RD
  # example -> smtp.gmail.com:carlos@gmail.com:y0uRpaSsw0RD

#+end_src

Here's how ~/etc/exim4/passwd.client~ works: the first field in this file is matched against the reverse DNS of the remote SMTP server (in this case Gmail). If this domain name (or IP, if the server IP doesn't have a reverse domain name configured) matches the first field of this file, Exim4 will try to authenticate for SMTP using this login / password.

5. If you restored ~/etc/exim4/passwd.client~ from the backup file, please make sure to restore its file ownership and permissions with

#+begin_src sh

  sudo chown root:Debian-exim /etc/exim4/passwd.client
  sudo chmod 640 /etc/exim4/passwd.client

#+end_src

6. Finally, run

#+begin_src sh

  sudo update-exim4.conf
  sudo invoke-rc.d exim4 restart
  sudo exim4 -qff

#+end_src

These 3 steps update the Exim4 configuration, reload it, and to force a delivery attempt for all messages including frozen ones.

7. After the configuration, please verify your configuration with

#+begin_src sh

  echo "Local mail" | mail -s "Root message" root
  echo "Outside mail" | mail -s "Carlos message" carlos@gmail.com
  sudo tail /var/log/exim4/mainlog

#+end_src

** Processes and Threads
*** Processes
Is simply a state (or image) in memory.

Running programs that the kernel manages.

*** Threads
In Linux, some processes are divided into pieces called *threads*. A thread is very similar to a process—it has an identifier (thread ID, or TID), and the kernel schedules and runs threads just like processes. However, unlike separate processes, which usually don’t share system resources such as memory and I/O connections with other processes, all threads inside a single process share their system resources and some memory.

Many processes have only one thread. A process with one thread is single-threaded, and a process with more than one thread is multithreaded. All processes start out single-threaded. This starting thread is usually called the main thread. The main thread may start new threads, making the process multithreaded, similar to the way a process can call ~fork()~ to start a new process.

The primary advantage of a multithreaded process is that when the process has a lot to do, threads can run simultaneously on multiple processors, potentially speeding up computation. Although you can also achieve simultaneous computation with multiple processes, threads start faster than processes, and it’s often easier or more efficient for threads to intercommunicate using their shared memory than it is for processes to communicate over a channel, such as a network connection or a pipe.

Some programs use threads to overcome problems managing multiple I/O resources. Traditionally, a process would sometimes use ~fork()~ to start a new subprocess in order to deal with a new input or output stream. Threads offer a similar mechanism without the overhead of starting a new process.

*** Background and Foreground Jobs
A process that connects to the terminal is called a *foreground* job. A job is said to be in the foreground because it can communicate with the user via the screen and the keyboard.

On the other hand, a process that disconnects from the terminal and cannot communicate with the user is called a *background* job. If the background job requires interaction with the user, it will stop and wait until establishing a connection to the terminal.

We can place the jobs that do not require interaction from the user as they run (like sorting a large file) in the background. This allows the user to access the terminal and continue to work, instead of waiting for a long job to finish.

#+begin_src

             +-----------------+
  $ cmd  --->|   Foreground    |--------------\
             +-----------------+               \
               |         ^      ^               \
         ctrl-z|       fg|       \               \
               |         |        \fg             \
               v         |         \               L
             +-----------------+    \           +----------+
             |     Stopped     |--kill---job#-->|  Killed  |
             +-----------------+    /           +----------+
               |           ^       /               ^
             bg|  stop job#|      /               /
               |           |     /               /
               v           |    /               /
             +-----------------+               /
  $ cmd & -->|   Background    |--------------/
             +-----------------+

#+end_src

*** Common Signals

- HUP Hangup (1) :: this is a vestige of the good old days when terminals were attached to remote computers with phone lines and modems. The signal is used to indicate to programs that the controlling terminal has “hung up.” The effect of this signal can be demonstrated by closing a terminal session. The foreground program running on the terminal will be sent the signal and will terminate. This signal is also used by many daemon programs to cause a reinitialization. This means that when a daemon is sent this signal, it will restart and reread its configuration file. The Apache web server is an example of a daemon that uses the ~HUP~ signal in this way.

- INT Interrupt (2) :: this performs the same function as a ~Ctrl-c~ sent from the terminal. It will usually terminate a program.

- KILL Kill (9) :: this signal is special. Whereas programs may choose to handle signals sent to them in different ways, including ignoring them all together, the ~KILL~ signal is never actually sent to the target program. Rather, the kernel immediately terminates the process. When a process is terminated in this manner, it is given no opportunity to “clean up” after itself or save its work. For this reason, the ~KILL~ signal should be used only as a last resort when other termination signals fail.

- TERM Terminate (15) :: this is the default signal sent by the kill command. If a program is still “alive” enough to receive signals, it will terminate.

- CONT Continue (18) :: this will restore a process after a ~STOP~ or ~TSTP~ signal. This signal is sent by the ~bg~ and ~fg~ commands.

- STOP Stop (19) :: this signal causes a process to pause without terminating. Like the ~KILL~ signal, it is not sent to the target process, and thus it cannot be ignored.

- TSTP Terminal stop  (20) :: this is the signal sent by the terminal when ~Ctrl-z~ is pressed. Unlike the ~STOP~ signal, the ~TSTP~ signal is received by the program, but the program may choose to ignore it.

- SEGV Segmentation violation (11) :: this signal is sent if a program makes illegal use of memory, that is, if it tried to write somewhere it was not allowed to write.

- WINCH Window change (28) :: this is the signal sent by the system when a window changes size. Some programs , such as ~top~ and ~less~ will respond to this signal by redrawing themselves to fit the new window dimensions.

- QUIT (3) :: quit.

*** Control Groups (cgroups)
The basic idea is that you place several processes into a cgroup, which allows you to manage the resources that they consume on a group-wide basis. For example, if you want to limit the amount of memory that a set of processes may cumulatively consume, a cgroup can do this.

After creating a cgroup, you can add processes to it, and then use a *controller* to change how those processes behave. For example, there is a ~cpu~ controller allowing you to limit the processor time, a ~memory~ controller, and so on.

There are two versions of cgroups, 1 and 2. Aside from a somewhat different feature set, the structural differences between the versions can be summed up as follows:

- cgroups v1 :: each type of controller (~cpu~, ~memory~, and so on) has its own set of cgroups. A process can belong to one cgroup per controller, meaning that a process can belong to multiple cgroups. For example, in v1, a process can belong to a cpu cgroup and a memory cgroup.

- cgroups v2 :: a process can belong to only one cgroup. You can set up different types of controllers for each cgroup.

To visualize the difference, consider three sets of processes, A, B, and C. We want to use the cpu and memory controllers on each of them.

#+begin_src

  The next figure shows the schematic for cgroups v1. We need six
  cgroups total, because each cgroup is limited to a single controller.

  +-----------------+   +--------------------+
  | CPU controllers |   | Memory controllers |
  |  +-----------+  |   |   +-----------+    |
  |  | cgroup A1 |  |   |   | cgroup A2 |    |
  |  +-----------+  |   |   +-----------+    |
  |  +-----------+  |   |   +-----------+    |
  |  | cgroup B1 |  |   |   | cgroup B2 |    |
  |  +-----------+  |   |   +-----------+    |
  |  +-----------+  |   |   +-----------+    |
  |  | cgroup C1 |  |   |   | cgroup C2 |    |
  |  +-----------+  |   |   +-----------+    |
  +-----------------+   +--------------------+

  The next figure shows how to do it in cgroups v2. We need only three
  cgroups, because we can set up multiple controllers per cgroup.

  +---------------------+ +---------------------+ +---------------------+
  |      cgroup A       | |      cgroup B       | |      cgroup C       |
  | +-----------------+ | | +-----------------+ | | +-----------------+ |
  | | CPU controller  | | | | CPU controller  | | | | CPU controller  | |
  | +-----------------+ | | +-----------------+ | | +-----------------+ |
  | +-----------------+ | | +-----------------+ | | +-----------------+ |
  | |Memory controller| | | |Memory controller| | | |Memory controller| |
  | +-----------------+ | | +-----------------+ | | +-----------------+ |
  +---------------------+ +---------------------+ +---------------------+

#+end_src

** I/O
*** Standard Input and Standard Output
Unix processes use I/O streams to read and write data. Processes read data from input streams and write data to output streams. The source of an input stream can be a file, a device, a terminal window, or even the output stream from another process.

Standard output is similar. The kernel gives each process a standard output stream where it can write its output. Standard input and output are often abbreviated as stdin and stdout.

There is a third standard I/O stream, called standard error (stderr); it’s an additional output stream for diagnostics and debugging.

*** Operator n>&m
Although lots of sh manual pages don't mention this, the shell reads arguments from left to right.

~cat food 2>&1 > file~
On the first command line, the shell sees ~2>&1~ first. That means "make the standard error /fd2/ go to the same place as the standard output /fd1/ is going." There's no effect because both /fd2/ and /fd1/ are already going to the terminal. Then ~> file~ redirects /fd1/ (stdout) to file. But /fd2/ (stderr) is still going to the terminal.

~cat food > file 2>&1~
On the second command line, the shell sees ~> file~ first and redirects stdout to file. Next ~2>&1~ sends /fd2/ (stderr) to the same place /fd1/ is going - that's to the file. And that's what you want.

~n>&m~
The Bourne shell operator ~n>&m~ rearranges the files and file descriptors. It says "make file descriptor ~n~ point to the same file as file descriptor ~m~."

You can use more than one of those ~n>&m~ operators. The shell reads them left-to-right before it executes the command.

To swap standard output and standard error - make stderr go down a pipe and stdout go to the screen. This is one place the other file descriptors, /3/ through /9/, come in handy. They normally aren't used. You can use one of them as a "holding place," to remember where another file descriptor "pointed." For example, one way to read the operator ~3>&2~ is "make /3/ point the same place as /2/." After you use ~3>&2~ to grab the location of /2/, you can make /2/ point somewhere else. Then, make /1/ point where /2/ used to (where /3/ points now).

#+begin_src shell

  command 3>&2 2>&1 1>&3 | ...

  var=`grep "Joe" file1 file2`
  # grep: file2: No such file or directory
  echo "$var"
  # file1: Joe Jones 423-4567

  var=`grep  "Joe" file1 file2 3>&2 2>&1 1>&3`
  # 3>&2 crea un FD3 que apunte a donde apunta el FD2 (en este caso a la
  # pantalla), es decir, el FD3 ahora apunta a la pantalla. Luego 2>&1
  # coloca el FD2 que apunte a el FD1 (en este caso al backquotes), es
  # decir, ahora el FD2 no apunta a donde estaba antes (la pantalla) sino
  # al backquotes. Luego 1>&3 coloca el FD1 que apunte a el FD3 (en este
  # caso la pantalla), es decir, el FD1 y el FD3 ambos apuntan a la
  # pantalla.
  # file1: Joe Jones 423-4567
  echo "$var"
  # grep: file2: No such file or directory

#+end_src

Open files are automatically closed when a process exits. But it's safer to close the files yourself as soon as you're done with them. That way, if you forget and use the same descriptor later for something else (for instance, use F.D. /3/ to redirect some other command, or a subprocess uses F.D. /3/ ), you won't run into conflicts. Use ~m<&-~ to close input file descriptor m and ~m>&-~ to close output file descriptor /m/. If you need to close standard input, use ~<&-~; ~>&-~ will close standard output.

*** Pipeline (unamed or anonymous)
Is a mechanism for inter-process communication using message passing. A pipeline is a set of processes chained together by their standard streams, so that the output text of each process (stdout) is passed directly as input (stdin) to the next one. The second process is started as the first process is still executing, and they are executed concurrently.

This pipeline is a anonymous pipes, where data written by one process is buffered by the operating system until it is read by the next process, and this uni-directional channel disappears when the processes are completed.

Each "~|~" tells the shell to connect the standard output of the command on the left to the standard input of the command on the right by an inter-process communication (IPC) mechanism called an (anonymous) pipe, implemented in the operating system. Pipes are unidirectional; data flows through the pipeline from left to right.

In ~bash~ (and other shells such as ~sh~), pipelines create *subshells*. These are copies of the shell and its environment that are used to execute the command in the pipeline.

Subshells in Unix-like systems create copies of the environment for the processes to use while they execute. When the processes finishes, the copy of the environment is destroyed. This means that *a subshell can never alter the environment of its parent process*.

Ejemplo de [[https://www.danielclemente.com/][Daniel Clemente]]:
Prueba un ~echo Hola | nano~ (no explicaré el | aquí), verás que no puedes escribir (lógico, porque la entrada la coge del ~echo Hola~, no de STDIN). Ahora prueba un ~cat | nano~, irá perfecto porque ~cat~ coge la entrada y la manda por la salida (pruébalo suelto, un cat). Pues, ¿por qué no pruebas a ponerlo todo como un sólo comando? Haz ~(echo Hola; cat) | nano~

PS: esto lo he visto en exploits que dan shell; si un ~echo $shellcode | ./programa~ crea una shell pero no te permite escribir, usa este truco.

*** Named pipe FIFO
Is an extension to the traditional pipe concept, and is one of the methods of inter-process communication (IPC). A traditional pipe is "[[Pipeline (unamed or anonymous)][unnamed]]" and lasts only as long as the process. A named pipe, however, can last as long as the system is up, beyond the life of the process. It can be deleted if no longer used. Usually a named pipe appears as a file, and generally processes attach to it for IPC.

Instead of a conventional, unnamed, shell pipeline, a named pipeline makes use of the filesystem. It is explicitly created using ~mkfifo()~ or ~mknod()~, and two separate processes can access the pipe by name — one process can open it as a reader, and the other as a writer.

*Examples*:
- Create a named pipe (also known as a FIFO), start one command writing to the named pipe in the background, then run the other command with the named pipe as input.

#+begin_src shell

  mkfifo /tmp/sort2.fifo
  sort file2 > /tmp/sort2.fifo &
  sort file1 | diff - /tmp/sort2.fifo
  rm /tmp/sort2.fifo

#+end_src

- One can create a pipe and set up gzip to compress things piped to it:

#+begin_src shell

  mkfifo my_pipe
  gzip -9 -c < my_pipe > out.gz &

  # In a separate process shell, independently, one could send the data to
  # be compressed:
  cat file > my_pipe

  #The named pipe can be deleted just like any file:
  rm my_pipe

#+end_src

- A named pipe can be used to transfer information from one application to another without the use of an intermediate temporary file. For example, you can pipe the output of gzip into a named pipe like so (here out.gz is from above example but it can be any gz):

#+begin_src shell

  mkfifo -m 0666 /tmp/namedPipe
  gzip -d < out.gz > /tmp/namedPipe

  #Then load the uncompressed data into a MySQL table[3] like so:
  LOAD DATA INFILE '/tmp/namedPipe' INTO TABLE tableName;

#+end_src

Without this named pipe one would need to write out the entire uncompressed version of file.gz before loading it into MySQL. Writing the temporary file is both time consuming and results in more I/O and less free space on the hard drive.

- Ejemplo de [[https://www.danielclemente.com/][Daniel Clemente]]:
Con Linux podemos crear fifos, y es muy sencillo: haz ~mkfifo fi~ y habrás creado uno (un ~ls -l~ lo muestra como tal). Ahora haz un ~cat fi~ (se quedará parado) y en otra terminal un ~echo Hola >fi~ . Verás que el ~cat~ que estaba esperando datos ya los ha recibido; el fifo ha hecho su trabajo.

Puedes hacer cosas muy raras con fifos: por ejemplo, imagínate que dices a ~cdrecord~ que te grabe la ISO ~mififo~ (se quedaría esperando a que le entre contenido al fifo) y después haces un ~wget servidorveloz.com/linux.iso -O mififo~ (esto especifica el fichero de destino). ¡Estarías grabando un CD al mismo tiempo que lo descargas! Usa la imaginación para descubrir otros utilidades...

*** Printing
**** A Brief History of Printing
Printing on Unix-like systems goes way back to the beginning of the operating system. In those days, printers and how they were used were much different from today.

Like computers, printers in the pre-PC era tended to be large, expensive, and centralized. The typical computer user of 1980 worked at a terminal connected to a computer some distance away. The printer was located near the computer and was under the watchful eyes of the computer’s operators.

When printers were expensive and centralized, as they often were in the early days of Unix, it was common practice for many users to share a printer. To identify print jobs belonging to a particular user, a banner page displaying the name of the user was often printed at the beginning of each print job. The computer support staff would then load up a cart containing the day’s print jobs and deliver them to the individual users.

*Character-Based Printers*
The printer technology of the 80s was very different from today in two respects. First, printers of that period were almost always impact printers. Impact printers use a mechanical mechanism that strikes a ribbon against the paper to form character impressions on the page. Two of the popular technologies of that time were daisy-wheel printing and dot-matrix printing.

The second, and more important characteristic of early printers was that printers used a fixed set of characters that were intrinsic to the device. For example, a daisy-wheel printer could only print the characters actually molded into the petals of the daisy wheel. This made the printers much like high-speed typewriters. As with most typewriters, they printed using monospaced (fixed width) fonts. This means that each character has the same width. Printing was done at fixed positions on the page, and the printable area of a page contained a fixed number of characters. Most printers printed ten characters per inch (CPI) horizontally and six lines per inch (LPI) vertically. Using this scheme, a US-letter sheet of paper is 85 characters wide and 66 lines high. Taking into account a small margin on each side, 80 characters was considered the maximum width of a print line. This explains why terminal displays (and our terminal emulators) are normally 80 characters wide. Using a monospaced font and and an 80 character wide terminal provides a What You See Is What You Get (WYSIWYG) view of printed output.

Data is sent to a typewriter-like printer in a simple stream of bytes containing the characters to be printed. For example, to print an “a”, the ASCII character code 97 is sent. In addition, the low-numbered ASCII control codes provided a means of moving the printer’s carriage and paper, using codes for carriage return, line feed, form feed, and so on. Using the control codes, it’s possible to achieve some limited font effects, such as boldface, by having the printer print a character, backspace, and print the character again to get a darker print impression on the page.

**** Graphical Printers
The development of GUIs led to major changes in printer technology. As computers moved to more picture-based displays, printing moved from character-based to graphical techniques. This was facilitated by the advent of the low-cost laser printer which, instead of printing fixed characters, could print tiny dots anywhere in the printable area of the page. This made printing proportional fonts (like those used by typesetters), and even photographs and high-quality diagrams, possible.

However, moving from a character-based scheme to a graphical scheme presented a formidable technical challenge. Here’s why: the number of bytes needed to fill a page using a character-based printer can be calculated this way (assuming 60 lines per page each containing 80 characters):

60 X 80 = 4,800 bytes

In comparison, a 300 dot per inch (DPI) laser printer (assuming an 8 by 10 inch print area per page) requires this many bytes:

(8 X 300) X (10 X 300) / 8 = 900,000 bytes

Many of the slow PC networks simply could not handle the nearly one megabyte of data required to print a full page on a laser printer, so it was clear that a clever invention was needed.

That invention turned out to be the page description language (PDL). A page description language is a programming language that describes the contents of a page. Basically it says, “Go to this position, draw the character ‘a’ in 10 point Helvetica, go to this posi-tion...” until everything on the page is described. The first major PDL was PostScript from Adobe Systems, which is still in wide use today. The PostScript language is a complete programming language tailored for typography and other kinds of graphics and imaging. It includes built-in support for 35 standard, high-quality fonts, plus the ability to accept additional font definitions at runtime. At first, support for PostScript was built into the printers themselves. This solved the data transmission problem. While the typical PostScript program was very verbose in comparison to the simple byte stream of character-based printers, it was much smaller than the number of bytes required to represent the entire printed page.

A PostScript printer accepted a PostScript program as input. The printer contained its own processor and memory (oftentimes making the printer a more powerful computer than the computer to which it was attached) and executed a special program called a PostScript interpreter, which read the incoming PostScript program and rendered the results into the printer’s internal memory, thus forming the pattern of bits (dots) that would be transferred to the paper. The generic name for this process of rendering something into a large bit pattern (called a bitmap) is raster image processor (RIP).

As the years went by, both computers and networks became much faster. This allowed the RIP to move from the printer to the host computer, which, in turn, permitted high-quality printers to be much less expensive.

Many printers today still accept character-based streams, but many low-cost printers do not. They rely on the host computer’s RIP to provide a stream of bits to print as dots. There are still some PostScript printers, too.

**** Printing with Linux
Modern Linux systems employ two software suites to perform and manage printing. The first, Common Unix Printing System (CUPS) provides print drivers and print-job management, and the second, Ghostscript, a PostScript interpreter, acts as a RIP.

CUPS manages printers by creating and maintaining print queues.

** Shell
*** Definition
Is a program that runs commands, like the ones that users enter into a terminal window. These commands can be other programs or built-in (like ~exec()~) features of the shell. The shell also serves as a small programming environment.

*** Shell Globbing (“Wildcards”)
The simple glob character ~*~, tells the shell to match any number of arbitrary characters.

The shell matches arguments containing globs to filenames, substitutes the filenames for those arguments, and then runs the revised command line. The substitution is called expansion because the shell substitutes all matching filenames for a simplified expression.

Another shell glob character, the question mark ~?~, instructs the shell to match exactly one arbitrary character.

If you don’t want the shell to expand a glob in a command, enclose the glob in single quotes (''). For example, the command echo ~'*'~ prints a star.

*Note*:
It is important to remember that the shell performs expansions before running commands, and only then. Therefore, if a ~*~ makes it to a command without expanding, the shell won’t do anything more with it; it’s up to the command to decide what it wants to do.

*** Environment and Shell Variables
**** Shell variable
The shell can store temporary variables, called *shell variables*, containing the values of text strings. Shell variables are very useful for keeping track of values in scripts, and some shell variables control the way the shell behaves.

To assign a value to a shell variable, use the equal sign (~=~) and to access this variable, use ~$~.

**** Environment  variable
Is like a shell variable, but it’s not specific to the shell. All processes on Unix systems have environment variable storage. The main difference between environment and shell variables is that the operating system passes all of your shell’s environment variables to programs that the shell runs, whereas shell variables cannot be accessed in the commands that you run.

You assign an environment variable with the shell’s ~export~ command.

*** Interactive shells
Interactive shells are those you use to run commands from a terminal, and they can be classified as *login* or *non-login*.

*Note*: noninteractive shells (such as those that run shell scripts) usually don’t read any startup files.

- Login Shells ::
Traditionally, a login shell is what you get when you first log in to a system with the terminal using a program such as ~/bin/login~. Logging in remotely with SSH also gives you a login shell. The basic idea is that the login shell *is an initial shell*. You can tell if a shell is a login shell by running echo ~$0~; if the first character is a ~-~, the shell’s a login shell.

When bash runs as a login shell, it runs ~/etc/profile~. Then it looks for a user’s ~.bash_profile~, ~.bash_login~, and ~.profile files~, running only the first one that it sees.

As strange as it sounds, it’s possible to run a noninteractive shell as a login shell to force it to run startup files. To do so, start the shell with the ~-l~ or ~--login~ option.

- Non-Login Shells ::
A non-login shell is an additional shell that you *run after you log in*. It’s simply any interactive shell that’s not a login shell. Windowing system terminal programs (xterm, GNOME Terminal, and so on) start non-login shells unless you specifically ask for a login shell.

Upon starting up as a non-login shell, bash runs ~/etc/bash.bashrc~ and then runs the user’s ~.bashrc~.

*** Example .bashrc

#+begin_src shell

  # COMMAND PATH
  PATH=/usr/local/bin:/usr/bin:/bin:/usr/games
  PATH=$HOME/bin:$PATH # at the front so that executables there
  # take precedence over the system versions.
  # If you need the system executables, add /sbin and /usr/sbin .
  # PS1 is the regular prompt.
  # Substitutions include:
  # \u username \h hostname \w current directory
  # \! history number \s shell name \$ $ if regular user
  # PROMPT
  PS1='\u\$ '
  # EDITOR and VISUAL determine the editor that programs such as less
  # and mail clients invoke when asked to edit a file.
  EDITOR=vi
  VISUAL=vi

  # PAGER is the default text file viewer for programs such as
  # man (that shows text one page at a time).
  PAGER=less
  # These are some handy options for less.
  # A different style is LESS=FRX
  # (F=quit at end, R=show raw characters, X=don't use alt screen)
  LESS=meiX
  # You must export environment variables.
  export PATH EDITOR VISUAL PAGER LESS
  # By default, give other users read-only access to most new files.
  # MASK
  umask 022

  # You can share this .bashrc file with .bash_profile via a symbolic
  # link, or you can make the relationship even clearer by creating
  # .bash_profile as this one-liner:
  # . $HOME/.bashrc

#+end_src

*** getty and login
The ~getty~ program attaches to terminals and displays a login prompt. On most Linux systems, ~getty~ is uncomplicated because the system uses it only for logins on virtual terminals. In a process listing, it usually looks something like this (for example, when running on ~/dev/tty1~):

#+begin_src sh

  $ ps ao args | grep getty
  #/sbin/agetty -o -p -- \u --noclear tty1 linux

#+end_src

On many systems, you may not even see a ~getty~ process until you access a virtual terminal with something like ~CTRL-ALT-F1~. This example shows ~agetty~, the version that many Linux distributions include by default.

After you enter your login name, ~getty~ replaces itself with the ~login~ program, which asks for your password. If you enter the correct password, login replaces itself (using ~exec()~) with your shell. Otherwise, you get a “Login incorrect” message. Much of the login program’s real authentication work is handled by PAM.

You’ll rarely even use ~getty~ and ~login~, because most users now log in either through a graphical interface such as ~gdm~ or remotely with ~SSH~, neither of which uses ~getty~ or ~login~.

** Command
A command can be one of four different things:

- an executable program :: like ~/usr/bin~.
- a command built into the shell itself ::  ~bash~ supports a number of commands internally called shell builtins. The ~cd~ command, for example, is a shell builtin.
- a shell function :: shell functions are miniature shell scripts incorporated into the environment.
- an alias  :: aliases are commands that we can define ourselves, built from other commands.

*Note*:
When square brackets appear in the description of a command's syntax, they indicate optional items. A vertical bar character indicates mutually exclusive items. In the case of the ~cd~ command above:

~cd [-L|[-P[-e]]] [dir]~

This notation says that the command ~cd~ may be followed optionally by either a ~-L~  or a ~-P~ and further, if the ~-P~ option is specified the ~-e~ option may also be included followed by the optional argument ~dir~.

** File Descriptor
Is a number that a process uses together with the system libraries and kernel to identify and manipulate a file.

** System Logging
Most system programs write their diagnostic output as messages to the *syslog* service. The traditional syslogd daemon performs this service by waiting for messages and, upon receiving one, sending it to an appropriate channel, such as a file or a database. On most contemporary systems, *journald* (which comes with systemd) does most of the work.

The system logger is one of the most important parts of the system. When something goes wrong and you don’t know where to start, it’s always wise to check the log. If you have journald, you’ll do this with the ~journalctl~ command.

A log message typically contains important information such as the process name, process ID, and timestamp. There can also be two other fields: the facility (a general category) and severity (how urgent the message is).

** Graphical display mechanism
*** Framebuffers
At the bottom of any graphical display mechanism is the framebuffer, a chunk of memory that the graphics hardware reads and transmits to the screen for display. A few individual bytes in the framebuffer represent each pixel of the display, so the idea is that if you want to change the way something looks, you need to write new values to the framebuffer memory.

*** The X Window System
The approach that the X Window System takes is to have a server (called the X server) that acts as a sort of “kernel” of the desktop to manage everything from rendering windows to configuring displays to handling input from devices, such as keyboards and mice. The X server doesn’t dictate the way anything should act or appear. Instead, X client programs handle the user interface. Basic X client applications, such as terminal windows and web browsers, make connections to the X server and ask to draw windows. In response, the X server figures out where to place the windows and where to render client graphics, and it takes a certain amount of responsibility for rendering graphics to the framebuffer. The X server also channels input to a client when appropriate.

*** Wayland
The name Wayland refers to a communications protocol between a compositing window manager and graphical client program. Unlike [[The X Window System][X]], Wayland is significantly decentralized by design. There’s no large display server managing the [[Framebuffers][framebuffer]] for a number of graphical clients, and there’s no centralized authority for rendering graphics. Instead, each client gets its own memory buffer (think of this as sort of a sub-framebuffer) for its own window, and a piece of software called a compositor combines all of the clients’ buffers into the necessary form for copying to the screen’s framebuffer.

One of the more unusual aspects of Wayland systems is the mechanism for drawing window decorations, such as title bars. In X, the window manager did it all, but in the initial implementations of Wayland, this was left up to the client applications, which sometimes led to windows having many different kinds of decorations on the same screen. Now there’s a part of the protocol called XDG-Decoration that allows the client to negotiate with the window manager to see if the window manager is willing to draw decorations.

*** Window Managers
A major difference between [[The X Window System][X]] and [[Wayland]] systems is in the window manager, the piece of software that determines how to arrange windows on the screen and is central to the user experience.

In X, the window manager is a client that acts as a helper to the server; it draws the windows’ decorations (such as title bars and close buttons), handles input events to those decorations, and tells the server where to move windows.

However, in Wayland, the window manager is the server, more or less. It is responsible for compositing all of the client window buffers into the display [[Framebuffers][framebuffer]], and it handles the channeling of input device events. As a result, it is required to do more work than a window manager in X, but much of that code can be common between window manager implementations.

*** Toolkits
Desktop applications include certain common elements, such as buttons and menus, called *widgets*. To speed up development and provide a common look, programmers use graphical *toolkits* to provide those elements. On operating systems like Windows or macOS, the vendor provides a common toolkit, and most programmers use that. On Linux, the GTK+ toolkit is one of the most common, but you’ll also frequently see widgets built on the Qt framework and others.

Toolkits usually consist of shared libraries and support files, such as images and theme information.

*** Desktop Environments
Although [[Toolkits][toolkits]] provide the user with a uniform outward appearance, some details of a desktop require a degree of cooperation between different applications. For example, one application may wish to share data with another or update a common notification bar on a desktop. To provide for those needs, toolkits and other libraries are bundled into larger packages called desktop environments. GNOME, KDE, and Xfce are some common Linux desktop environments.

Toolkits are at the core of most desktop environments, but to create a unified desktop, environments must also include numerous support files, such as icons and configurations, that make up themes. All of this is bound together with documents that describe design conventions, such as how application menus and titles should appear and how applications should react to certain system events.

*** D-Bus
Is an interprocess communication service used in many parts of the system. D-Bus is important because it serves as an interprocess communication mechanism that allows desktop applications to talk to each other, and because most Linux systems use it to notify processes of system events, such as inserting a USB drive.

D-Bus itself consists of a library that standardizes interprocess communication with a protocol and supporting functions for any two processes to talk to each other. By itself, this library doesn’t offer much more than a fancy version of normal IPC facilities, such as Unix domain sockets. What makes D-Bus useful is a central “hub” called ~dbus-daemon~. Processes that need to react to events can connect to ~dbus-daemon~ and register to receive certain kinds of events.

Connecting processes also create the events. For example, the process ~udisks-daemon~ monitors udev for disk events and sends them to ~dbus-daemon~, which then retransmits the events to applications interested in disk events.

The applications such as web browsers and the terminal window normally stand alone, but they often use interprocess communication to become aware of pertinent events. For example, an application can express interest when you attach a new storage device or when you receive new email or an instant message. This communication usually occurs over D-Bus.

** Virtualization
*** Hypervisors
Overseeing one or more virtual machines on a computer is a piece of software called a *hypervisor* or virtual machine monitor (VMM), which works similarly to how an operating system manages processes. There are two types of hypervisors, and the way you use a virtual machine depends on the type.

- the type 2 hypervisor :: is the most familiar, because it runs on a normal operating system such as Linux. For example, VirtualBox is a type 2 hypervisor, and you can run it on your system without extensive modifications.

- type 1 hypervisor :: is more like its own operating system (especially the kernel), built specifically to run virtual machines quickly and efficiently. This kind of hypervisor might occasionally employ a conventional companion system such as Linux to help with management tasks. Creating an instance of an operating system on a cloud service such as AWS is creating a virtual machine on a type 1 hypervisor.

In general, a virtual machine with its operating system is called a *guest*. The *host* is whatever runs the hypervisor. For type 2 hypervisors, the host is just your native system. For type 1 hypervisors, the host is the hypervisor itself, possibly combined with a specialized companion system.

*** Containers
Virtual machines are great for insulating an entire operating system and its set of running applications, but sometimes you need a lighter-weight alternative.  Container technology is now a popular way to fulfill this need.

A *container* can be loosely defined as a restricted runtime environment for a set of processes, the implication being that those processes can’t touch anything on the system outside that environment. In general, this is called *operating system–level virtualization*.

It’s important to keep in mind that a machine running one or more containers still has only one underlying Linux kernel. However, the processes inside a container can use the user-space environment from a Linux distribution different than the underlying system.

The restrictions in containers are built with a number of kernel features. Some of the important aspects of processes running in a container are:
- They have their own cgroups.
- They have their own devices and filesystem.
- They cannot see or interact with any other processes on the system.
- They have their own network interfaces.

Pulling all of those things together is a complicated task. It’s possible to alter everything manually, but it can be challenging; just getting a handle on the cgroups for a process is tricky. To help you along, many tools can perform the necessary subtasks of creating and managing effective containers. Two of the most popular are [[Docker]] and LXC.

*** Docker
First you need to create an image, which comprises the filesystem and a few other defining features for a container to run with.

*Note*: It’s easy to confuse images and containers. You can think of an image as the container’s filesystem; processes don’t run in an image, but they do run in containers. This is not quite accurate (in particular, when you change the files in a Docker container, you aren’t making changes to the image), but it’s close enough for now.

Install Docker on your system (your distribution’s add-on package is probably fine), make a new directory somewhere, change to that directory, and create a file called ~Dockerfile~ containing these lines:

#+begin_src sh

  FROM alpine:latest
  RUN apk add bash
  CMD ["/bin/bash"]

#+end_src

This configuration uses the lightweight Alpine distribution. The only change we’re making is adding the bash shell, which we’re doing not just for an added measure of interactive usability but also to create a unique image and see how that procedure works. It’s possible (and common) to use public images and make no changes to them whatsoever. In that case, you don’t need a ~Dockerfile~.

*Note*: Anything done with the ~RUN~ command in a Dockerfile happens during the image build, not afterward, when you start a container with the image. The ~CMD~ command is for the container runtime; this is why it occurs at the end.

Build the image with the following command, which reads the ~Dockerfile~ in the current directory and applies the identifier ~hlw_test~ to the image:

#+begin_src sh

  docker build -t hlw_test .

  # Sending build context to Docker daemon 2.048kB
  # Step 1/3 : FROM alpine:latest
  # latest: Pulling from library/alpine
  # cbdbe7a5bc2a: Pull complete
  # Digest:
  # sha256:9a839e63dad54c3a6d1834e29692c8492d93f90c59c978c1ed79109ea4b9a54
  # Status: Downloaded newer image for alpine:latest
  # ---> f70734b6a266
  In this step, Docker has created a new image with the identifier
  f70734b6a266 for the basic Alpine distribution image (it’s not the
  final image). An image that isn’t intended to be a final product is
  called an intermediate image.

  # Step 2/3 : RUN apk add bash
  # ---> Running in 4f0fb4632b31
  # fetch http://dl-
  # cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
  # fetch http://dl-
  # cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
  # (1/4) Installing ncurses-terminfo-base (6.1_p20200118-r4)
  # (2/4) Installing ncurses-libs (6.1_p20200118-r4)
  # (3/4) Installing readline (8.0.1-r0)
  # (4/4) Installing bash (5.0.11-r1)
  # Executing bash-5.0.11-r1.post-install
  # Executing busybox-1.31.1-r9.trigger
  # OK: 8 MiB in 18 packages
  # Removing intermediate container 4f0fb4632b31
  # ---> 12ef4043c80a
  This part of our configuration is the bash shell package installation
  in Alpine. After setting up the (temporary) container with ID
  4f0fb4632b31 , Docker ran the apk command inside that container to
  install bash, and then saved the resulting changes to the filesystem
  into a new intermediate image with the ID 12ef4043c80a .

  #Step 3/3 : CMD ["/bin/bash"]
  #---> Running in fb082e6a0728
  #Removing intermediate container fb082e6a0728
  #---> 1b64f94e5a54
  #Successfully built 1b64f94e5a54
  #Successfully tagged hlw_test:latest
  Finally, Docker makes the final changes required to run a bash shell
  when starting a container from the new image.

#+end_src

In this example, you now have a final image with the ID ~1b64f94e5a54~, but because you tagged it (in two separate steps), you can also refer to it as ~hlw_test~ or ~hlw_test:latest~. Run ~docker images~ to verify that your image and the Alpine image are present:

#+begin_src sh

  docker images
  REPOSITORY	TAG	IMAGE ID	CREATED		SIZE
  hlw_test	latest	1b64f94e5a54	1 minute ago	9.19MB
  alpine	latest	f70734b6a266	3 weeks ago	5.61MB

#+end_src

*Running Docker Containers*
Let’s jump right into it and start container with the image that you just built:

#+begin_src sh

  docker run -it hlw_test
  # You should get a bash shell prompt where you can run commands in
  # thecontainer. That shell will run as the root user. (-it options
  # (interactive, connect a terminal))

  # If you’re the curious type, you’ll probably want to take a look around
  # the container. Run some commands, such as mount and ps , and explore
  # the filesystem in general.
  ps aux
  #PID	USER	TIME	COMMAND
  #	1 root	0:00 /bin/bash
  #	6 root	0:00 ps aux

#+end_src

** Shutting Down System
1. ~init~ asks every process to shut down cleanly.
2. If a process doesn’t respond after a while, init kills it, first trying a ~TERM~ signal.
3. If the ~TERM~ signal doesn’t work, init uses the ~KILL~ signal on any stragglers.
4. The system locks system files into place and makes other preparations for shutdown.
5. The system unmounts all filesystems other than the root.
6. The system remounts the root filesystem read-only.
7. The system writes all buffered data out to the filesystem with the sync program.
8. The final step is to tell the kernel to reboot or stop with the ~reboot(2)~ system call. This can be done by init or an auxiliary program, such as ~reboot~, ~halt~, or ~poweroff~.

* Kernel
** Definition
Is the core of the operating system. The kernel is software residing in memory that tells the CPU where to look for its next task. Acting as a mediator, the kernel manages the hardware (especially main memory) and is the primary interface between the hardware and any running program.

There is a critical difference between how the kernel and the user processes run: the kernel runs in kernel mode, and the user processes run in user mode. Code running in kernel mode has unrestricted access to the processor and main memory. The memory area that only the kernel can access is called kernel space.

The Linux kernel can run kernel threads, which look much like processes but have access to kernel space.

One of the kernel’s tasks is to split memory into many subdivisions, and it must maintain certain state information about those subdivisions at all times. Each process gets its own share of memory, and the kernel must ensure that each process keeps to its share.

** Boots
*** Boot process
1. The machine’s BIOS or boot firmware loads and runs a boot loader.
2. The boot loader finds the kernel image on disk, loads it into memory, and starts it.
3. The kernel initializes the devices and its drivers.
4. The kernel mounts the root filesystem.
5. The kernel starts a program called ~init~ with a process ID of ~1~. This point is the user space start.
6. ~init~ sets the rest of the system processes in motion.
7. At some point, ~init~ starts a process allowing you to log in, usually at the end or near the end of the boot sequence.

*** GRUB
GRUB stands for Grand Unified Boot Loader. One of GRUB’s most important capabilities is filesystem navigation that allows for easy kernel image and configuration selection.

*** How GRUB Works
1. The PC BIOS or firmware initializes the hardware and searches its boot-order storage devices for boot code.
2. Upon finding the boot code, the BIOS/firmware loads and executes it. This is where GRUB begins.
3. The GRUB core loads.
4. The core initializes. At this point, GRUB can now access disks and filesystems.
5. GRUB identifies its boot partition and loads a configuration there.
6. GRUB gives the user a chance to change the configuration.
7. After a timeout or user action, GRUB executes the configuration.
8. In the course of executing the configuration, GRUB may load additional code (modules) in the boot partition. Some of these modules may be preloaded.
9. GRUB executes a boot command to load and execute the kernel as specified by the configuration’s linux command.

*** Boot Loader Tasks

- Select from multiple kernels.
- Switch between sets of kernel parameters.
- Allow the user to manually override and edit kernel image names and parameters (for example, to enter single-user mode).
- Provide support for booting other operating systems.

** Managing tasks

- Processes :: The kernel is responsible for determining which processes are allowed to use the CPU.

- Memory :: The kernel needs to keep track of all memory—what is currently allocated to a particular process, what might be shared between processes, and what is free.

- Device drivers :: The kernel acts as an interface between hardware and processes. It’s usually the kernel’s job to operate the hardware.

- System calls and support Processes :: normally use system calls to communicate with the kernel.

** Process Management
Process management describes the starting, pausing, resuming, scheduling, and terminating of processes.

In practice, each process uses the CPU for a small fraction of a second, then pauses; then another process uses the CPU for another small fraction of a second; then another process takes a turn, and so on. The act of one process giving up control of the CPU to another process is called a [[Context switching][context switch]].

The context switch answers the important question of when the kernel runs. The answer is that it runs between process time slices during a context switch.

** Context switching
To understand how the context switch works, let’s think about a situation in which a process is running in user mode but its time slice is up.

1. The CPU interrupts the current process based on an internal timer, switches into kernel mode, and hands control back to the kernel.
2. The kernel records the current state of the CPU and memory, which will be essential to resuming the process that was just interrupted.
3. The kernel performs any tasks that might have come up during the preceding time slice (such as collecting data from input and output, or I/O, operations).
4. The kernel is now ready to let another process run. The kernel analyzes the list of processes that are ready to run and chooses one.
5. The kernel prepares the memory for this new process and then prepares the CPU.
6. The kernel tells the CPU how long the time slice for the new process will last.
7. The kernel switches the CPU into user mode and hands control of the CPU to the process.

** Memory Management
The kernel must manage memory during a context switch, which can be a complex job. The following conditions must hold:

- The kernel must have its own private area in memory that user processes can’t access.
- Each user process needs its own section of memory.
- One user process may not access the private memory of another process. User processes can share memory.
- Some memory in user processes can be read-only.

When the process accesses some of its memory, the memory management unit MMU intercepts the access and uses a memory address map to translate the memory location from the process point of view into an actual physical memory location in the machine. The kernel must still initialize and continuously maintain and alter this memory address map. The implementation of a memory address map is called a *page table*.

** System Calls and Support
Perform specific tasks, an interaction between a process and the kernel.

Two system calls, ~fork()~ and ~exec()~, are important to understanding how processes start:
- fork() :: when a process calls ~fork()~, the kernel creates a nearly identical copy of the process.
- exec() :: when a process calls ~exec(program)~, the kernel loads and starts ~program~, replacing the current process.

Other than [[init]], all new user processes on a Linux system start as a result of ~fork()~, and most of the time, you also run ~exec()~ to start a new program instead of running a copy of an existing process.

When you enter ~ls~ into a terminal window, the shell that’s running inside the terminal window calls ~fork()~ to create a copy of the shell, and then the new copy of the shell calls ~exec(ls)~ to run ~ls~.

#+begin_src

  +-------+    +--------+    +-------+
  | shell |--->| fork() |--->| shell |
  +-------+    +--------+ |  +-------+
                          |
                          |  +------------+    +----------+    +----+
                          -->| copy shell |--->| exec(ls) |--->| ls |
                             +------------+    +----------+    +----+

#+end_src

The kernel also supports user processes with features other than traditional system calls, the most common of which are *pseudodevices*. Pseudodevices look like devices to user processes, but they’re implemented purely in software. This means they don’t technically need to be in the kernel, but they are usually there for practical reasons. For example, the kernel random number generator device ~/dev/random~ would be difficult to implement securely with a user process.

Technically, a user process that accesses a pseudodevice must use a system call to open the device, so processes can’t entirely avoid system calls.

** How Memory Works
The CPU has a memory management unit (MMU) to add flexibility in memory access. The kernel assists the MMU by breaking down the memory used by processes into smaller chunks called pages. The kernel maintains a data structure, called a page table, that maps a process’s virtual page addresses to real page addresses in memory. As a process accesses memory, the MMU translates the virtual addresses used by the process into real addresses based on the kernel’s page table.

A user process doesn’t actually need all of its memory pages to be immediately available in order to run. The kernel generally loads and allocates pages as a process needs them; this system is known as *on-demand paging or just demand paging*. To see how this works, consider how a program starts and runs as a new process:

1. The kernel loads the beginning of the program’s instruction code into memory pages.
2. The kernel may allocate some working-memory pages to the new process.
3. As the process runs, it might reach a point where the next instruction in its code isn’t in any of the pages that the kernel initially loaded. At this point, the kernel takes over, loads the necessary page into memory, and then lets the program resume execution.
4. Similarly, if the program requires more working memory than was initially allocated, the kernel handles it by finding free pages (or by making room) and assigning them to the process.

* systemd
** Daemon
Is a computer program that runs as a background process, rather than being under the direct control of an interactive user.

In a Unix environment, the parent process of a daemon is often, but not always, the init process. A daemon is usually created either by a process forking a child process and then immediately exiting, thus causing init to adopt the child process, or by the init process directly launching the daemon.

Systems often start daemons at boot time that will respond to network requests, hardware activity, or other programs by performing some task. Daemons such as cron may also perform defined tasks at scheduled times.

** Introduction
Is a software suite that provides an array of system components for Linux operating systems.

The systemd init is one of the newest init implementations on Linux. In addition to handling the regular boot process, systemd aims to incorporate the functionality of a number of standard Unix services, such as cron and inetd.

Where systemd really stands out from its predecessors is its advanced service management capabilities. Unlike a traditional init, systemd can track individual service daemons after they start, and group together multiple processes associated with a service, giving you more power and insight into exactly what is running on the system.

systemd is goal-oriented. At the top level, you can think of defining a goal, called a *unit*, for some system task. A unit can contain instructions for common startup tasks, such as starting a daemon, and it also has dependencies, which are other units. When starting (or activating) a unit, systemd attempts to activate its dependencies and then moves on to the details of the unit.

When starting services, systemd does not follow a rigid sequence; instead, it activates units whenever they are ready. After boot, systemd can react to system events (such as the uevents outlined) by activating additional units.

Requests to activate, reactivate, and restart units are called *jobs* in systemd, and they are essentially unit state changes. These jobs also have nothing to do with the shell’s job control.

Following its integrated approach, systemd also provides replacements for various daemons and utilities, including the startup shell scripts, ~pm-utils~, ~inetd~, ~acpid~, ~syslog~, ~watchdog~, ~cron~ and ~atd~. systemd's core components include the following:

- systemd :: is a system and service manager for Linux operating systems.
- systemctl :: is a command to introspect and control the state of the systemd system and service manager. Not to be confused with sysctl.
- systemd-analyze :: may be used to determine system boot-up performance statistics and retrieve other state and tracing information from the system and service manager.

On the system shown below, ~default.target~ groups the units necessary to start a GUI.

#+begin_src

                    +----------------+
                    | default.target |
                    +----------------+
                            |
                            v
                   +-------------------+
                   | multi-user.target |
                   +-------------------+
                 /          |             \
                v           v              v
  +--------------+   +--------------+   +--------------+
  | basic.target |   | cron.service |   | dbus.service |
  +--------------+   +--------------+   +--------------+
         |
         v
  +----------------+
  | sysinit.target |
  +----------------+

#+end_src

** Units and Unit Types
One way that systemd is more ambitious than previous versions of init is that it doesn’t just operate processes and services; it can also manage filesystem mounts, monitor network connection requests, run timers, and more. Each capability is called a *unit type*, and each specific function (such as a service) is called a *unit*. When you turn on a unit, you activate it. Each unit has its own configuration file.

These are the most significant *unit types* that perform the boot-time tasks on a typical Linux system:
- service units :: control the service daemons found on a Unix system.
- target units :: control other units, usually by grouping them.
- socket units :: represent incoming network connection request locations.
- mount units :: represent the attachment of filesystems to the system.

*Ordering*:
To activate units in a particular order, use the following dependency modifiers:

- Before :: the current unit will activate before the listed unit(s). For example, if ~Before=bar.target~ appears in ~foo.target~, systemd activates ~foo.target~ before ~bar.target~.
- After :: the current unit activates after the listed unit(s).

When you use ordering, systemd waits until a unit has an active status before activating its dependent units.

** Dependencies
To accommodate the need for flexibility and fault tolerance, systemd offers several dependency types and styles.

- Requires :: strict dependencies. When activating a unit with a ~Requires~ dependency unit, systemd attempts to activate the dependency unit. If the dependency unit fails, systemd also deactivates the dependent unit.
- Wants :: dependencies for activation only. Upon activating a unit, systemd activates the unit’s ~Wants~ dependencies, but it doesn’t care if those dependencies fail.
- Requisite :: units that must already be active. Before activating a unit with a ~Requisite~ dependency, systemd first checks the status of the dependency. If the dependency hasn’t been activated, systemd fails on activation of the unit with the dependency.
- Conflicts :: negative dependencies. When activating a unit with a ~Conflict~ dependency, systemd automatically deactivates the opposing dependency if it’s active. Simultaneous activation of conflicting units fails.

The ~Wants~ dependency type is especially significant because it doesn’t propagate failures to other units. The ~systemd.service(5)~ manual page states that this is how you should specify dependencies if possible, and it’s easy to see why. This behavior produces a much more robust system, giving you the benefit of a traditional init, where the failure of an earlier startup component doesn’t necessarily prohibit later components from starting.

** Service types
There are several different start-up types to consider when writing a custom service file. This is set with the ~Type=~ parameter in the ~[Service]~ section:

- Type=simple (default) :: systemd considers the service to be started up immediately. The process must not fork. Do not use this type if other services need to be ordered on this service, unless it is socket activated.
- Type=forking :: systemd considers the service started up once the process forks and the parent has exited. For classic daemons, use this type unless you know that it is not necessary. You should specify ~PIDFile=~ as well so systemd can keep track of the main process.
- Type=oneshot :: this is useful for scripts that do a single job and then exit. You may want to set ~RemainAfterExit=yes~ as well so that systemd still considers the service as active after the process has exited. Setting ~RemainAfterExit=yes~ is appropriate for the units which change the system state (e.g., mount some partition).
- Type=notify :: identical to ~Type=simple~, but with the stipulation that the daemon will send a signal to systemd when it is ready. The reference implementation for this notification is provided by ~libsystemd-daemon.so~.
- Type=dbus :: the service is considered ready when the specified BusName appears on DBus's system bus.
- Type=idle :: systemd will

*Note*: If a service is of type ~forking~, the ~ExecStart=~ command is expected to be a short-lived init process that forks of a main process. The service is considered to be active as soon as the ~ExecStart=~ command terminates successfully, and it remains in this state until the forked off process terminates.

 Unless ~Type=forking~ is set, the process started via this command line will be considered the main process of the daemon.

** Unit Files
The format for unit files is a section names in square brackets ~[]~ and variable and value assignments (options) in each section.

There are two sections, ~[Unit]~ and ~[Service]~. The ~[Unit]~ section gives some details about the unit and contains description and dependency information. In the most service unit, you’ll find the details about the service in the ~[Service]~ section, including how to prepare, start, and reload the service.

#+begin_src conf

  # bus-daemon.service unit file for the desktop bus daemon.
  [Unit]
  Description=D-Bus System Message Bus
  Documentation=man:dbus-daemon(1)
  Requires=dbus.socket # requires the dbus.socket unit as a dependency
  RefuseManualStart=yes

  [Service]
  ExecStart=/usr/bin/dbus-daemon --system --address=systemd: --
  nofork --nopidfile --systemd-activation --syslog-only
  ExecReload=/usr/bin/dbus-send --print-reply --system --
  type=method_call --dest= org.freedesktop.DBus /
  org.freedesktop.DBus.ReloadConfig

#+end_src

** Adding Units to systemd
Adding units to systemd is primarily a matter of creating, then activating and possibly enabling, unit files. You should normally put your own unit files in the system configuration directory [[ /etc/systemd/system]] so that you won’t confuse them with anything that came with your distribution and so that the distribution won’t overwrite them when you upgrade.

As an example of create two targets, one with a dependency on the other, follow these steps:

1. Create a unit file named ~test1.target~ in ~/etc/systemd/system~:

#+begin_src conf

  [Unit]
  Description=test 1

#+end_src

2. Create a ~test2.target~ file with a dependency on ~test1.target~:

#+begin_src conf

  [Unit]
  Description=test 2
  Wants=test1.target # Wants keyword defines a dependency

#+end_src

3. Activate the ~test2.target~ unit to see it in action and verify that both units are active:

#+begin_src sh

  systemctl start test2.target
  systemctl status test1.target test2.target

#+end_src

4. If your unit file has an ~[Install]~ section, you need to “enable” the unit before activating it (the [Install] section is another way to create a dependency).

#+begin_src sh

  systemctl enable unit

#+end_src

*Removing Units from systemd*:
1. Deactivate the unit if necessary:

#+begin_src sh

  systemctl stop test1.target

#+end_src

2. If the unit has an ~[Install]~ section, disable the unit to remove any symbolic links created by the dependency system. Disabling a unit that is implicitly enabled (that is, does not have an [Install] section) has no effect.

#+begin_src sh

  systemctl disable test1.target

#+end_src

You can then remove the unit file if you like.

** The [Install] Section
*** Brief
This section is optional and is used to define the behavior or a unit if it is enabled or disabled. Enabling a unit marks it to be automatically started at boot. In essence, this is accomplished by latching the unit in question onto another unit that is somewhere in the line of units to be started at boot.

Because of this, only units that can be enabled will have this section. The directives within dictate what should happen when the unit is enabled:

- WantedBy= :: the ~WantedBy=~ directive is the most common way to specify how a unit should be enabled. This directive allows you to specify a dependency relationship in a similar way to the ~Wants=~ directive does in the ~[Unit]~ section. The difference is that this directive is included in the ancillary unit allowing the primary unit listed to remain relatively clean. When a unit with this directive is enabled, a directory will be created within [[ /etc/systemd/system]] named after the specified unit with .wants appended to the end. Within this, a symbolic link to the current unit will be created, creating the dependency. For instance, if the current unit has ~WantedBy=multi-user.target~, a directory called ~multi-user.target.wants~ will be created within ~/etc/systemd/system~ (if not already available) and a symbolic link to the current unit will be placed within. Disabling this unit removes the link and removes the dependency relationship.

- RequiredBy= :: this directive is very similar to the ~WantedBy=~ directive, but instead specifies a required dependency that will cause the activation to fail if not met. When enabled, a unit with this directive will create a directory ending with ~.requires~.

- Alias= :: this directive allows the unit to be enabled under another name as well. Among other uses, this allows multiple providers of a function to be available, so that related units can look for any provider of the common aliased name.

- Also= :: this directive allows units to be enabled or disabled as a set. Supporting units that should always be available when this unit is active can be listed here. They will be managed as a group for installation tasks.

- DefaultInstance= :: for template units (covered later) which can produce unit instances with unpredictable names, this can be used as a fallback value for the name if an appropriate name is not provided.

*** Example
Apart from defining dependencies in a dependent unit’s configuration file. It’s also possible to do this “in reverse”—that is, by specifying the dependent unit in a dependency’s unit file. You can do this by adding a ~WantedBy~ or ~RequiredBy~ parameter in the ~[Install]~ section. This mechanism allows you to alter when a unit should start without modifying additional configuration files (for example, when you’d rather not edit a system unit file).

To see how this works, consider the [[Adding Units to systemd][example]] units. We had two units, ~test1.target~ and ~test2.target~, with ~test2.target~ having a ~Wants~ dependency on ~test1.target~. We can change them so that ~test1.target~ looks like this:


#+begin_src conf

  [Unit] Description=test 1
  [Install] WantedBy=test2.target

#+end_src

And ~test2.target~ is as follows:

#+begin_src conf

  [Unit] Description=test 2

#+end_src

Because you now have a unit with an ~[Install]~ section, you need to enable the unit with ~systemctl~ before you can start it. Here’s how that works with ~test1.target~:

#+begin_src sh

  systemctl enable test1.target
  # Created symlink
  # /etc/systemd/system/test2.target.wants/test1.target →
  # /etc/systemd/system/test1.target.

#+end_src

Notice the output here—the effect of enabling a unit is to create a symbolic link in a ~.wants~ subdirectory corresponding to the dependent unit (~test2.target~ in this case). You can now start both units at the same time with ~systemctl start test2.target~ because the dependency is in place.

*Note*: Enabling a unit does not activate it.

To disable the unit (and remove the symbolic link), use ~systemctl~ as follows:

#+begin_src sh

  systemctl disable test1.target
  # Removed /etc/systemd/system/test2.target.wants/test1.target.

#+end_src

The ~[Install]~ section is usually responsible for the ~.wants~ and ~.requires~ directories in the system configuration directory ~/etc/systemd/system~. However, the unit configuration directory ~[/usr]/lib/systemd/system~ also contains ~.wants~ directories, and you may also add links that don’t correspond to ~[Install]~ sections in the unit files. These manual additions are a simple way to add a dependency without modifying a unit file that may be overwritten in the future (by a software upgrade, for instance), but they’re not particularly encouraged because a manual addition is difficult to trace.

*Note*: ~WantedBy~ can also be used to activate units in advance, for example, when installing a server like ~python -m http.server~ you would put ~WantedBy=multi-user.target~ so when you start ~multi-user.target~ at boot time, it will activate the server via symlink (IDK).

#+begin_src conf

  # python3-server.service
  [Unit]
  Description=Local File Server
  After=network.target

  [Service]
  ExecStart=/usr/bin/python3 -m http.server 9000 -d /home/rich/Downloads/
  Restart=always
  RestartSec=5

  [Install]
  WantedBy=multi-user.target

  # /etc/systemd/system/multi-user.target.wants/python3-server.service ->
  # /etc/systemd/system/python3-server.service

#+end_src

** The [Service] Section
- Type=forking :: systemd considers the service started up once the process forks and the parent has exited. For classic daemons, use this type unless you know that it is not necessary. You should specify ~PIDFile=~ as well so systemd can keep track of the main process.

If a service is of type ~forking~, the ~ExecStart=~ command is expected to be a short-lived init process that forks of a main process. The service is considered to be active as soon as the ~ExecStart=~ command terminates successfully, and it remains in this state until the forked off process terminates.

 Unless ~Type=forking~ is set, the process started via this command line will be considered the main process of the daemon.

*Example*: this is a service that enables a keylogger and leaves it running while the computer is on. In this case, you'll need to use ~Type=forking~ otherwise the service would be activated and immediately deactivated automatically.

#+begin_src conf

  [Unit]
  Description=Keylogger

  [Service]
  Type=forking
  ExecStart=sudo /home/rich/.local/bin/logkeys-start.sh
  User=root
  Group=root

  [Install]
  WantedBy=multi-user.target

#+end_src

** Socket Unit and Service
A simple network echo service. The idea of an echo service is to repeat anything that a network client sends after connecting; ours will listen on TCP port 22222. We’ll start building it with a *socket unit* to represent the port, as shown in the following ~echo.socket~ unit file:

#+begin_src conf

  [Unit]
  Description=echo socket

  [Socket]
  ListenStream=22222
  Accept=true

#+end_src

Note that there’s no mention of the service unit that this socket supports inside the unit file. So, what is that corresponding service unit file?

Its name is ~echo@.service~. The link is established by naming convention; if a service unit file has the same prefix as a ~.socket~ file (in this case, *echo*), systemd knows to activate that service unit when there’s activity on the socket unit. In this case, systemd creates an instance of ~echo@.service~ when there’s activity on ~echo.socket~. Here’s the ~echo@.service~ unit file:

#+begin_src conf

  [Unit]
  Description=echo service

  [Service]
  ExecStart=/bin/cat
  StandardInput=socket

#+end_src

*Note*: If you don’t like the implicit activation of units based on the prefixes, or you need to link units with different prefixes, you can use an explicit option in the unit defining your resource. For example, use ~Socket=bar.socket~ inside ~foo.service~ to have ~bar.socket~ hand its socket to ~foo.service~.

To get this example unit running, you need to start the ~echo.socket~ unit:

#+begin_src sh

  systemctl start echo.socket

#+end_src

Now you can test the service by connecting to your local TCP port 22222 with a utility such as ~telnet~. The service repeats what you enter; here’s an example interaction:

#+begin_src sh

  $ telnet localhost 22222
  # Trying 127.0.0.1...
  # Connected to localhost.
  # Escape character is '^]'.
  Hi there.
  # Hi there.

#+end_src

To stop the service, stop the socket unit like so:

#+begin_src sh

  systemctl stop echo.socket

#+end_src

Because the ~echo@.service~ unit supports multiple simultaneous instances, there’s an ~@~ in the name (the ~@~ specifier signifies parameterization). Why would you want multiple instances? Say you have more than one network client connecting to the service at the same time, and you want each connection to have its own instance. In this case, the service unit must support multiple instances because we included the ~Accept=true~ option in ~echo.socket~. That option instructs systemd not only to listen on the port, but also to accept incoming connections on behalf of the service unit and pass it to them, creating a separate instance for each connection. Each instance reads data from the connection as standard input, but it doesn’t necessarily need to know that the data is coming from a network connection.

*Note*: Most network connections require more flexibility than just a simple gateway to standard input and output, so don’t expect to be able to create complex network services with a service unit file like the ~echo@.service~ unit file shown here.

If a service unit can do the work of accepting a connection, don’t put an ~@~ in its unit filename, and don’t put ~Accept=true~ in the socket unit. In this case, the service unit takes complete control of the socket from systemd, which in turn does not attempt to listen on the network port again until the service unit finishes.

** Timer Units
An alternative to creating a cron job for a periodic task is to build a systemd timer unit. For an entirely new task, you must create two units: a timer unit and a service unit. The reason for two units is that a timer unit doesn’t contain any specifics about the task to perform; it’s just an activation mechanism to run a service unit.

Let’s look at a typical timer/service unit pair, starting with the timer unit. Let’s call this ~loggertest.timer~; as with other custom unit files, we’ll put it in [[ /etc/systemd/system]].

#+begin_src conf

  [Unit]
  Description=Example timer unit

  [Timer]
  OnCalendar=*-*-* *:00,20,40
  Unit=loggertest.service

  [Install]
  WantedBy=timers.target

#+end_src

This timer runs every 20 minutes, with the ~OnCalendar~ option resembling the cron syntax. In this example, it’s at the top of each hour, as well as 20 and 40 minutes past each hour.

The ~OnCalendar~ time format is ~year-month-day hour:minute:second~. The field for seconds is optional. As with cron, a ~*~ represents a sort of wildcard, and commas allow for multiple values. The periodic ~/~ syntax is also valid; in the preceding example, you could change the ~*:00,20,40 to *:00/20~ (every 20 minutes) for the same effect.

The associated service unit is named ~loggertest.service~. We explicitly named it in the timer with the ~Unit~ option, but this isn’t strictly necessary because systemd looks for a ~.service~ file with the same base name as the timer unit file. This service unit also goes in [[ /etc/systemd/system]].

#+begin_src conf

  [Unit]
  Description=Example Test Service

  [Service]
  Type=oneshot
  ExecStart=/usr/bin/logger -p local3.debug I\'m a logger

#+end_src

The meat of this is the ~ExecStart~ line, which is the command that the service runs when activated. This particular example sends a message to the system log.

Note the use of ~oneshot~ as the service type, indicating that the service is expected to run and exit, and that systemd won’t consider the service started until the command specified by ~ExecStart~ completes. This has a few advantages for timers:

- You can specify multiple ~ExecStart~ commands in the unit file.
- It’s easier to control strict dependency order when activating other units using ~Wants~ and ~Before~ dependency directives.
- You have better records of start and end times of the unit in the journal.

* Device
** Disk schematic

#+begin_src

  +-----------------------------------------------------------------+
  |  +-----------------------------------------------------------+  |
  |  | Partition Table                                           |  |
  |  +-----------------------------------------------------------+  |
  |                          |                            |         |
  |                          v                            v         |
  |  +--------------------------------------------+  +-----------+  |
  |  |  Partition                                 |  | Partition |  |
  |  |  +--------------------------------------+  |  |           |  |
  |  |  |      +----------------------------+  |  |  |           |  |
  |  |  |      | Filesystem Data Structures |  |  |  |           |  |
  |  |  |      +----------------------------+  |  |  |           |  |
  |  |  |                    |                 |  |  |           |  |
  |  |  |                    v                 |  |  |           |  |
  |  |  |  +--------------------------------+  |  |  |           |  |
  |  |  |  | File Data                      |  |  |  |           |  |
  |  |  |  |                                |  |  |  |           |  |
  |  |  |  |                                |  |  |  |           |  |
  |  |  |  |                                |  |  |  |           |  |
  |  |  |  +--------------------------------+  |  |  |           |  |
  |  |  +--------------------------------------+  |  |           |  |
  |  +--------------------------------------------+  +-----------+  |
  +-----------------------------------------------------------------+

#+end_src

** Device Drivers
A device is typically accessible only in kernel mode because improper access could crash the machine. A notable difficulty is that different devices rarely have the same programming interface, even if the devices perform the same task.

** Device Files

- block device :: programs access data from a block device in fixed chunks. Disks can be easily split up into blocks of data. Because a block device’s total size is fixed and easy to index, processes have quick random access to any block in the device with the help of the kernel.

- character device :: character devices work with data streams. You can only read characters from or write characters to character devices. Character devices don’t have a size; when you read from or write to one, the kernel usually performs a read or write operation on it. Printers directly attached to your computer are represented by character devices.

- pipe device :: named pipes are like character devices, with another process at the other end of the I/O stream instead of a kernel driver.

- socket device :: are special-purpose interfaces that are frequently used for interprocess communication.

** SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, electrical, optical and logical interfaces.

** MBR Basics
The MBR table in example below contains primary, extended, and logical partitions. A primary partition is a normal subdivision of the disk; partition 1 is an example. The basic MBR has a limit of four primary partitions, so if you want more than four, you must designate one as an extended partition. An extended partition breaks down into logical partitions, which the operating system can then use as it would any other partition. In this example, partition 2 is an extended partition that contains logical partition 5.

#+begin_src sh

  parted -l

  Model: ATA KINGSTON SM2280S (scsi)
  Disk /dev/sda: 240GB
  Sector size (logical/physical): 512B/512B Partition Table: msdos
  Disk Flags:

  Number	Start	End	Size	Type	File system	Flags
  1	1049kB	223GB	223GB	primary	ext4	boot
  2	223GB	240GB	17.0GB	extended
  5	223GB	240GB	17.0GB	logical	linux-swap(v1)

  Model: Generic Flash Disk (scsi)
  Disk /dev/sdf: 4284MB
  Sector size (logical/physical): 512B/512B Partition Table: gpt
  Disk Flags:

  Number	Start	End	Size	File system	Name	Flags
  1	1049kB	1050MB	1049MB		myfirst
  2	1050MB	4284MB	3235MB		mysecond

#+end_src

** Swap Space
*** Definition
If you run out of real memory, the Linux virtual memory system can automatically move pieces of memory to and from disk storage. This is called *swapping* because pieces of idle programs are swapped to the disk in exchange for active pieces residing on the disk. The disk area used to store memory pages is called *swap space* (or just *swap*).

*** Using a Disk Partition as Swap Space
To use an entire disk partition as swap, follow these steps:

1. Make sure the partition is empty.
2. Run ~mkswap dev~, where ~dev~ is the partition’s device. This command puts a *swap signature* on the partition, marking it as swap space (rather than a filesystem or otherwise).
3. Execute ~swapon dev~ to register the space with the kernel.

After creating a swap partition, you can put a new swap entry in your ~/etc/fstab~ file to make the system use the swap space as soon as the machine boots. Here’s a sample entry that uses ~/dev/sda5~ as a swap partition:
~/dev/sda5 none swap sw 0 0~

*** Using a File as Swap Space
You can use a regular file as swap space if you’re in a situation where you would be forced to repartition a disk in order to create a swap partition. You shouldn’t notice any problems when doing this.

Use these commands to create an empty file, initialize it as swap, and add it to the swap pool:

#+begin_src sh

  dd if=/dev/zero of=swap_file bs=1024k count=num_mb
  mkswap swap_file
  swapon swap_file

#+end_src

Here, ~swap_file~ is the name of the new swap file, and ~num_mb~ is the desired size in megabytes.

To remove a swap partition or file from the kernel’s active pool, use the ~swapoff~ command. Your system must have enough free remaining memory (real and swap combined) to accommodate any active pages in the part of the swap pool that you’re removing.

*Example*:

#+begin_src sh

  ## Check current swap status.
  sudo swapon -s
  ## Create the swap file (1024k*7168=7340032->7GB)
  sudo dd if=/dev/zero of=/swapfile bs=1024k count=7168
  ## Set Swap Permissions.
  sudo chmod 600 /swapfile
  ## Set up the file as Linux swap area.
  sudo mkswap /swapfile
  ## Enable the swap.
  sudo swapon /swapfile
  ## Make the swap file permanent adding the following
  ## line at the end of the /etc/fstab file.
  /swapfile none swap sw 0 0

  ## To verify that the swap is active.
  # sudo swapon --show

  ## Remove Swap Space
  # swapoff /swapfile
  ## Remove the swap file entry /swapfile none swap sw 0 0
  ## from the /etc/fstab file.
  ## Remove the swap file
  # sudo rm /swapfile

#+end_src

** Major and Minor Numbers
Char devices are accessed through names in the filesystem. Those names are called special files or device files or simply nodes of the filesystem tree; they are conventionally located in the [[ /dev]] directory. Special files for char drivers are identified by a ~c~ in the first column of the output of ~ls -l~. Block devices appear in [[ /dev]] as well, but they are identified by a ~b~.

Each device node’s type (block or character) and numbers (known as the major and minor number) serve as identifiers for the kernel. You can view the current devices in the [[ /proc/devices]]  file.

  - major number :: *identifies the driver associated with the device*. For example, ~/dev/null~ and ~/dev/zero~ are both managed by driver ~1~, whereas virtual consoles and serial terminals are managed by driver ~4~. The kernel uses the major number at open time to dispatch execution to the appropriate driver.
  - minor number :: is used only by the driver specified by the major number; other parts of the kernel don’t use it, and merely pass it along to the driver. It is common for a driver to control several devices; the minor number *provides a way for the driver to differentiate among them* (typically refers to an instance).

#+begin_src sh

  ls -l /dev/vd*
  # brw-rw---- 1 root disk 253,  0 Feb  3 09:09 /dev/vda
  # brw-rw---- 1 root disk 253,  1 Feb  3 09:09 /dev/vda1
  # brw-rw---- 1 root disk 253,  2 Feb  3 09:09 /dev/vda2
  # brw-rw---- 1 root disk 253,  3 Feb  3 09:09 /dev/vda3
  # brw-rw---- 1 root disk 253, 16 Feb  3 09:09 /dev/vdb
  # brw-rw---- 1 root disk 253, 32 Feb  3 09:09 /dev/vdc
  # brw-rw---- 1 root disk 253, 33 Feb  3 09:09 /dev/vdc1

  # In the first line 253 is the major number virblk identifier and the
  # minor number is 0 which differentiates it from the others.

#+end_src

** Creating a Partition Table
1. You’ll start at the command prompt with the device name (first unmount it, if needed).
   ~sudo fdisk /dev/sdd~

2.  =m= will display the program menu.

3. First, print the current table with the =p= option.

#+begin_src sh

  Disk /dev/sdd: 4 GiB, 4284481536 bytes, 8368128 sectors
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos
  Disk identifier: 0x88f290cc

  Device Boot Start End Sectors Size Id Type
  /dev/sdd1 2048 8368127 8366080 4G c W95 FAT32 (LBA)

#+end_src

You can delete the existing ones pressing =d=.

Remember that ~fdisk~ doesn’t make changes until you explicitly write the partition table, so you haven’t yet modified the disk. If you make a mistake you can’t recover from, use the =q= option to quit ~fdisk~ without writing the changes.

4. Now you’ll create the first 200MB partition with the =n= option (the default values are quite often what you want).

5. When you’re finished laying out the partitions, use the =p= (print) option to review.

#+begin_src sh

  Device        Boot    Start     End  Sectors  Size Id Type
  /dev/sdd1     2048	 411647  409600           200M 83 Linux
  /dev/sdd2	411648	8368127 7956480           3.8G 83 Linux

#+end_src

6. When you’re ready to write the partition table, use the =w= option.

Additionally,

7. You can create a filesystem ~ext4~ partition on ~/dev/sdd1~ with this command.
   ~sudo  mkfs -t ext4 /dev/sdd1~

8. To mount the Fourth Extended filesystem found on the device ~/dev/sdd1~ on ~/home/extra~, use this command.
   ~sudo mount -t ext4 /dev/sdd1 /home/extra~

* File system
** Definition
Is a form of database; it supplies the structure to transform a simple block device into the sophisticated hierarchy of files and subdirectories that users can understand.

** The Initial RAM Filesystem
The Linux boot process is, for the most part, fairly straightforward. However, one component has always been somewhat confounding: *initramfs*, or the *initial RAM filesystem*. Think of it as a little user-space wedge that goes in front of the normal user mode start.

The Linux kernel does not talk to the PC BIOS interface or EFI to get data from disks, so in order to mount its root filesystem, it needs driver support for the underlying storage mechanism. Unfortunately, there are so many storage controller drivers that distributions can’t include all of them in their kernels, so many drivers are shipped as loadable modules. But loadable modules are files, and if your kernel doesn’t have a filesystem mounted in the first place, it can’t load the driver modules that it needs.

The workaround is to gather a small collection of kernel driver modules along with a few other utilities into an archive. The boot loader loads this archive into memory before running the kernel. Upon start, the kernel reads the contents of the archive into a temporary RAM filesystem (the initramfs), mounts it at ~/~, and performs the user-mode handoff to the init on the initramfs. Then, the utilities included in the initramfs allow the kernel to load the necessary driver modules for the real root filesystem. Finally, the utilities mount the real root filesystem and start the true init.

** Symbolic and hard links
*** Symbolic link
A *symbolic link* is a file that points to another file or a directory, effectively creating an alias (shortcut).

Your system may also have links that point to other links, which are called chained symbolic links and can be a nuisance when you’re trying to track them down.

*** Hard link
~ln~ creates a *hard link*, giving an additional real filename to a single file. The new filename has the status of the old one; it points (links) directly to the file data instead of to another filename as a symbolic link does.

By default, every file has a single hard link that gives the file its name.  When we create a hard link, we create an additional directory entry for a file. Hard links have two important limitations:

1. A hard link cannot reference a file outside its own file system. This means a link cannot reference a file that is not on the same disk partition as the link itself.
2. A hard link may not reference a directory.

A hard link is indistinguishable from the file itself. Unlike a symbolic link, when we list a directory containing a hard link we will see no special indication of the link. When a hard link is deleted, the link is removed but the contents of the file itself continue to exist (that is, its space is not deallocated) until all links to the file are deleted.

** Special-Purpose Filesystems
Not all filesystems represent storage on physical media. Most versions of Unix have filesystems that serve as system interfaces. That is, rather than serving only as a means to store data on a device, a filesystem can represent system information, such as process IDs and kernel diagnostics. This idea goes back to the [[ /dev]] mechanism, which is an early model of using files for I/O interfaces.

Some of the special filesystem types in common use on Linux include:
- proc :: mounted on [[ /proc]].
- sysfs :: mounted on [[ /sys]].
- tmpfs :: mounted on [[ /run]] and other locations. With ~tmpfs~, you can use your physical memory and swap space as temporary storage. You can mount ~tmpfs~ where you like, using the size and ~nr_blocks~ long options to control the maximum size. However, be careful not to pour things constantly into a ~tmpfs~ location, because your system will eventually run out of memory and programs will start to crash.
- squashfs :: a type of read-only filesystem where content is stored in a compressed format and extracted on demand through a loopback device. One example use is in the snap package management system that mounts packages under the ~/snap~ directory.
- overlay :: a filesystem that merges directories into a composite. Containers often use overlay filesystems.

** Inside a Traditional Filesystem
A traditional Unix filesystem has two primary components: a pool of data blocks where you can store data and a database system that manages the data pool. The database is centered around the inode data structure.

An *inode* is a set of data that describes a particular file, including its type, permissions, and—perhaps most important—where in the data pool the file data resides. Inodes are identified by numbers listed in an inode table.

Filenames and directories are also implemented as inodes. A directory inode contains a list of filenames and links corresponding to other inodes.

User-level representation of a filesystem:

#+begin_src

                           +------+
                           |(root)|
                           +------+
                         _____/\____
                  _____/            \_____
                 /                        \
             +----+                      +----+
             |dir1|                      |dir2|
             +----+                      +----+
              /|\                          /\
           /   |   \                      /  \
       /       |      \                  /    \
  +-----+   +-----+   +-----+      +-----+   +-----+
  |file1|   |file2|   |file3|      |file4|   |file5|
  +-----+   +-----+   +-----+      +-----+   +-----+

#+end_src

*Note*: For any ext2/3/4 filesystem, you start at inode number 2, which is the root inode (try not to confuse this with the system root filesystem).

The link count field is the number of total directory entries (across all directories) that point to an inode.

There is one small exception in link counts. The root inode 2 has a link count of 4. However, Figure shows only three directory entry links. The “fourth” link is in the filesystem’s superblock because the superblock tells you where to find the root inode.

Inode structure of the filesystem shown in previous filesystem:

#+begin_src

      inode table
   #  link count  type                     data pool
  +------+---+------+                +-------------------+
  | 2    | 4 | dir  |-------         | .      inode 2    |
  +------+---+------+       \------->| dir1   inode 12   |
  +------+---+------+                | dir2   inode 7633 |
  | 12   | 2 | dir  |---             +-------------------+
  +------+---+------+   \---         +-------------------+
  +------+---+------+       \---     | .      inode 12   |
  | 13   | 1 | file |--         \--->| ..     inode 2    |
  +------+---+------+  \             | file1  inode 13   |
  +------+---+------+    \           | file2  inode 14   |
  | 14   | 1 | file |-     \         | file3  inode 15   |
  +------+---+------+ \      \       +-------------------+
  +------+---+------+  \       \     +-------------------+
  | 15   | 2 | file |-  \        \-->| "a"               |
  +------+---+------+ \  \           +-------------------+
  +------+---+------+  \  \          +-------------------+
  | 16   | 1 | file |-  \  \         | .      inode 7633 |
  +------+---+------+ \  \  \        | ..     inode 2    |
  +------+---+------+  \  \  \       | file4  inode 16   |
  | 7633 | 2 | dir  |--------------->| file5  inode 15   |
  +------+---+------+    \  \  \     +-------------------+
                          \  \  \    +-------------------+
                           \  \  \-->| "b"               |
                            \  \     +-------------------+
                             \  \    +-------------------+
                              \  \-->| "c"               |
                               \     +-------------------+
                                \    +-------------------+
                                 \-->| "d"               |
                                     +-------------------+

#+end_src

*Note*: Portable Filenames
To ensure that a filename is portable between multiple platforms (i.e., different types of computers and operating systems), care must be taken to limit which characters are included in a filename. There is a standard called the POSIX Portable Filename Character Set that can be used to maximize the chances that a filename will work across different systems. The standard is pretty simple. The only characters allowed are the uppercase letters ~A-Z~, the lowercase letters ~a-z~, the numerals ~0-9~, period ( ~.~ ), hyphen ( ~-~ ), and underscore ( ~_~ ). The standard further suggests that filenames not begin with a hyphen.

** Permissions
*** File permissions

#+begin_src sh

  ls -l filename

  -rw-r--r-- 12 linuxize users 12.0K Apr  8 20:51 filename
  |[-][-][-]-   [------] [---]
  | |  |  | |      |       |
  | |  |  | |      |       +-----------> 7. Group
  | |  |  | |      +-------------------> 6. Owner
  | |  |  | +--------------------------> 5. Alternate Access Method
  | |  |  +----------------------------> 4. Others Permissions
  | |  +-------------------------------> 3. Group Permissions
  | +----------------------------------> 2. Owner Permissions
  +------------------------------------> 1. File Type

#+end_src

*** The Permissions Mask
A shell’s built-in ~umask~ (permissions mask) facility sets the default permissions. Include the ~umask~ command in one of your startup files to make certain that any program you run creates files with your desired permissions. There are two reasonable choices:

- ~077~ This mask is the most restrictive permissions mask; it doesn’t give any other users access to new files and directories. This is often appropriate on a multi-user system where you don’t want other users to look at any of your files. However, when set as the default, this mask can sometimes lead to problems when your users want to share files but don’t understand how to set permissions correctly.

- ~022~ This mask gives other users read access to new files and directories. This can be a good choice on a single-user system because many daemons that run as pseudo-users won’t be able to see files and directories created with the more restrictive ~077~ umask.

Everywhere a 1 appears in the binary value of the mask, an attribute is unset.

| Original file | --- rw- rw- rw- | 0666 |
| Mask          | 000 000 010 010 | 0022 |
|---------------+-----------------+------|
| Result        | --- rw- r-- r-- | 0644 |

*** The setuide bit
Meaning that when you execute the program, it runs as though the file owner is the user instead of you. Many programs use this setuid bit to run as root in order to get the privileges they need to change system files. One example is the passwd program, which needs to change the ~/etc/passwd~ file.

*** The setgid bit
Like the [[The setuide bit][setuid]] bit, changes the effective group ID from the real group ID of the real user to that of the file owner. If the setgid bit is set on a directory, newly created files in the directory will be given the group ownership of the directory rather the group ownership of the file's creator. This is useful in a shared directory when members of a common group need access to all the files in the directory, regardless of the file owner's primary group.

*** The sticky bit
This is a holdover from ancient Unix, where it was possible to mark an executable file as “not swappable.” On files, Linux ignores the sticky bit, but if applied to a directory, it prevents users from deleting or renaming files unless the user is either the owner of the directory, the owner of the file, or the superuser. This is often used to control access to a shared directory, such as [[ /tmp]].

** Tasks
*** Mounting and Unmounting Storage Devices

#+begin_src sh

  # Viewing a List of Mounted File Systems
  mount
  # /dev/sda2 on / type ext4 (rw)
  # device /dev/sda2 is mounted as the root file system, is of type ext4,
  # and is both readable and writable

  # Unmount the disc with the umount command
  umount /dev/sdc

  # Create a new mount point for the disk.
  # First create a new directory.
  mkdir /mnt/cdrom
  # Mount the CD-ROM at the new mount point.
  mount -t iso9660 /dev/sdc /mnt/cdrom


  # Mount the flash drive (other example).
  sudo mkdir /mnt/flash
  sudo mount /dev/sdb1 /mnt/flash

#+end_src

*** Reformat the flash with a Linux file system
To reformat the flash drive with a Linux native file system, rather than the FAT32 system for example. This involves two steps.

- (optional) Create a new partition layout if the existing one is not to our liking.
- Create a new, empty file system on the drive.

1. First unmount it (if needed) and then invoke the ~fdisk~ program:

#+begin_src sh

  sudo umount /dev/sdb1
  sudo fdisk /dev/sdb

#+end_src

2. Entering an =m= will display the program menu.

3. The first thing we want to do is examine the existing partition layout. We do this by entering =p= to print the partition table for the device.

#+begin_src sh

  Command (m for help): p

  Disk /dev/sdb: 16 MB, 16006656 bytes
  1 heads, 31 sectors/track, 1008 cylinders
  Units = cylinders of 31 * 512 = 15872 bytes

  Device Boot  Start   End  Blocks  Id  System
  /dev/sdb1           2  1008  15608+   b   W95 FAT32
  # We see a 16 MB device with a single partition (1) that uses 1,006 of
  # the available 1,008 cylinders on the device. The partition is
  # identified as a Windows 95 FAT32 partition.

#+end_src

We see that the ~ID b~ is used to specify the existing *partition*. To see a list of the available *partition types*, we refer to the program menu.

If we enter =l= at the prompt, a large list of possible types is displayed. Among them we see ~b~ for our existing partition type and ~83~ for Linux.

4. We enter =t= (change a *partition's system* id) at the prompt and enter the new ID.

#+begin_src sh

  Command (m for help): t
  Selected partition 1
  Hex code (type L to list codes): 83
  Changed system type of partition 1 to 83 (Linux)

#+end_src

5. This completes all the changes we need to make. Up to this point, the device has been untouched (all the changes have been stored in memory, not on the physical device), so we will write the modified partition table to the device and exit. To do this, we enter =w= at the prompt.

Additionally, to create an ~ext4~ *file system* on the device, we use the ~-t~ option to specify the ~ext4~ system type, followed by the name of the device containing the partition we want to format.

#+begin_src sh

  sudo mkfs -t ext4 /dev/sdb1

#+end_src

To reformat the device to its original FAT32 file system, specify ~vfat~ as the file system
type.

#+begin_src sh

  sudo mkfs -t vfat /dev/sdb1

#+end_src

** ACL
An access-control list (ACL) is a list of permissions associated with a system resource (object or facility). An ACL specifies which users or system processes are granted access to resources, as well as what operations are allowed on given resourcess. Each entry in a typical ACL specifies a subject and an operation. For instance, If a file object has an ACL that contains ~(Alice: read,write; Bob: read)~, this would give Alice permission to read and write the file and give Bob permission only to read it.

** Filesystem ACLs
A filesystem ACL is a data structure (usually a table) containing entries that specify individual user or group rights to specific system objects such as programs, processes, or files. These entries are known as access-control entries (ACEs). Each accessible object contains an identifier to its ACL. The privileges or permissions determine specific access rights, such as whether a user can read from, write to, or execute an object. In some implementations, an ACE can control whether or not a user, or group of users, may alter the ACL on an object.

** Directories and partitions
The following is a list of important considerations regarding directories and partitions.

- / :: root partition / must always physically contain /etc, /bin, /sbin, /lib, /dev and /usr, otherwise you won't be able to boot. This means that you should provide at least 600–750MB of disk space for the root partition including /usr, or 5–6GB for a workstation or a server installation.

- /var :: variable data like news articles, e-mails, web sites, databases, the packaging system cache, etc. will be placed under this directory. The size of this directory depends greatly on the usage of your system, but for most people will be dictated by the package management tool's overhead. If you are going to do a full installation of just about everything Debian has to offer, all in one session, setting aside 2 or 3 GB of space for /var should be sufficient. If you are going to install in pieces (that is to say, install services and utilities, followed by text stuff, then X, ...), you can get away with 300–500 MB. If hard drive space is at a premium and you don't plan on doing major system updates, you can get by with as little as 30 or 40 MB.

- /tmp :: temporary data created by programs will most likely go in this directory. 40–100MB should usually be enough. Some applications — including archive manipulators, CD/DVD authoring tools, and multimedia software — may use /tmp to temporarily store image files. If you plan to use such applications, you should adjust the space available in /tmp accordingly.

- /home :: every user will put his personal data into a subdirectory of this directory. Its size depends on how many users will be using the system and what files are to be stored in their directories. Depending on your planned usage you should reserve about 100MB for each user, but adapt this value to your needs. Reserve a lot more space if you plan to save a lot of multimedia files (pictures, MP3, movies) in your home directory.

* Filesystem hierarchy
** /
*** /vmlinuz
Is normally a binary file that contains the kernel. A boot loader loads this file into memory and sets it in motion when the system boots. Once the boot loader starts the kernel, the main kernel file is no longer used by the running system.

** /bin
Contains ready-to-run programs (also known as executables), including most of the basic Unix commands such as ~ls~ and ~cp~.

This directory contains executable files that are part of the core operating system (those required by the system for emergency repairs, booting, and single user mode). These files need to be accessible before [[ /usr]] gets mounted. (for instance, the ~mount~ command is in ~/bin/mount~). But in modern systems it is the same as the [[ /usr/bin]].

This directory are specified by the Linux Filesystem Hierarchy Standard to contain only files supplied and maintained by the Linux distributor.

** /boot
Contains kernel boot loader files. These files pertain only to the very first stage of the Linux startup procedure, so you won’t find information about how Linux starts up its services in this directory.

*** /boot/vmlinuz
Is normally a binary file that contains the kernel A boot loader loads this file into memory and sets it in motion when the system boots. Once the boot loader starts the kernel, the main kernel file is no longer used by the running system.

*** /boot/grub
The GRUB configuration directory.

** /dev
Contains device files.

*** /dev/sd*
These are the current hard disks connected to Linux systems. These devices represent entire disks.

*** /dev/sr*
Optical storage drives. This devices are read only, and they are used only for reading from discs.

*** /dev/tty
This device is the controlling terminal of the current process. If a program is currently reading from and writing to a terminal, this device is a synonym for that terminal. A process does not need to be attached to a terminal.

*** /dev/tty1
The first virtual console, which is the one you see when you press ~CTRL-ALT-F1~ or ~ALT-F1~.

*** /dev/pts/0
The first pseudoterminal device.

*** /dev/input
Contains the input of the several kernel devices (mouse, keyboard, touch-pad,...).

*** /dev/null
This file is a system device often referred to as a *bit bucket*, which accepts input and does nothing with it.

Descarta toda la información que se escribe en o se redirige hacia él. A su vez, no proporciona ningún dato a cualquier proceso que intente leer de él, devolviendo simplemente un EOF o fin de archivo.

** /etc
Host-specific system configuration.

This core system configuration directory contains the user password, boot, device, networking, and other setup files.

The configurations for a single machine, such as user information ~/etc/passwd~ and network details ~/etc/network~, go into this directory. However, general application details, such as a distribution’s defaults for a user interface, don’t belong here. System default configuration files not meant to be customized also are usually found elsewhere, as with the prepackaged systemd unit files in ~/usr/lib/systemd~.

*** /etc/apt/sources.list
Archivo que utiliza APT que enlista las "fuentes" en donde se encuentran los paquetes. El archivo puede contener varios tipos de líneas. APT sabe como interpretar líneas del tipo http, ftp, file (archivos locales, p.e., un directorio que contiene un CD-ROM) y ssh.

#+begin_src sh

  #servidor oficial
  deb http://security.debian.org/debian-security bullseye-security main non-free contrib
  #paquetes fuente (normalmente usados para probar o recompilar)
  deb-src http://security.debian.org/debian-security bullseye-security main non-free contrib

  deb http://deb.debian.org/debian bullseye-proposed-updates main non-free contrib

  #servidor de actualizaciones de seguridad
  deb http://deb.debian.org/debian/ bullseye main non-free contrib
  #paquetes fuente (normalmente usados para probar o recompilar)
  deb-src http://deb.debian.org/debian/ bullseye main non-free contrib #Added by software-properties

#+end_src

- La primera palabra en cada línea indica el tipo de archivo: /deb/ paquetes binarios pre-compilados que normalmente se usan. /deb-src/  paquetes fuente que son los códigos originales.
- El primer argumento es la URL raiz de los archivos Debian.
- El segundo argumento es el nombre de la distribución.
- El tercer argumento y los siguientes son la lista de nombres de área de archivo válidos del archivo de Debian: /main/ cumple con DFSG y sin dependencias con *non-free* . /contrib/ cumple con DFSG pero con dependencias con *non-free* . /non-free/ no cumple con DFSG.

*Nota*: suite name(codename)= stable(bullseye), testing(bookworm), unstable(siempre: sid).

*** /etc/passwd
This plaintext file maps usernames to user IDs.

Each line represents one user and has seven fields separated by colons.

#+begin_src

    ┌ Login name
    |   ┌ Password
    |   |   ┌ User ID
    |   |   |    ┌ Group ID
    |   |   |    |         ┌ Real name (GECOS)
    |   |   |    |         |            ┌ Home directory
    │   |   |    |         |            |          ┌ Shell
  ┌─┴─┐┌┴┐┌─┴─┐┌─┴─┐┌──────┴──────┐┌────┴────┐ ┌───┴───┐
  juser:x:3119:1000:J. Random User:/home/juser:/bin/bash

#+end_src

- The first field is the username.
- The second field in passwd or shadow is the encrypted password. An ~x~ indicates that the encrypted password is stored in the shadow file. An asterisk (~*~) indicates that the user cannot log in. If this password field is blank (~::~), no password is required to log in.
- The user ID (UID), which is the user’s representation in the kernel.
- The group ID (GID) Groups determine file permissions and little else.
- The user’s real name (often called the GECOS field).
- The user’s home directory.
- The user’s shell

*** /etc/shadow
The shadow password file normally contains user authentication information, including the encrypted passwords and password expiration information that correspond to the users in [[ /etc/passwd]].

*** /etc/group
This file defines the group IDs (such as the ones found in the [[ /etc/passwd]] file).

Each line in this file is a set of fields separated by colons.

An ~x~ in password file means that there’s a corresponding entry in ~/etc/gshadow~, and this is also nearly always a disabled password, denoted with a ~*~ or ~!~.

#+begin_src

   ┌ Group name
   |  ┌ Password
   |  |  ┌ Group ID
   |  |  |       ┌ Additional members
  ┌┴┐┌┴┐┌┴┐┌─────┴─────┐
  disk:*:6:juser,beazley

#+end_src

*** /etc/sudoers
Configure the privileged users.

Example, this file gives ~user1~ and ~user2~ the power to run any command as root without having to enter a password:

#+begin_src sh

  User_Alias ADMINS = user1, user2

  ADMINS ALL = NOPASSWD: ALL root ALL=(ALL) ALL

#+end_src

The first line defines an ~ADMINS~ user alias with the two users, and the second line grants the privileges. The ~ALL = NOPASSWD: ALL~ part means that the users in the ~ADMINS~ alias can use sudo to execute commands as root. The second ~ALL~ means “any command.” The first ~ALL~ means “any host.” (If you have more than one machine, you can set different kinds of access for each machine or group of machines.)

The ~root ALL=(ALL) ALL~ simply means that the superuser may also use sudo to run any command on any host. The extra ~(ALL)~ means that the superuser may also run commands as any other user. You can extend this privilege to the ~ADMINS~ users by adding ~(ALL)~ to the second ~/etc/sudoers~ line, as shown here:

#+begin_src sh

  ADMINS ALL = (ALL) NOPASSWD: ALL

#+end_src

*** /etc/fstab
List of filesystems and options.

Inside this file, each line corresponds to one filesystem and is broken into six fields. From left to right, these fields are:

- the device or UUID :: most current Linux systems no longer use the device in ~/etc/fstab~, preferring the UUID.
- the mount point :: indicates where to attach the filesystem.
- the filesystem type :: you may not recognize ~swap~ in this list; this is a swap partition.
- options :: long options, separated by commas.
- backup information for use by the dump command :: the ~dump~ command is a long-obsolete backup utility; this field is no longer relevant. You should always set it to 0.
- the filesystem integrity test order :: to ensure that ~fsck~ always runs on the root first, always set this to 1 for the root filesystem and 2 for any other locally attached filesystems on a hard disk or SSD. Use 0 to disable the bootup check for every other filesystem, including read-only devices, swap, and the ~/proc~ filesystem.

You’ll often see the ~defaults~ option. These options are defined as follows:

- defaults :: this sets the ~mount~ defaults: read-write mode, enable device files, executables, the ~setuid~ bit, and so on. Use this when you don’t want to give the filesystem any special options but you do want to fill all fields in ~/etc/fstab~.
- errors :: this ext2/3/4-specific parameter sets the kernel behavior when the system has trouble mounting a filesystem. The default is normally ~errors=continue~, meaning that the kernel should return an error code and keep running. To have the kernel try the mount again in read-only mode, use ~errors=remount-ro~. The ~errors=panic~ setting tells the kernel (and your system) to halt when there’s a problem with the mount.
- noauto :: this option tells a ~mount -a~ command to ignore the entry. Use this to prevent a boot-time mount of a removable-media device, such as a flash storage device.
- user :: this option allows unprivileged users to run ~mount~ on a particular entry, which can be handy for allowing certain kinds of access to removable media. Because users can put a ~setuid-root~ file on removable media with another system, this option also sets ~nosuid~, ~noexec~, and ~nodev~ (to bar special device files). Keep in mind that for removable media and other general cases, this option is now of limited use, because most systems use ~ubus~ along with other mechanisms to automatically mount inserted media. However, this option can be useful in special cases when you want to grant control over mounting specific directories.

*** /etc/systemd/system
System unit configuration directory (local definitions).

You should normally put your own unit files in this directory so that you won’t confuse them with anything that came with your distribution and so that the distribution won’t overwrite them when you upgrade.

*** /etc/rc*.d
The contents of the ~rc*.d~ directories are actually symbolic links to files in yet another directory, [[ /etc/init.d]].

The ~rc~ stands for run commands, which many people refer to as scripts, programs, or services, and the ~5~ in ~rc5.d~ tells us that we’re talking about runlevel ~5~.

The ~rc5.d~ directory contain file like this ~S10sysklogd~, ~S20ppp~, ~S99gpm~... in this files the capital ~S~ in a command name means that the command should run in start mode, and the number (~00~ through ~99~) determines where in the sequence ~rc~ starts the command. The ~rc*.d~ commands are usually shell scripts that start programs in [[ /sbin]] or [[ /usr/sbin]].

Some ~rc*.d~ directories contain commands that start with ~K~ (for “kill,” or stop mode). In this case, ~rc~ runs the command with the ~stop~ argument instead of ~start~. You’ll most likely encounter ~K~ commands in runlevels that shut down the system.

*** /etc/init.d
Contains a number of start/stop scripts for various services on the system (changing init states).

To start and stop services by hand, use the script in this ~init.d~ directory. For example, one way to start the *httpd web* server program manually is to run ~init.d/httpd start~. Similarly, to kill a running service, you can use the ~stop~ argument (~httpd stop~, for instance).

The mechanism that System V init uses to run the ~init.d~ scripts has found its way into many Linux systems, regardless of whether they use System V init. It’s a utility called ~run-parts~, and the only thing it does is run a bunch of executable programs in a given directory, in some kind of predictable order.

The default behavior is to run all programs in a directory, but you often have the option to select certain programs and ignore others.

Distributions, such as Debian and Ubuntu, have a more complicated ~run-parts~ program. Their features include the ability to run programs based on a regular expression (for example, using the ~S[0-9]{2}~ expression for running all “start” scripts in an ~/etc/init.d~ runlevel directory) and to pass arguments to the programs. These capabilities allow you to start and stop System V runlevels with a single command.

*** /etc/anacrontab
Configuration file for anacron.

Contains specifications of jobs.  See ~anacrontab(5)~ for a complete description.

*** /etc/cron.{daily,weekly,monthly}
These directories contain scripts (not crontabs!) that get executed sequentially in lexical sort order via run-parts from [[ /etc/crontab]] in the respective intervals. It is currently unknown whether there are packages that depend on the lexical sort order of cron jobs to be executed.

*** /etc/crontab
Usually only holding entries to run the jobs from [[ /etc/cron.{daily,weekly,monthly}]]. Packages MUST NOT touch this file according to Policy.

Useful for *scheduling system maintenance tasks* like updates.

Many common cron-activated system tasks are run as the superuser. However, rather than editing and maintaining a superuser’s crontab to schedule these, Linux distributions normally have an ~/etc/crontab~ file for the entire system. You won’t use ~crontab~ to edit this file, and in any case, it’s slightly different in format: before the command to run, there’s an additional field specifying the user that should run the job. (This gives you the opportunity to group system tasks together even if they aren’t all run by the same user.)

#+begin_src sh

  # this cron job runs at 6:42 AM as the superuser (root 1)
  42 6 * * * root1 /usr/local/bin/cleansystem > /dev/null 2>&1

#+end_src

A star (~*~) in any field means to match every value. The example below runs ~spmake~ daily because the day of month, month, and day of week fields are all filled with stars, which cron reads as “run this job every day, of every month, of every day of the week.” (these examples are similar, usually goin in another file, they don't specify the user who runs the command).

#+begin_src sh

  ┌ Minute (0 - 59)
  |  ┌ Hour (0 - 23)
  |  |  ┌ Day of month (1 - 31)
  |  |  |  ┌ Month (1 - 12)
  |  |  |  |  ┌ Day of week (0 - 6) (Sunday=0 or 7)
  |  |  |  |  |          ┌ Command
  ┌┴┐┌┴┐┌┴┐┌┴┐┌┴┐┌────────┴────────┐
  15 09 * * * /home/juser/bin/spmake

  # run spmake only on the 14th day of each month
  15 09 14 * * /home/juser/bin/spmake

  # run spmake on the 5th and the 14th day of each month
  15 09 5,14 * * /home/juser/bin/spmake

#+end_src

*** /etc/cron.d
These files are considered as additions to the global [[ /etc/crontab]] and treated the same. This allows both packages and the local administrator to drop in system-wide cron jobs without having to modify ~/etc/crontab~.

Useful for *separating different types of cron tasks* into individual files instead of a single large ~/etc/crontab~.

*** /etc/hosts
Contains the Internet Protocol (IP) host names and addresses for the local host and other hosts in the Internet network. This file is used to resolve a name into an address (that is, to translate a host name into its Internet address).

#+begin_src sh

  127.0.0.1  localhost
  10.23.2.3  atlantic.aem7.net    atlantic
  10.23.2.4  pacific.aem7.net     pacific
  ::1        localhost ip6-localhost
  # The other entries here illustrate a simple way to add hosts
  # on a local subnet.

#+end_src

*** /etc/resolv.conf
Thi is the traditional configuration file for DNS servers.

*** /etc/nsswitch.conf
Is the traditional interface for controlling several name-related precedence settings on your system, such as user and password information, and it has a host lookup setting.

It can be complicated to remember all of the places that name lookups can happen, but if you ever need to trace something from the bottom up, start with this file.

#+begin_src shell

  #Dicta el orden de resolución de host.
  hosts:          files mdns4_minimal [NOTFOUND=return] dns myhostname

#+end_src

- Si se encuentra primero "files", buscara primero en el archivo ~/etc/hosts~ y devolvera todas las direcciones validas para él y sale.
- Se invoca el método "dns". Si la consulta al (DNS) de Internet identificado por el archivo ~/etc/resolv.conf~  encuentra el nombre de host, devuelve todas las direcciones válidas para él y sale.

*** /etc/services
Contains the service names and port assignments of specific z/OS UNIX applications, i.e. translates well-known port numbers into names.

*** /etc/ssh/sshd_config
This is the sshd server system-wide configuration file.

*** /etc/ssh/ssh_config
This is the ssh client system-wide configuration file.

*** /etc/fonts/conf.d
Each file in this directory is a fontconfig configuration file. These files are normally installed in [[ /usr/share/fontconfig/conf.avail]] and then symlinked here, allowing them to be easily installed and then enabled/disabled by adjusting the symlinks.

*** /etc/profile
A global configuration script for [[Interactive shells][login shell]] sessions that applies to all users.

*** /etc/bash.bashrc
A global configuration script for [[Interactive shells][non-login shell]] sessions that applies to all users.

*** /etc/netwok/interfaces
Interface configuration (if not controlled via network-manager).

*** /etc/default
Default values for many daemons and services.

*** /etc/exim4/passwd.client
The authentication information for the external SMTP service using ~exim4~ is stored in this file.

** /home
Holds home (personal) directories for regular users.

*** /home/user/_bash_profile
A user's personal startup file for [[Interactive shells][login shell]]. This can be used to extend or override settings in the global configuration script.

*** /home/user/_bash_login
If [[ /home/user/_bash_profile]] is not found, bash attempts to read this script ([[Interactive shells][login shell]]).

*** /home/user/_profile
If neither [[ /home/user/_bash_profile]] nor [[ /home/user/_bash_login]] is found, bash attempts to read this file. ([[Interactive shells][login shell]]) This is the default in Debian-based distributions, such as Ubuntu.

*** /home/user/_bashrc
A user's personal startup file for [[Interactive shells][non-login shell]]. It can be used to extend or override settings in the global configuration script.

*** /home/user/dot.local/share/Trash
The user trash-bins (or wastebaskets) folder. *Note*: ~dot.local~ is actually ~.local~.

The other two trash-bins are:
- The administrator wastebasket : ~/root/.local/share/Trash~
- The external wastebaskets : locates on your external disks, they are usually named ~/media/your_id/your_disk/.Trash_1000~

** /lib
An abbreviation for library, this directory holds library files containing code that executables can use. This directory should not contain static libraries.

*** /lib/modules
Are modules that the kernel loads and unloads on demand during the course of normal system operation.

*** /lib/systemd/system/anacron.service
This file provides systemd service for anacron.

*** /lib/systemd/system/anacron.timer
This  file  provides  systemd  timer  for anacron. Currently the service is triggered hourly through systemd timer.

** /media
A base attachment point for removable media such as flash drives that is found in many distributions.

** /mnt
Mount point for mounting a file system temporarily.

** /proc
It's a virtual file system maintained by the Linux kernel. Provides system *(process/runtime) statistics* through a browsable directory-and-file interface. The ~/proc~ directory contains information about currently running processes as well as some kernel parameters. The files are readable and will give us a picture of how the kernel sees the computer.

Each numbered directory inside ~/proc~ refers to the ID of a current process on the system; the files in each directory represent various aspects of that process. The directory ~/proc/self~ represents the current process. The Linux ~proc~ filesystem includes a great deal of additional kernel and hardware information in files like ~/proc/cpuinfo~. Keep in mind that the kernel design guidelines recommend moving information unrelated to processes out of ~/proc~ and into ~/sys~, so system information in ~/proc~ might not be the most current interface.

*** /proc/<pid>
Show directories and files assosiate with <pid> process.

*** /proc/<pid>/fd
Conjunto de descriptores de archivos ([[File Descriptor]]) abiertos en un proceso PID. Cada proceso en UNIX debe tener tres descriptores de archivo POSIX estándar, correspondientes a los tres flujos estándar:
- El descriptor de archivo ~0~ es el *stdin* .
- El descriptor de archivo ~1~ es el *stdout* .
- El descriptor de archivo ~2~ es el *stderr* .
Los otros descriptores de archivos pueden ser archivos que ha abierto el proceso para utilizar, como por ejemplo, un archivo llamado ~file~ abierto por el proceso  ~less file~ .

En la implementación tradicional de Unix, los descriptores de archivos se indexan en una tabla de descriptores de archivos por proceso mantenida por el kernel, que a su vez se indexa en una tabla de archivos de todo el sistema abierta por todos los procesos, llamada tabla de archivos. Esta tabla registra el modo con el que se ha abierto el archivo (u otro recurso): para lectura, escritura, anexión y posiblemente otros modos. También se indexa en una tercera tabla llamada tabla de inodos que describe los archivos subyacentes reales. Para realizar la entrada o la salida, el proceso pasa el descriptor de archivo al kernel a través de una llamada al sistema y el kernel accederá al archivo en nombre del proceso. El proceso no tiene acceso directo al archivo ni a las tablas de inodos.

*** /proc/<pid>/cgroup
List the v1 and v2 [[Control Groups (cgroups)][cgroups]] for any process.
Every line of output here starts with a number and is a different cgroup.

- Numbers 2–12 are for cgroups v1. The controllers for those are listed next to the number.
- Number 1 is also for version 1, but it does not have a controller. This cgroup is for management purposes only (in this case, systemd configured it).
- The last line, number 0, is for cgroups v2. No controllers are visible here. On a system that doesn’t have cgroups v1, this will be the only line of output.
- Names are hierarchical and look like parts of file paths. You can see in this example that some of the cgroups are named ~/user.slice~ and others ~/user.slice/user-1000.slice/session-2.scope~.
- The name ~/testcgroup~ was created to show that in cgroups v1, the cgroups for a process can be completely independent.
- Names under ~user.slice~ that include session are login sessions, assigned by systemd. You’ll see them when you’re looking at a shell’s cgroups. The cgroups for your system services will be under ~system.slice~.

#+begin_src sh

  cat /proc/self/cgroup
  # 12:rdma:/
  # 11:net_cls,net_prio:/
  # 10:perf_event:/
  # 9:cpuset:/
  # 8:cpu,cpuacct:/user.slice
  # 7:blkio:/user.slice
  # 6:memory:/user.slice
  # 5:pids:/user.slice/user-1000.slice/session-2.scope
  # 4:devices:/user.slice
  # 3:freezer:/
  # 2:hugetlb:/testcgroup
  # 1:name=systemd:/user.slice/user-1000.slice/session-2.scope
  # 0::/user.slice/user-1000.slice/session-2.scope

  # Let’s explore the cgroup setup of a shell.
  cd /sys/fs/cgroup/user.slice/user-1000.slice/session-2.scope/
  ls
  # Lists the processes in the cgroup.
  cat cgroup.procs
  # Show the controllers currently in use for the cgroup.
  cat cgroup.controllers
  # Number of threads running in the cgroup.
  cat pids.current
  # Show the maximum amount of memory that the cgroup can consume.
  cat memory.max

#+end_src

*** /proc/self
Is a link to the currently running process. This allows a process to look at itself without having to know its process ID.

*** /proc/devices
Show the block and character devices for which your system currently has drivers. Each line consists of a number and name. The number is the [[Major and Minor Numbers][major number]] of the device.

*** /proc/meminfo
Show how much real memory is being used for caches and buffers.

*** /proc/cpuinfo
Incluye gran cantidad de información adicional sobre el kernel y el hardware.

*** /proc/version
Version of the Linux kernel used, its name, the version of the compiler used.

*** /proc/partitions
Full partition information.

*** /proc/swaps
Contiene las particiones swap disponibles.

*** /proc/cmdline
Show the parameters passed to your system’s currently running kernel.

*** /proc/stat
Contains a variety of statistics about the system, such as the number of processes running, the number of interrupts, and the amount of time spent in each CPU state.

*** /proc/uptime
Contains the amount of time the system has been running.

*** /proc/sys
Contiene interfaces para cambiar ciertos parámetros del kernel en tiempo de ejecución. (Se puede hacer lo mismo mediante el comando especializado ~sysctl(8)~ o su archivo de precarga/configuración ~/etc/sysctl.conf~ ).

** /root
This is the home directory for the root account.

** /run
Contains runtime variable data specific to the system, including certain process IDs, socket files, status records, and, in many cases, system logging.

El directorio ~/run~ se monta como ~tmpfs~ en el proceso de arranque inicial. Esto permite escribir en él incluso cuando el directorio "~/~" está montado como de solo lectura. Esta es la nueva ubicación para el almacenamiento de archivos de estado transitorio y reemplaza varias ubicaciones descritas en Filesystem Hierarchy Standard.

*** /run/systemd/system
Archivos de configuración generados en tiempo de ejecución que anulan los archivos de configuración instalados.

** /sbin
Essential system binaries.

Programs in ~/sbin~ directories relate to system management, so regular users usually do not have ~/sbin~ components in their command paths.

These are programs that perform vital system tasks that are generally reserved for the superuser.

** /sys
Virtual directory for system information. Exposes lower-level kernel and hardware objects, allowing granular monitoring and configuration of components like devices, modules, network stack via virtual files. (component/hardware access)

This directory is similar to [[ /proc]] in that it provides a device and system interface.

~/sys~ and ~/proc~ provide complete system ovservability spanning high level metrics to low level component interactions.

*** /sys/devices
To provide a uniform view for attached devices based on their actual hardware attributes, the Linux kernel offers the sysfs interface through a system of files and directories.

This path has different purposes than ~/dev/sda~ filename, which is also a directory. The [[ /dev]] file enables user processes to use the device, whereas the ~/sys/devices~ path is used to view information and manage the device.

*** /sys/block
Contain all of the block devices available on a system. However, those are just symbolic links.

*** /sys/fs/cgroup
[[Control Groups (cgroups)][cgroups]] are accessed entirely through the filesystem, which is usually mounted as a cgroup2 filesystem under this directory.

*** /sys/class/power_supply/AC/online
Checks if power cable is connected.

** /tmp
A storage area for smaller, temporary files created by various programs. Some configurations cause this directory to be emptied each time the system is rebooted.

Any user may read to and write from ~/tmp~, but the user may not have permission to access another user’s files there.

As of debian Trixie the new default behavior is to automatically deleted files from this directorory after 10 days from the time they were last used (as well as after a reboot). Files in [[ /var/tmp]] are deleted after 30 days (but not deleted after a reboot).

** /usr
Unix System Resorces contains a large directory hierarchy, including the bulk of the Linux system. ~/usr~ is where most of the user-space programs and data reside.

*** /usr/local
Is where administrators can install their own software (programs that are not included with the distribution). Its structure should look like that of ~/~ and ~/usr~.

*** /usr/local/src
Is where administrators can place the source code directory of a program for installation.

*** /usr/local/share/fonts
Install a font manually.

Install a font manually by downloading the appropriate ~.ttf~ or ~otf~ files and placing them into ~/usr/local/share/fonts~ (system-wide), ~~/.local/share/fonts~ (user-specific) or ~~/.fonts~ (user-specific). These files should have the permission 644 (-rw-r--r--), otherwise they may not be usable.

Fonts can be added system-wide to Debian by installing the appropriate package. As default system-wide fonts will be installed into [[/usr/share/fonts]] by the package-manager.

*** /usr/share/fonts
Fonts can be added system-wide to Debian by installing the appropriate package. As default system-wide fonts will be installed into ~/usr/share/fonts~ by the package-manager.

*Notes*:
- For most uses, you’ll want TrueType (TTF) and OpenType (OTF) fonts – these packages start with ~fonts-~.
- Some fonts might have a -variable build, which are known as (VF), variable font.
- Some non-free font downloader packages are in ~contrib~, which you will need to add to your ~sources~ if not present.

*Manually*
Install a font manually by downloading the appropriate ~.ttf~ or ~otf~ files and placing them into [[/usr/local/share/fonts]] (system-wide), ~~/.local/share/fonts~ (user-specific) or ~~/.fonts~ (user-specific). These files should have the permission 644 (-rw-r--r--), otherwise they may not be usable.

*** /usr/bin
Contains executable files that are not part of the core operating system. These are considered to be "system wide binaries". But in modern systems it is the same as the [[ /bin]].

This directory are specified by the Linux Filesystem Hierarchy Standard to contain only files (executables for nearly all packaged user software) supplied and maintained by the Linux distributor.

*** /usr/include
Holds header files used by the C compiler.

*** /usr/lib/systemd/user
Another system unit directory.

*** /usr/man
Contains manual pages.

*** /usr/share
Contains files that should work on other kinds of Unix machines with no loss of functionality. These are usually auxiliary data files that programs and libraries read as necessary.

This includes things such as default configuration files, icons, screen backgrounds, sound files, etc.

It typically contains architecture-independent (shared) data. This means it holds files that are not specific to the architecture of your machine, such as certain shell scripts, man pages, locale information, and other miscellaneous system-wide data.

This is particularly useful in networked environments where multiple machines can share a common data source, thereby saving storage and ensuring consistency.

*** /usr/share/doc
Some packages dump their available documentation into this directory with no regard for online manual systems such as man or info. See this directory on your system if you find yourself searching for documentation.

*** /usr/share/zoneinfo
This directory contains a lot of time zones and aliases for time zones.

To set your system’s time zone manually, either copy one of the files in ~/usr/share/zoneinfo~ to ~/etc/localtime~ (or make a symbolic link) or change it with your distribution’s time zone tool. The command-line program ~tzselect~ may help you identify a time zone file.

*** /usr/share/fontconfig/conf.avail
Each file in this directory is a fontconfig configuration file.

*** /usr/share/dict
Contains a dictionaries.

** /var
The variable subdirectory date, where programs record information that can change over the course of time. System logging, user tracking, caches, and other files that system programs create and manage are here.

*** /var/tmp
Temporary files that are deleted after 30 days (but not deleted after a reboot).

*** /var/log
Contains log files, records of various system activity.

If you have a version of syslogd, this directory should contain many files, most created by your syslog daemon. However, there will be a few files here that are maintained by other services. In addition, there may be further subdirectories containing logs.

*** /var/log/messages
The main log file that show what the system is doing in near real-time. It's used to store non-debug and non-critical system messages. This includes messages with log levels such as =info,= =notice,= and =warn.= It’s often referred to as the “general system activity” log.

Useful for real-time view to determine how the removable device is named when it is attached.

*Tip*: Using the ~tail -f /var/log/messages~ technique is a great way to watch what the system is doing in near real-time.

*** /var/log/syslog
Is a standard logging facility. It collects messages of various programs and services including the kernel, and stores them, depending on setup, in a bunch of log files typically under [[ /var/log]]. It logs everything except authentication-related messages.

In some datacenter setups there are hundreds of devices each with its own log; syslog comes in handy here too. One just sets up a dedicated syslog server which collects all the individual device logs over the network.

En este se puede encontrar los mensajes ~anacron~.

*Note* (StackExchange):
In Debian 12, the traditional syslog system has been replaced with systemd-journald. This means that the old log files like ~syslog~, ~messages~, ~auth.log~ are no longer used. Instead, all system logs are now stored in a centralized journal, which is accessible using the ~journalctl~ command.

*DuckAssist*:
If you prefer syslog files, you can install the ~rsyslog~ package to restore them.

*** /var/log/apt/history.log
Log file used on Debian-based systems to track actions performed by the APT (Advanced Package Tool) package management system. This file contains information about package installations, updates, and removals performed on the system.

Other similar usuful files are ~/var/log/apt/term.log~ and ~/var/log/dpkg.log~.

*** /var/log/journal
Is where *journald* stores its (binary) logfiles.

The journal is a component of [[systemd]]. It’s a centralized location for all messages logged by different components in a systemd-enabled Linux system. This includes kernel and boot messages, messages coming from syslog, or different services.

The journal is controlled by the systemd-journald daemon. It collects information from different sources and loads the messages into the journal.

*** /var/log/kern.log
View the messages in the *kernel* ring buffer.

*** /var/log/auth.log
Containing the most recent *log* messages.

*** /var/log/exim4/mainlog
Verification of the configuration and troubleshooting the exim4 mail system.

*** /var/mail/account_name
When system e-mails are delivered they are added to a file in this file. The e-mails can be read using ~mutt~.

*** /var/spool/anacron
This directory is used by Anacron for storing timestamp files.

*** /var/spool/cron/crontabs
Holding a crontab per user. These crontabs usually get maintained via the ~crontab(1)~ tool. root's "personal" crontab is a great place to hide jobs you will never find again.

Typical location for *user automation and personal scripts*.

Each user can have their own crontab file, usually found in this directory. Normal users can’t write to this directory; the ~crontab~ command installs, lists, edits, and removes a user’s crontab.

The easiest way to install a crontab is to put your crontab entries into a file and then use ~crontab file~ to install ~file~ as your current crontab. The ~crontab~ command checks the file format to make sure that you haven’t made any mistakes. To list your cron jobs, run ~crontab -l~. To remove the crontab, use ~crontab -r~.

After you’ve created your initial crontab, it can be a bit messy to use temporary files to make further edits. Instead, you can edit and install your crontab in one step with the ~crontab -e~ command. If you make a mistake, ~crontab~ should tell you where the mistake is and ask if you want to try editing again.

*** /var/lib/apt/lists/*
Find package information.

*** /var/lib/dpkg/available
List of available packages from repositories.

*** /var/lib/dpkg/status
Status of installed and available packages. This file contains information about whether a package is marked for removal or not, whether it is installed or not, etc. A package marked reinst-required is broken and requires re-installation.

*** /var/run/dbus
Contain the unix domain socket files for D-Bus.

*** /var/cache/apt/archives
Almacena los paquetes obtenidos vía mecanismos APT hasta que estos sean borrados, por lo general mediante ~apt clean~.

*Estructura de un paquete binario .deb*:
package-name_upstream-version-debian.revision_architecture.deb
~chromium-common_106.0.5249.119-1~deb11u1_amd64.deb~

*** /var/www/html/index.html
Archivo html que sirve para probar el servidor local apache2.

** /srv
Data for services provided by the system.

** /opt
Add-on application software packages. This may contain additional third-party software.

** /lost+found
Each formatted partition or device using a Linux file system, such as ext4, will have this directory. It is used in the case of a partial recovery from a file system corruption event.  Unless something really bad has happened to our system, this directory will remain empty.

It provides a way to ~fsck~ (File System Check utility) to recover parts as small data structure.

When ~fsck~ asks you about reconnecting an inode, it has found a file that doesn’t appear to have a name. When reconnecting such a file, ~fsck~ places the file in the ~lost+found~ directory in the filesystem, with a number as the filename. If this happens, you need to guess the name based on the file’s contents; the original filename is probably gone.

* The C Compiler
** Compiling
Compiling is the process of translating *source code* (the human-readable description of a program written by a programmer) into the native language of the computer’s processor.

The computer’s processor (or [[CPU]]) works at an elemental level, executing programs in what is called *machine language*. This is a numeric code that describes extremely small operations, such as “add this byte,” “point to this location in memory,” or “copy this byte.” Each of these instructions is expressed in binary (ones and zeros). The earliest computer programs were written using this numeric code.

This problem was overcome by the advent of *assembly language*, which replaced the numeric codes with (slightly) easier to use character mnemonics such as ~CPY~ (for copy) and ~MOV~ (for move). Programs written in assembly language are processed into machine language by a program called an *assembler*. Assembly language is still used today for certain specialized programming tasks, such as device drivers and embedded systems.

We next come to what are called *high-level programming languages*. They are called this because they allow the programmer to be less concerned with the details of what the processor is doing and more with solving the problem at hand. The early ones (developed during the 1950s) include FORTRAN (designed for scientific and technical tasks) and COBOL (designed for business applications).

Programs written in high-level programming languages are converted into machine language by processing them with another program, called a *compiler*. Some compilers translate high-level instructions into assembly language and then use an assembler to perform the final stage of translation into machine language.

A process often used in conjunction with compiling is called *linking*. There are many common tasks performed by programs. Take, for instance, opening a file. Many programs perform this task, but it would be wasteful to have each program implement its own routine to open files. It makes more sense to have a single piece of programming that knows how to open files and to allow all programs that need it to share it. Providing support for common tasks is accomplished by what are called *libraries*. They contain multiple *routines*, each performing some common task that multiple programs can share. If we look in the [[ /lib]] and ~/usr/lib~ directories, we can see where many of them live. A program called a *linker* is used to form the connections between the output of the compiler and the libraries that the compiled program requires. The final result of this process is the *executable program* file, ready for use.

** Object file
Is a binary file that a processor can almost understand, except that there are still a few loose ends. First, the operating system doesn’t know how to start up an object file, and second, you likely need to combine several object files and some system libraries to make a complete program.

To build a fully functioning executable program from one or more object files, you must run the *linker*, the ~ld~ command in Unix. The ~make~ system is the traditional Unix standard for managing and automating compiles.

** C library
Is a collection of common precompiled components that you can build into your program, and it’s really not much more than a bundle of object files along with some header files.

Libraries come into play primarily at link time, when the linker program ~ld~ creates an executable from object files. Linking using a library is often called *linking against* a library.

** Shared and Static Libraries
A library file ending with ~.a~ (such as ~libcurses.a~) is called a *static library*. When you link a program against a static library, the linker copies the necessary machine code from the library file into your executable. Once it does this, the final executable no longer needs the original library file when it runs, and because your executable has its own copy of the library code, the executable’s behavior is not subject to change if the ~.a~ file changes.

Linking a program against a *shared library* doesn’t copy the code into the final executable; it just adds references to names in the code of the library file. When you run the program, the system loads the library’s code into the process memory space only when necessary. Many processes can share the same shared library code in memory. And if you need to slightly modify the library code, you can generally do so without recompiling any programs. When updating software on your Linux distribution, the packages that you’re updating can include shared libraries. When your update manager asks you to reboot your machine, sometimes it’s doing so to be sure that every part of your system is using new versions of shared libraries.

A shared library has a suffix that contains ~.so~ (shared object), as in ~libc-2.15.so~ and ~libc.so.6~.

** Header (Include)
C *header files* are additional source code files that usually contain type and library function declarations.

Double quotes in header files like ~#include "myheader.h"~ mean that the header file is not in a system include directory, and usually indicate that the include file is in the same directory as the source file. If you encounter a problem with double quotes, you’re probably trying to compile incomplete source code.

** The C Preprocessor
The C compiler doesn’t actually do the work of looking for the [[Header (Include)][include]] files. That task falls to the C preprocessor, a program that the compiler runs on your source code before parsing the actual program. The preprocessor rewrites source code into a form that the compiler understands; it’s a tool for making source code easier to read (and for providing shortcuts).

Preprocessor commands in the source code are called *directives*, and they start with the ~#~ character. There are three basic types of directives:

- Include files :: an ~#include~ directive instructs the preprocessor to include an entire file.
- Macro definitions :: a line such as ~#define BLAH something~ tells the preprocessor to substitute ~something~ for all occurrences of ~BLAH~ in the source code.
- Conditionals :: you can mark out certain pieces of code with ~#ifdef~, ~#if~, and ~#endif~. The ~#ifdef~ =MACRO= directive checks to see whether the preprocessor macro =MACRO= is defined, and ~#if condition~ tests to see whether ~condition~ is nonzero.

** make
*** Briefly
The basic idea behind ~make~ is the target, a goal that you want to achieve. A target can be a file (a ~.o~ file, an executable, and so on) or a label. In addition, some targets depend on other targets; for instance, you need a complete set of ~.o~ files before you can link your executable. These requirements are called *dependencies*.

To build a target, ~make~ follows a *rule*, such as one specifying how to go from a ~.c~ source file to a ~.o~ object file. ~make~ already knows several rules, but you can customize them to create your own.

One last ~make~ fundamental concept to know is that, in general, the goal is to bring targets up to date with their dependencies.

The following very simple Makefile builds a program called ~myprog~ from ~aux.c~ and ~main.c~:

#+begin_src c

  /* main.c */
  void hello_call();
  int main() {
    hello_call();
  }

  /* aux.c */
  #include <stdio.h>
  void hello_call() {
    printf("Hello, World.\n");
  }

#+end_src

#+begin_src sh

  # object files
  OBJS=aux.o main.o # macro definition sets OBJS variable to two object

  all: myprog       # (all) first target (is always the default)
                    # all depends on myprog
                    # myprog (is the rule for all and the target of OBJS)
  myprog: $(OBJS)   # Makefile uses the macro $(OBJS) in the dependencies
            $(CC) -o myprog $(OBJS) # The whitespace before $(CC) is a tab

#+end_src

*Note*: The whitespace before ~$(CC)~ in the Makefile file is a tab. ~make~ is very strict about tabs.You *must* insert a tab before any system command, on its own line.

This Makefile assumes that it has two C source files named ~aux.c~ and ~main.c~ in the same directory. Running ~make~ on the Makefile yields the following output, showing the commands that ~make~ is running:

#+begin_src sh

  make
  cc	-c -o aux.o aux.c
  cc	-c -o main.o main.c
  cc -o myprog aux.o main.o

#+end_src

Makefile dependencies

#+begin_src

              +--------+
              | myprog |
              +--------+
               ---/\---
          --/           \--
    +--------+        +-------+
    | main.o |        | aux.o |
    +--------+        +-------+
        |                 |
        v                 v
    +--------+        +-------+
    | main.c |        | aux.c |
    +--------+        +-------+

#+end_src

*** macros, variables, targets
**** macros

- CFLAGS :: C compiler options. When creating object code from a ~.c~ file, ~make~ passes this as an argument to the compiler.
- LDFLAGS :: like ~CFLAGS~, but these options are for the linker when creating an executable from object code.
- LDLIBS :: if you use ~LDFLAGS~ but don’t want to combine the library name options with the search path, put the library name options in this file.
- CC :: the C compiler. The default is ~cc~.
- CPPFLAGS :: C preprocessor options. When make runs the C preprocessor in some way, it passes this macro’s expansion on as an argument.
- CXXFLAGS :: GNU make uses this for C++ compiler flags.

**** variables

- $@ :: when inside a rule, this variable expands to the current target.
- $< :: when inside a rule, this variable expands to the first dependency of the target.
- $* :: this variable expands to the basename or stem of the current target. For example, if you’re building ~blah.o~, this expands to ~blah~.

**** targets

- clean :: the ~clean~ target is ubiquitous; a ~make clean~ usually instructs ~make~ to remove all of the object files and executables so that you can make a fresh start or pack up the software.
- distclean :: a Makefile created by way of the GNU autotools system always has a ~distclean~ target to remove everything that wasn’t part of the original distribution, including the Makefile.
- install :: this target copies files and compiled programs to what the Makefile thinks is the proper place on the system.
- test or check :: some developers provide ~test~ or ~check~ targets to make sure that everything works after performing a build.
- depend :: this target creates dependencies by calling the compiler with ~-M~ to examine the source code.
- all :: this is commonly the first target in the Makefile. You’ll often see references to this target instead of an actual executable.

** Configure and Makefile
*** Configure
The configure program is a shell script that is supplied with the source tree. Its job is to analyze the build environment. Most source code is designed to be portable. That is, it is designed to build on more than one kind of Unix-like system. But to do that, the source code may need to undergo slight adjustments during the build to accommodate differences between systems. configure also checks to see that necessary external tools and components are installed.

*** Makefile
The *makefile* is a configuration file that instructs the ~make~ program exactly how to build the program. Without it, make will refuse to run.

The ~make~ program takes as input a makefile (which is normally named ~Makefile~), which describes the relationships and dependencies among the components that comprise the finished program.

** Software Build Systems
*** Installing a package from C source code
1. Unpack the source code archive.
2. Configure the package (~./configure~).
3. Run ~make~ or another build command to build the programs.
4. Run ~make install~ or a distribution-specific install command to install the package.

*** GNU autoconf
GNU autoconf is a popular system for automatic Makefile generation. Packages using this system come with files named ~configure~, ~Makefile.in~, and ~config.h.in~. The ~.in~ files are templates; the idea is to run the ~configure~ script in order to discover the characteristics of your system, and then make substitutions in the ~.in~ files to create the real build files.

The default prefix in GNU autoconf and many other packages is ~/usr/local~, the traditional directory for locally installed software. Operating system upgrades ignore ~/usr/local~, so you won’t lose anything installed there during an operating system upgrade, and for small local software installations, ~/usr/local~ is fine. The only problem is that if you have a lot of custom software installed, this can turn into a terrible mess. Thousands of odd little files can make their way into the ~/usr/local~ hierarchy, and you may have no idea where the files came from.

If things really start to get unruly, you should create your own packages with ~# checkinstall make install~.

*Autoconf targets*:
- make clean :: removes all object files, executables, and libraries.
- make distclean :: this is similar to ~make clean~ except it removes all automatically generated files, including ~Makefiles~, ~config.h~, ~config.log~, and so on. The idea is that the source tree should look like a newly unpacked distribution after running ~make distclean~.
- make check :: some packages come with a battery of tests to verify that the compiled programs work properly; the command ~make check~ runs those tests.
- make install-strip :: this is like ~make install~ except it strips the symbol table and other debugging information from executables and libraries when installing. Stripped binaries require much less space.

*** An autoconf example
1. We'll install the GNU coreutils package in the own home directory.
2. Get the package from http://ftp.gnu.org/gnu/coreutils/, unpack it, change to its directory.
3. Configure it like this: ~./configure --prefix=$HOME/mycoreutils~.
4. Now run ~make~.
5. Try to run one of the executables you just created, such as ~./src/ls~, and try running ~make check~ to run a series of tests on the package.
6. Do a dry run with ~make -n install~ first to see what ~make install~ does without actually doing the install.
7. Do the install ~make install~.

** Advantages of installing on your own
*Advantages*:
- You can customize package defaults.
- When installing a package, you often get a clearer picture of how to use it. You control the release that you run.
- It’s easier to back up a custom package.
- It’s easier to distribute self-installed packages across a network.

*Disadvantages*:
- If the package you want to install is already installed on your system, you might overwrite important files, causing problems. Avoid this by using the ~/usr/local~ install prefix.
- It takes time.
- Custom packages do not automatically upgrade themselves.
- There is a potential for misconfiguring packages.

** Wrapper Script
If you must use ~LD_LIBRARY_PATH~ to run some crummy program for which you don’t have the source (or an application that you’d rather not recompile, like Firefox or some other beast), use a wrapper script. Let’s say your executable is ~/opt/crummy/bin/crummy.bin~ and needs some shared libraries in ~/opt/crummy/lib~. Write a wrapper script called crummy that looks like this:

#+begin_src sh

  #!/bin/sh
  LD_LIBRARY_PATH=/opt/crummy/lib
  export LD_LIBRARY_PATH
  exec /opt/crummy/bin/crummy.bin $@

#+end_src

Avoiding ~LD_LIBRARY_PATH~ prevents most shared library problems. But one other significant problem that occasionally comes up with developers is that alibrary’s API may change slightly from one minor version to another, breaking installed software. The best solutions here are preventive: either use a consistent methodology to install shared libraries with ~-Wl~, ~-rpath~ to create a runtime link path, or simply use the static versions of obscure libraries.

* Hardware
** Main Memory
In its rawest form, main memory is just a big storage area for a bunch of 0s and 1s. This is where the running kernel and processes reside—they’re just big collections of bits. All input and output from peripheral devices flows through main memory, also as a bunch of bits.

** CPU
Is just an operator on [[Main Memory][memory]]; it reads its instructions and data from the memory and writes data back out to the memory.

** I/O

** Disk

** Network Card Interface

* Programs
** Coreutils
*** top
**** Summary Area
***** System time, uptime and user sessions
Displays the current time. This is followed by the system uptime, which tells us the time for which the system has been running.

***** Load average
The load average section represents the average “load” over one, five and fifteen minutes. “Load” is a measure of the amount of computational work a system performs. On Linux, the load is the number of processes in the R (Running) and D (Uninterruptible sleep) states at any given moment. The “load average” value gives you a relative measure of how long you must wait for things to get done.

On a single core system, a load average of 0.4 means the system is doing only 40% of work it can do. A load average of 1 means that the system is exactly at capacity — the system will be overloaded by adding even a little bit of additional work. A system with a load average of 2.12 means that it is overloaded by 112% more work than it can handle.

*Note*: [[https://www.linuxjournal.com/article/9001][Examining Load Average]]
The Intuitive Interpretation

The three load-average values in the first line of top output are the 1-minute, 5-minute and 15-minute average. (These values also are displayed by other commands, such as uptime, not only top.) That means, reading from left to right, one can examine the aging trend and/or duration of the particular system state. The state in question is CPU load—not to be confused with CPU percentage. In fact, it is precisely the CPU load that is measured, because load averages do not include any processes or threads waiting on I/O, networking, databases or anything else not demanding the CPU. It narrowly focuses on what is actively demanding CPU time. This differs greatly from the CPU percentage. The CPU percentage is the amount of a time interval (that is, the sampling interval) that the system's processes were found to be active on the CPU. If top reports that your program is taking 45% CPU, 45% of the samples taken by top found your process active on the CPU. The rest of the time your application was in a wait. (It is important to remember that a CPU is a discrete state machine. It really can be at only 100%, executing an instruction, or at 0%, waiting for something to do. There is no such thing as using 45% of a CPU. The CPU percentage is a function of time.) However, it is likely that your application's rest periods include waiting to be dispatched on a CPU and not on external devices. That part of the wait percentage is then very relevant to understanding your overall CPU usage pattern.

The load averages differ from CPU percentage in two significant ways: 1) load averages measure the trend in CPU utilization not only an instantaneous snapshot, as does percentage, and 2) load averages include all demand for the CPU not only how much was active at the time of measurement.

Authors tend to overuse analogies and sometimes run the risk of either insulting the reader's intelligence or oversimplifying the topic to the point of losing important details. However, freeway traffic patterns are a perfect analogy for this topic, because this model encapsulates the essence of resource contention and is also the chosen metaphor by many authors of queuing theory books. Not surprisingly, CPU contention is a queuing theory problem, and the concepts of arrival rates, Poisson theory and service rates all apply. A four-processor machine can be visualized as a four-lane freeway. Each lane provides the path on which instructions can execute. A vehicle can represent those instructions. Additionally, there are vehicles on the entrance lanes ready to travel down the freeway, and the four lanes either are ready to accommodate that demand or they're not. If all freeway lanes are jammed, the cars entering have to wait for an opening. If we now apply the CPU percentage and CPU load-average measurements to this situation, percentage examines the relative amount of time each vehicle was found occupying a freeway lane, which inherently ignores the pent-up demand for the freeway—that is, the cars lined up on the entrances. So, for example, vehicle license XYZ 123 was found on the freeway 30% of the sampling time. Vehicle license ABC 987 was found on the freeway 14% of the time. That gives a picture of how each vehicle is utilizing the freeway, but it does not indicate demand for the freeway.

Moreover, the percentage of time these vehicles are found on the freeway tells us nothing about the overall traffic pattern except, perhaps, that they are taking longer to get to their destination than they would like. Thus, we probably would suspect some sort of a jam, but the CPU percentage would not tell us for sure. The load averages, on the other hand, would.

This brings us to the point. It is the overall traffic pattern of the freeway itself that gives us the best picture of the traffic situation, not merely how often cars are found occupying lanes. The load average gives us that view because it includes the cars that are queuing up to get on the freeway. It could be the case that it is a nonrush-hour time of day, and there is little demand for the freeway, but there just happens to be a lot of cars on the road. The CPU percentage shows us how much the cars are using the freeway, but the load averages show us the whole picture, including pent-up demand. Even more interesting, the more recent that pent-up demand is, the more the load-average value reflects it.

Taking the discussion back to the machinery at hand, the load averages tell us by increasing duration whether our physical CPUs are over- or under-utilized. The point of perfect utilization, meaning that the CPUs are always busy and, yet, no process ever waits for one, is the average matching the number of CPUs. If there are four CPUs on a machine and the reported one-minute load average is 4.00, the machine has been utilizing its processors perfectly for the last 60 seconds. This understanding can be extrapolated to the 5- and 15-minute averages.

In general, the intuitive idea of load averages is the higher they rise above the number of processors, the more demand there is for the CPUs, and the lower they fall below the number of processors, the more untapped CPU capacity there is. But all is not as it appears.
The Wizard behind the Curtain

The load-average calculation is best thought of as a moving average of processes in Linux's run queue marked running or uninterruptible. The words “thought of” were chosen for a reason: that is how the measurements are meant to be interpreted, but not exactly what happens behind the curtain. It is at this juncture in our journey when the reality of it all, like quantum mechanics, seems not to fit the intuitive way as it presents itself.

***** Tasks
*State of a process*:
-  Runnable (R) :: A process in this state is either executing on the CPU, or it is present on the run queue, ready to be executed.
-  Interruptible sleep (S) :: Processes in this state are waiting for an event to complete.
-  Uninterruptible sleep (D) :: In this case, a process is waiting for an I/O operation to complete.
-  Stopped (T) :: These processes have been stopped by a job control signal (such as by pressing Ctrl+Z) or because they are being traced.
-  Zombie (Z) :: The kernel maintains various data structures in memory to keep track of processes. A process may create a number of child processes, and they may exit while the parent is still around. However, these data structures must be kept around until the parent obtains the status of the child processes. Such terminated processes whose data structures are still around are called zombies.

Processes in the D and S states are shown in “sleeping”, and those in the T state are shown in “stopped”. The number of zombies are shown as the “zombie” value.

***** CPU usage
The CPU usage section shows the percentage of CPU time spent on various tasks. The ~us~ value is the time the CPU spends executing processes in userspace. Similarly, the ~sy~ value is the time spent on running kernelspace processes.

Linux uses a “nice” value to determine the priority of a process. A process with a high “nice” value is “nicer” to other processes, and gets a low priority. Similarly, processes with a lower “nice” gets higher priority. The default “nice” value can be changed. The time spent on executing processes with a manually set “nice” appear as the ~ni~ value.

This is followed by ~id~, which is the time the CPU remains idle. Most operating systems put the CPU on a power saving mode when it is idle. Next comes the ~wa~ value, which is the time the CPU spends waiting for I/O to complete.

Interrupts are signals to the processor about an event that requires immediate attention. Hardware interrupts are typically used by peripherals to tell the system about events, such as a keypress on a keyboard. On the other hand, software interrupts are generated due to specific instructions executed on the processor. In either case, the OS handles them, and the time spent on handling hardware and software interrupts are given by ~hi~ and ~si~ respectively.

In a virtualized environment, a part of the CPU resources are given to each virtual machine (VM). The OS detects when it has work to do, but it cannot perform them because the CPU is busy on some other VM. The amount of time lost in this way is the “steal” time, shown as ~st~.

***** Memory usage
The ~avail Mem~ value is the amount of memory that can be allocated to processes without causing more swapping.

The Linux kernel also tries to reduce disk access times in various ways. It maintains a “disk cache” in RAM, where frequently used regions of the disk are stored. In addition, disk writes are stored to a “disk buffer”, and the kernel eventually writes them out to the disk. The total memory consumed by them is the ~buff/cache~ value.

**** Task Area
***** PR and NI
The ~NI~ field shows the “nice” value of a process. The ~PR~ field shows the scheduling priority of the process from the perspective of the kernel. The nice value affects the priority of a process.

The PR (priority) column lists the kernel’s current schedule priority for the process. The higher the number, the less likely the kernel is to schedule the process if others need CPU time. The schedule priority alone doesn’t determine the kernel’s decision to give CPU time to a process, however, and the kernel may also change the priority during program execution according to the amount of CPU time the process consumes.

The priority column is the NI (nice value) column, which gives a hint to the kernel’s scheduler. This is what you care about when trying to influence the kernel’s decision. The kernel adds the nice value to the current priority to determine the next time slot for the process. When you set the nice value higher, you’re being “nicer” to other processes because the kernel prioritizes them.

By default, the nice value is 0. Now, say you’re running a big computation in the background that you don’t want to bog down your interactive session. To make that process take a back seat to other processes and run only when the other tasks have nothing to do, you can change the nice value to 20 with the ~renice~ command (where pid is the process ID of the process that you want to change)

***** VIRT, RES, SHR and %MEM
These three fields are related with to memory consumption of the processes. ~VIRT~ (Virtual memory used) is the total amount of memory consumed by a process. This includes the program’s code, the data stored by the process in memory, as well as any regions of memory that have been swapped to the disk. ~RES~ (Resident memory) (that's more or less how much memory a program needs) is the memory consumed by the process in RAM, and ~%MEM~ (The share of physical memory) expresses this value as a percentage of the total RAM available. Finally, ~SHR~ (Shared Memory) is the amount of memory shared with other processes.

***** S
As we have seen before, a process may be in various states. This field shows the process state in the single-letter form.

***** TIME+
This is the total CPU time used by the process since it started, precise to the hundredths of a second.

**** Manage
***** Killing processes
If you want to kill a process, simply press ~k~ when top is running. This will bring up a prompt, which will ask for the process ID of the process and press enter.

Next, enter the signal using which the process should be killed. If you leave this blank, top uses a ~SIGTERM~, which allows processes to terminate gracefully. If you want to kill a process forcefully, you can type in ~SIGKILL~ here. You can also type in the signal number here. For example, the number for ~SIGTERM~ is ~15~ and ~SIGKILL~ is ~9~.

***** Sorting the process list
One of the most frequent reasons to use a tool like top is to find out which process is consuming the most resources. You can press the following keys to sort the list:

-  ~M~ to sort by memory usage
-  ~P~ to sort by CPU usage
-  ~N~ to sort by process ID
-  ~T~ to sort by the running time

**** Example

- 14:59:20 :: the current time of day.
- up 6:30 :: uptime. It is the amount of time since the machine was last booted.
- load average: :: refers to the number of processes that are waiting to run, that is, the number of processes that are in a runnable state and are sharing the cpu. Values less than 1.0 indicate that the machine is not busy.
- Cpu(s) :: this row describes the character of the activities that the CPU is performing.
- 0.7%us :: 0.7 percent of the CPU is being used for user processes. This means processes outside the kernel.
- 1.0%sy :: 1.0 percent of the CPU is being used for system (kernel) processes.
- 0.0%ni :: 0.0 percent of the CPU is being used by “nice” (low-priority) processes.
- 98.3%id :: 98.3 percent of the CPU is idle.
- 0.0%wa :: 0.0 percent of the CPU is waiting for I/O.

#+begin_src sh

  top - 14:59:20 up 6:30, 2 users, load average: 0.07, 0.02, 0.00
  Tasks:  109 total, 1 running, 106 sleeping, 0 stopped, 2 zombie
  Cpu(s): 0.7%us, 1.0%sy, 0.0%ni, 98.3%id, 0.0%wa, 0.0%hi, 0.0%si
  Mem:    319496k total, 314860k used,   4636k free,  19392k buff
  Swap:   875500k total, 149128k used, 726372k free, 114676k cach

  PID   USER  PR NI  VIRT  RES  SHR S %CPU %MEM   TIME+  COMMAND
  6244  me    39 19 31752 3124 2188 S  6.3  1.0 16:24.42 trackerd
  11071 me    20  0  2304 1092  840 R  1.3  0.3 00:00.14 top
  6180  me    20  0  2700 1100  772 S  0.7  0.3 00:03.66 dbus-dae
  6321  me    20  0 20944 7248 6560 S  0.7  2.3 02:51.38 multiloa
  4955  root  20  0 104m  9668 5776 S  0.3  3.0 02:19.39 Xorg
  1     root  20  0 2976   528  476 S  0 0  0.2 00:03.14 init
  2     root  15 -5    0     0    0 S  0.0  0.0 00:00.00 kthreadd
  3     root  RT -5    0     0    0 S  0.0  0.0 00:00.00 migratio
  4     root  15 -5    0     0    0 S  0.0  0.0 00:00.72 ksoftirq
  5     root  RT -5    0     0    0 S  0.0  0.0 00:00.04 watchdog
  6     root  15 -5    0     0    0 S  0.0  0.0 00:00.42 events/0
  7     root  15 -5    0     0    0 S  0.0  0.0 00:00.06 khelper
  41    root  15 -5    0     0    0 S  0.0  0.0 00:01.08 kblockd/
  67    root  15 -5    0     0    0 S  0.0  0.0 00:00.00 kseriod
  114   root  20  0    0     0    0 S  0.0  0.0 00:01.62 pdflush
  116   root  15 -5    0     0    0 S  0.0  0.0 00:02.44 kswapd0

#+end_src

*** ps
The STAT column is short for “state” and reveals the current status of the process.

- R (running) :: this means that the process is running or ready to run.
- S (sleeping) :: the process is not running; rather, it is waiting for an event, such as a keystroke or network packet.
- D (uninterruptible sleep) :: the process is waiting for I/O such as a disk drive.
- T (stopped) :: the process has been instructed to stop. More on this later in the chapter.
- Z (a defunct or “zombie” process) :: this is a child process that has terminated but has not been cleaned up by its parent.
- < (a high-priority process) :: it's possible to grant more importance to a process, giving it more time on the CPU. This property of a process is called niceness. A process with high priority is said to be less nice because it's taking more of the CPU's time, which leaves less for everybody else.
- N (a low-priority process) :: a process with low priority (a “nice” process) will get processor time only after other processes with higher priority have been serviced.

*** find
**** Overview
GNU ~find~ searches the directory tree rooted at each given starting-point by evaluating the given expression from left to right, according to the rules of precedence, until the outcome is known (the left hand side is false for ~and~ operations, true for ~or~), at which point find moves on to the next file name.  If no starting-point is specified, ( ~.~ ) is assumed.

*Informal syntax*: ~find dir [ [atttributes logical-relationship] ... ] action~

~find~ allows actions to be performed based on the search results. There are a set of predefined actions and several ways to apply user-defined actions.

By default, ~find~ evaluates from left to right. That is, the logical relationship between the tests and actions determines which of them are performed, so the order of the tests and actions is important.

In the ~find~ command, there is, by default, an implied ~-and~ relationship between each test and action (that is, when no operator is present, ~-and~ is implied by default).

The operators ~-and~, ~-or~ and ~-not~ can be abbreviated by ~-a~, ~-o~ and ~!~.

#+begin_src sh

  # These commands are the same.
  find . -type f -name '*.bak' -print

  find . -type f -a -name '*.bak' -a -print

  find . -type f -and -name '*.bak' -and -print

#+end_src

*Warning*: use extreme caution when using the ~-delete~ action. Always test the command first by substituting the ~-print~ action for ~-delete~ to confirm the search results.

*Dealing with Funny Filenames (-print0)*:
Unix-like systems allow embedded spaces (and even newlines!) in filenames. This causes problems for programs like ~xargs~ that construct argument lists for other programs. An embedded space will be treated as a delimiter, and the resulting command will interpret each space-separated word as a separate argument. To overcome this, ~find~ and ~xargs~ allow the optional use of a null character as an argument separator. The ~find~ command provides the action ~-print0~, which produces null-separated output, and the ~xargs~ command has the ~--null~ (or ~-0~) option, which accepts null separated input.

#+begin_src sh

  find . -iname '*.jpg' -print0 | xargs --null ls -l
  # Using this technique, we can ensure that all files, even those
  # containing embedded spaces in their names, are handled correctly.

#+end_src

*Note*: ~find~ does not produce results in sorted order. Its order is determined by the layout of the storage device.

**** Tests
Tests return a true or false value, usually on the basis of some property of a file we are considering. Adding the test ~-type~ limited the search to directories.

#+begin_src sh

  # List only directories.
  find . -type d

#+end_src

| /Type/ | /Description/                   |
|------+-------------------------------|
| /b/    | Block special device file     |
| /c/    | Character special device file |
| /d/    | Directory                     |
| /f/    | Regular file                  |
| /l/    | Symbolic link                 |
| /s/    | Socket                        |

**** Test size
We can also search by file size.

We add the ~-size~ test followed by the string ~+1M~. The leading plus sign ( ~+~ )  indicates that we are looking for files larger than the specified number. A leading minus ( ~-~ )  sign would change the meaning of the string to be smaller than the specified number. Using no sign means, “match the value exactly.” The trailing letter ~M~ indicates that the unit of measurement is megabytes.

#+begin_src sh

  # Match the wildcard *.jpg and are larger than one megabyte.
  find . -type f -name "*.jpg" -size +1M

#+end_src

| /Char/ | /Unit/                                  |
|------+---------------------------------------|
| /b/    | 512-byte blocks (default)             |
| /c/    | Bytes                                 |
| /w/    | 2-byte words                          |
| /k/    | Kilobytes (units of 1024 bytes)       |
| /M/    | Megabytes (units of 1048576 bytes)    |
| /G/    | Gigabytes (units of 1073741824 bytes) |

**** Table test
The cases where a numeric argument is required, the ( ~+~ ) mean "files larger than the specified number" and ( ~-~ ) "files smaller than the specified number" and no sign means, "match the value exactly".

| /Test/           | /Match/                                               |
|----------------+-----------------------------------------------------|
| /-empty/         | empty fs and ds                                     |
| /-type c/        | fs of type c                                        |
| /-size n/        | fs of size n                                        |
| /-name pattern/  | fs a ds with the specified wildcard pattern         |
| /-iname pattern/ | like the -name test but case-insensitive            |
| /-perm mode/     | fs o ds that have permissions set to specified mode |
| /-cmin n/        | fs o ds whose conts o atts last mod exactly n min   |
| /-cnewer file/   | fs o ds whose conts o atts last mod more recently   |
| /-ctime n/       | fs o ds whose conts o atts last mod n*24 hours      |
| /-mmin n/        | fs o ds whose conts were last mod n minutes ago     |
| /-mtime n/       | fs o ds whose conts were last mod n*24 hours ago    |
| /-newer file/    | fs a ds whose conts were mod more recently than f * |
| /-inum n/        | f with inode number n (find hardlinks to inode)     |
| /-samefile name/ | similar -inum. fs share the same inode as file name |
| /-user name/     | fs o ds belonging to user name                      |
| /-nouser/        | f a ds that do not belong to a valid user *         |
| /-group name/    | f o ds belonging to group.                          |
| /-nogroup/       | fs a ds that do not belong to a valid group         |

| /Abbreviation/ | /Name/                   |
|--------------+------------------------|
| /fs/           | files                  |
| /fs o ds/      | files or directories   |
| /f o ds/       | file or directories    |
| /fs a ds/      | files and directories  |
| /f a ds/       | file and directories   |
| /conts/        | contents               |
| /conts o atts/ | contents or attributes |
| /mod/          | modified               |
| /last mod/     | were last modified     |

*Notes*: *
- This ~-newer file~ is useful when writing shell scripts that perform file backups. Each time you make a backup, update a file (such as a log) and then use find to determine which files have changed since the last update.
- This ~-nouser~ can be used to find files belonging to deleted accounts or to detect activity by attackers.

**** Operators
~find~ provides a way to combine tests using logical operators to create more complex logical relationships.

#+begin_src sh

  # Looking for all the files with permissions that are not 0600 and the
  # directories with permissions that are not 0700.
  find . \( -type f -not -perm 0600 \) -o \( -type d -not -perm 0700 \)

#+end_src

| /Opt/  | /Match/                                                |
|------+------------------------------------------------------|
| /-and/ | if the tests on both sides of the operator are true  |
| /-or/  | if a test on either side of the operator is true     |
| /-not/ | if the test following the operator is false          |
| /( )/  | Groups tests and operators to form larger expression |

~( )~ This is used to control the precedence of the logical evaluations. By default, ~find~ evaluates from left to right. It is often necessary to override the default evaluation order to obtain the desired result. Even if not needed, it is helpful sometimes to include the grouping characters to improve the readability of the command. Note that since the parentheses have special meaning to the shell, they must be quoted when using them on the command line to allow them to be passed as arguments to find. Usually the backslash character is used to escape them.

**** Predefined Actions
~find~ allows actions to be performed based on the search results. There are a set of predefined actions and several ways to apply user-defined actions.

#+begin_src sh

  # Every file in dir and its subdirectories is searched and when they are
  # found, they are deleted.
  find . -type f -name '*.bak' -delete

  # Could express this way to make the logical relationships easier to see.
  find . -type f -a -name '*.bak' -a -print

#+end_src

*Warning*: Extreme caution when using the ~-delete~ action. Always test the command first by substituting the ~-print~ action for ~-delete~ to confirm the search results.

| /Action/  | /Description/                                      |
|---------+--------------------------------------------------|
| /-delete/ | delete the currently matching file               |
| /-ls/     | perform the equivalent of ls -dils on match file |
| /-print/  | output the full path of the matc file to stdout  |
| /-quit/   | quit once a match has been made                  |

**** User-Defined Actions
We can also invoke arbitrary commands. The traditional way of doing this is with the ~-exec~ action. This action works like this:

~-exec command {} ;~

Here *command* is the name of a command, ~{}~ is a symbolic representation of the current pathname, and the semicolon ( ~;~ ) is a required delimiter indicating the end of the command. Note that the brace and semicolon characters have special meaning to the shell, so they must be quoted or escaped.

#+begin_src sh

  # In this example -exec act like the -delete action.
  -exec rm '{}' ';'


  # It’s possible to execute a user-defined action interactively. By using
  # the -ok action in place of -exec
  find . -type f -name 'foo*' -ok ls -l '{}' ';'
  # < ls ... /home/me/bin/foo > ? y
  # -rwxr-xr-x 1 me me 224 2007-10-29 18:44 /home/me/bin/foo
  # < ls ... /home/me/foo.txt > ? y
  # -rw-r--r-- 1 me me 0 2016-09-19 12:53 /home/me/foo.txt

#+end_src

*Improving Efficiency*:
When the ~-exec~ action is used, it launches a new instance of the specified command each time a matching file is found. There are times when we might prefer to combine all of the search results and launch a single instance of the command.

By changing the trailing semicolon character to a plus sign, we activate the ability of ~find~ to combine the results of the search into an argument list for a single execution of the desired command.

#+begin_src sh

  # This will execute ls each time a matching file is found.
  find . -type f -name 'foo*' -exec ls -l '{}' ';'
  # Lower performance since it launches a new instance of the specified
  # command each time a matching file is found.

  # Same results, but the system has to execute the ls command only once.
  find . -type f -name 'foo*' -exec ls -l '{}' +
  # Better performace since combine the results of the search into an
  # argument list for a single execution of the command.

  # Another approach with xargs command.
  find . -type f -name 'foo*' -print | xargs ls -l
  # The output of the find command piped into xargs, which, in turn,
  # constructs an argument list for the ls command and then executes it.

#+end_src

*Note*: While the number of arguments that can be placed into a command line is quite large, it’s not unlimited. It is possible to create commands that are too long for the shell to accept. When a command line exceeds the maximum length supported by the system, ~xargs~ executes the specified command with the maximum number of arguments possible and then repeats this process until standard input is exhausted. To see the maximum size of the command line, execute ~xargs~ with the ~--show-limits~ option.

**** Regexp
There is an important consideration to keep in mind when using regular expressions in ~find~ versus ~grep~. Whereas ~grep~ will print a line when the line *contains* a string that matches an expression, ~find~ requires that the pathname *exactly match* the regular expression.

#+begin_src sh

  # Find every pathname that contains any character that is not a member
  # of the set: [-_./0-9a-zA-Z]
  # Such a scan would reveal pathnames that contain embedded spaces and
  # other potentially offensive characters.
  find . -regex '.*[^-_./0-9a-zA-Z].*'

#+end_src

**** Options
We have the options. The options are used to control the scope of a ~find~ search.  They may be included with other tests and actions when constructing ~find~ expressions.

| /Option/    | /Description/                                        |
|-----------+----------------------------------------------------|
| /-depth/    | Direct find to process a directory’s files before  |
|           | dir itself (default when the -delete is specified) |
| /-maxdepth/ | Set the max of levels that find will descend into  |
|           | a directory tree when performing tests and actions |
| /-mindepth/ | Set the min of levels that find will descend into  |
|           | a directory tree before applying tests and actions |
| /-mount/    | Direct find not to traverse directories that are   |
|           | mounted on other file systems                      |

*** du
~du~ vs /gnome apps/

|              | /du/                           | /gnome apps/      |
|--------------+------------------------------+-----------------|
| /size/         | device usage/--apparent-size | --apparent-size |
| /empty folder/ | 4.0K / 4.1K                  | 4.0K            |
| /display */    | inside folder and folder     | inside folder   |

***: Example of how ~du~ and some /gnome app/ displays the results of a folder.

Some /gnome apps/ display the total size of the contents *inside* the folder, while ~du~ does the same, but also display the size of the "*block*" of the *folder itself* (for folders with a lot of content this data would be negligible).

#+begin_src sh

  folder
  +---------------+
  | folder-inside |
  | +-----------+ |
  | |           | |
  | +-----------+ |
  +---------------+

  mkdir -p folder/folder-inside

  cd folder; du                    cd folder; du --apparent-size --si
  # 4   ./folder-inside            # 4.1k   ./folder-inside
  # 8   .                          # 8.2k   .

  gnome apps
  # folder
  # 4.1Kb, 1 item

#+end_src

*** sort
On computers, dates are usually formatted in ~YYYY-MM-DD~ order to make chronological sorting easy, but ours are in the American format of ~MM/DD/YYYY~. How can we sort this list in chronological order?

Fortunately, sort provides a way. The key option allows specification of offsets within fields, so we can define keys within fields.

#+begin_src sh

  sort -k 3.7nbr -k 3.1nbr -k 3.4nbr file_dates
  # Fedora     10      11/25/2008
  # Ubuntu     8.10    10/30/2008
  # SUSE       11.0    06/19/2008
  # By specifying -k 3.7, we instruct sort to use a sort key that begins
  # at the seventh character within the third field, which corresponds
  # to the start of the year. Likewise, we specify -k 3.1 and -k 3.4 to
  # isolate the month and day portions of the date. We also add the n
  # and r options to achieve a reverse numeric sort. The b option is
  # included to suppress the leading spaces (whose numbers vary from
  # line to line, thereby affecting the outcome of the sort) in the date
  # field.

  sort --key=1,1 --key=2n file_dates
  # Fedora       5       03/20/2006
  # Fedora       6       10/24/2006
  # Fedora       7       05/31/2007
  # Though we used the long form of the option for clarity, -k 1,1 -k 2n
  # would be exactly equivalent. In the first instance of the key
  # option, we specified a range of fields to include in the first
  # key. Since we wanted to limit the sort to just the first field, we
  # specified 1,1 which means “start at field 1 and end at field 1.” In
  # the second instance, we specified 2n, which means field 2 is the
  # sort key and that the sort should be numeric. An option letter may
  # be included at the end of a key specifier to indicate the type of
  # sort to be performed.

  sort -t ':' -k 7 /etc/passwd
  # me:x:1001:1001:Myself,,,:/home/me:/bin/bash
  # root:x:0:0:root:/root:/bin/bash
  # dhcp:x:101:102::/nonexistent:/bin/false
  # By specifying the colon character as the field separator, we can sort
  # on the seventh field.

#+end_src

*** xargs
~xargs~ reads items from the standard input, delimited by blanks (which can be protected with double or single quotes or a backslash) or newlines, and executes the command (default is /bin/echo) one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored.

That is, ~xargs~ accepts input from standard input and converts it into an argument list for a specified command.

#+begin_src sh

  # These command are the same.
  find . -type f -name 'foo*' -print | xargs ls -l

  find . -type f -name 'foo*' -exec ls -l '{}' ';'
  # Lower performance since it launches a new instance of the specified
  # command each time a matching file is found.

  find . -type f -name 'foo*' -exec ls -l '{}' +
  # Better performace since combine the results of the search into an
  # argument list for a single execution of the command.

#+end_src

*Dealing with Funny Filenames (-print0)*:
Unix-like systems allow embedded spaces (and even newlines!) in filenames. This causes problems for programs like ~xargs~ that construct argument lists for other programs. An embedded space will be treated as a delimiter, and the resulting command will interpret each space-separated word as a separate argument. To overcome this, ~find~ and ~xargs~ allow the optional use of a null character as an argument separator. The ~find~ command provides the action ~-print0~, which produces null-separated output, and the ~xargs~ command has the ~--null~ (or ~-0~) option, which accepts null separated input.

#+begin_src sh

  find . -iname '*.jpg' -print0 | xargs --null ls -l
  # Using this technique, we can ensure that all files, even those
  # containing embedded spaces in their names, are handled correctly.

#+end_src

*Note*: While the number of arguments that can be placed into a command line is quite large, it’s not unlimited. It is possible to create commands that are too long for the shell to accept. When a command line exceeds the maximum length supported by the system, ~xargs~ executes the specified command with the maximum number of arguments possible and then repeats this process until standard input is exhausted. To see the maximum size of the command line, execute ~xargs~ with the ~--show-limits~ option.

*** diff
**** Default format
In the default format, each group of changes is preceded by a *change command* in the form of *range operation range* to describe the *positions and types* of changes required to *convert the first file to the second file*.

In this format, a range is a comma-separated list of the starting line and the ending line.

| /Change/ | /Description/                                              |
|--------+----------------------------------------------------------|
| /r1ar2/  | Append the lines at the position r2 in the second        |
|        | file to the position r1 in the first file                |
| /r1cr2/  | Change (replace) the lines at position r1 with the       |
|        | lines at the position r2 in the second file              |
| /r1dr2/  | Delete the lines in the first file at position r1, which |
|        | would have appeared at range r2 in the second file       |

#+begin_src sh

  cat file1   cat file2
  # a         b
  # b         d
  # c         d
  # d         e

  diff file1 file2
  # 1d0
  # < a # < Means delete that line.
  # 4a4
  # > e # > Means add that line.

#+end_src

**** Context format
The unified format is specified with the ~-c~ option.

| /Indic/ | /Meaning/                                                |
|-------+--------------------------------------------------------|
| /blank/ | A line shown for context. It doesn't indicate          |
|       | a difference between the two files                     |
| /-/     | A line deleted. This line will appear in the           |
|       | first file but not in the second file                  |
| /+/     | A line added. This line will appear in the             |
|       | second file but not in the first file                  |
| /!/     | A line changed.The 2 version of line will be displayed |
|       | , each in its respective section of change group       |

#+begin_src sh

  cat file1   cat file2
  # a         b
  # b         d
  # c         d
  # d         e

  diff -c file1 file2
  # *** file1.txt 2008-12-23 06:40:13.000000000 -0500
  # --- file2.txt 2008-12-23 06:40:34.000000000 -0500
  # ***************
  # *** 1,4 ****  # lines 1 through 4 in the first file
  # - a
  #   b
  #   c
  #   d
  # --- 1,4 ----  # lines 1 through 4 in the second file
  #   b
  #   c
  #   d
  # + e

#+end_src

**** Unified format
The unified format is specified with the ~-u~ option.

~@@ -1,4 +1,4 @@~ This indicates the lines in the first file and the lines in the second file described in the change group. Following this are the lines themselves, with the default three lines of context. Each line starts with one of three possible characters listed below.

| /Char/  | /Meaning/                                   |
|-------+-------------------------------------------|
| /blank/ | this line is shared by both files         |
| /-/     | this line was removed from the first file |
| /+/     | this line was added to the first file     |

#+begin_src sh

  cat file1   cat file2
  # a         b
  # b         d
  # c         d
  # d         e

  diff -c file1 file2
  # --- file1.txt 2008-12-23 06:40:13.000000000 -0500
  # +++ file2.txt 2008-12-23 06:40:34.000000000 -0500
  # @@ -1,4 +1,4 @@
  # -a
  #  b
  #  c
  #  d
  # +e

#+end_src

**** Example

#+begin_src sh

  cat a.txt   cat b.txt
  one         one
  two         two
  three       THREE
  four        FOUR
  five        FIVE
  six         six
  7           10
  8           9
  9           8
  10          7
  eleven      eleven
  twelve      twelve
  13          0
  14          13
  15          14
  16          15
  16

  # diff show how to change the first file to match the second.
  diff -u a.txt b.txt
  --- a.txt	2023-09-28 12:32:42.998667401 -0400 # --- first file.
  +++ b.txt	2023-09-28 12:32:41.958654590 -0400 # +++ second file.
  @@ -1,15 +1,16 @@
  # -1,15 show line 1 and the following 15 lines.
  # +1,16 show line 1 and the following 16 lines.
  # Show 3 lines before and after the change.
  # - deleted lines + add lines.
  # whitespace at the begining indicates the line doesn't change.
  one
  two
  -three
  -four
  -five
  +THREE
  +FOUR
  +FIVE
  six
  -7
  -8
  -9
  10
  +9
  +8
  +7
  eleven
  twelve
  +0
  13
  14
  15

#+end_src

*** sed
In general, sed takes an address and an operation as one argument. The address is a set of lines (every line by default), and the command determines what to do with the lines. With no file arguments, sed reads from the standard input.

| /Address/     | /Description/                                     |
|-------------+-------------------------------------------------|
| /n/           | A line number where n is a positive integer     |
| /$/           | The last line                                   |
| /regexp/      | Lines matching a POSIX basic regexp             |
| /addr1,addr2/ | A range of lines from addr1 to addr2, inclusive |
| /addr1,+n/    | Match addr1 and the following n lines           |
| /addr!/       | Match all lines except addr                     |
| /first~step/  | Match the line represented by the number first, |
|             | then each subsequent line at step intervals     |

| /Command/     | /Description/                                      |
|--------------+--------------------------------------------------|
| /=/            | Output the current line number                   |
| /a/            | Append text after the current line               |
| /i/            | Insert text in front of the current line         |
| /d/            | Delete the current line                          |
| /p/            | Print the current line                           |
| /q/            | Exit sed without processing any more lines       |
| /Q/            | Exit sed without processing any more lines       |
| /s/regex/rep// | Sub cont of replacement wherever regexp is found |
| /y/set1/set2/  | Perform transliteration by converting characters |
|              | from set1 to the corresponding charactrs in set2 |

- /p/ By default, ~sed~ prints every line and only edits lines that match a specified address within the file. The default behavior can be overridden by specifying the ~-n~ option.
- *replacement* may include the special character ~&~, which is equivalent to the text matched by regexp. In addition, replacement may include the sequences ~\1~ through ~\9~, which are the contents of the corresponding subexpressions in regexp. After the trailing slash following replacement, an optional flag may be specified to modify the s command’s behavior.

*Notes*:
- when any command has extra slashes in its regular expression, this would confuse sed when it tries to interpret the command, so it must be applied the backslashes to escape the offending characters.

#+begin_src sh

  sed 's/([0-9]{2})/([0-9]{4})$\2-\1/' file
  # Application of backslashes to escape the offending characters.
  sed 's/\([0-9]\{2\}\)\/\([0-9]\{4\}\)$/\2-\1/' file

#+end_src

- ~sed~ does not support character ranges (for example, ~[a-z]~), nor does it support POSIX character classes.

*** evtest
Utility to monitor Linux input device events

evtest monitors a Linux input device, displaying all the events it generates.

It can be used to determine mice button bindings, keymaps for exotic keyboards... (keys like Ctrl, AltGr, Caps lock, ....)

It is commonly used to debug issues with input devices in X.Org. (also wayland)

*Note*: another good program is ~xev~, but the difference is that ~xev~ operates at the X Window System level and is focused on GUI events (shows events after X server processing), and ~evtest~ operates at the kernel level and deals with raw input events (before any system processing) from devices. If you remap =Caps Lock= to =Menu=, then ~xev~ will display =Menu=, while ~evtest~ will display =Caps Lock=.

The following command scans all input events, then you can enter a number to test a specific input device:

#+begin_src sh

  evtest

#+end_src

Running ~xev~ immediately captures any input device.

** Network
*** rsync
Some tips on *how to exclude patterns*:

- You can have as many ~--exclude~ parameters as you like.
- If you use the same patterns repeatedly, place them in a plaintext file (one pattern per line) and use ~--exclude-from=file~.
- To exclude directories named *item* but include files with this name, use a trailing slash: ~--exclude=item/~.
- The exclude pattern is based on a full file or directory name component and may contain simple globs (wildcards). For example, ~t*s~ matches ~this~, but it does not match ~ethers~.
- If you exclude a directory or filename but find that your pattern is too restrictive, use ~--include~ to specifically include another file or directory.

*** curl
curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GO‐ PHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.

#+begin_src sh

  curl --trace-ascii trace_file http://www.example.org/

  # curl attempts to establish the TCP connection to the server (transport
  # layer orbelow).
  == Info: Trying 93.184.216.34...
  == Info: TCP_NODELAY set
  == Info: Connected to www.example.org (93.184.216.34) port 80 (#0)

  # If connection succeeds, curl then tries to send the request (header);
  # this is where the application layer starts.
  => Send header, 79 bytes (0x4f) # curl debugging output.
  # Next lines show what curl sends to the server.
  # The hexadecimal numbers adds help to keep track of how much data was
  # sent or received.
  0000: GET / HTTP/1.1  # Requests the target resource.
  0010: Host: www.example.org
  0027: User-Agent: curl/7.58.0
  0040: Accept: */*
  004d:                # empty line

  # Next, the server sends a reply, first with its own header.
  <= Recv header, 17 bytes (0x11)  # debugging output.
  # In curl , the header won’t count toward the offset; that’s why all of
  # these lines begin with 0000.
  0000: HTTP/1.1 200 OK
  <= Recv header, 22 bytes (0x16)
  0000: Accept-Ranges: bytes
  <= Recv header, 12 bytes (0xc)
  0000: Age: 17629

  # Headers and the actual requested document.
  <= Recv header, 22 bytes (0x16)
  0000: Content-Length: 1256
  <= Recv header, 2 bytes (0x2)
  0000:                         # blank line means end of headers HTTP
  <= Recv data, 1256 bytes (0x4e8)
  0000: <!doctype html>.<html>.<head>. <title>Example Domain</title>.
  0040: . <meta charset="utf-8" />. <meta http-equiv="Content-type

#+end_src

*** iptables
Administration tool for IPv4/IPv6 packet filtering and NAT.

Each firewall chain has a default policy that specifies what to do with a packet if no rule matches the packet.
~ACCEPT~ -> kernel allows the packet to pass through the packet-filtering system.
~DROP~ -> tells the kernel to discard the packet.
The kernel reads the chain from top to bottom, using the first rule that matches. When a rule matches, the kernel carries out the action and looks no further down in the chain.

#+begin_src sh

  Chain INPUT (policy ACCEPT)
  target      prot opt source        destination

  Chain FORWARD (policy ACCEPT)
  target      prot opt source        destination

  Chain OUTPUT (policy ACCEPT)
  target      prot opt source         destination

#+end_src

~# iptables -A INPUT -s 192.168.34.63 -j DROP~
Drop packets from /192.168.34.63/ host.

#+begin_src sh

  # If someone at 192.168.34.63 is annoying you this command prevent them
  # from talking to machine.
  iptables -A INPUT -s 192.168.34.63 -j DROP
  # Chain INPUT (policy ACCEPT)
  # target prot opt source           destination
  # DROP   all  --  192.168.34.63    anywhere

  # The 192.168.34.63 has told everyone on his subnet to open connections
  # to your SMTP port (TCP port 25). To get rid of that traffic, run:
  iptables -A INPUT -s 192.168.34.0/24 -p tcp --destination-port 25 -j DROP
  # Chain INPUT (policy ACCEPT)
  # target prot opt source           destination
  # DROP   all  --  192.168.34.63    anywhere
  # DROP   tcp  --  192.168.34.0/24  anywhere       tcp dpt:smtp

  # Someone at 192.168.34.37 saying that she can’t send you email because
  # you blockedher machine. Insert that rule at the top of the chain:
  iptables -I INPUT -s 192.168.34.37 -j ACCEPT

  # Insert a rule in 4 position:
  iptables -I INPUT 4 -s 192.168.34.37 -j ACCEPT

#+end_src

*** tcpdump
**** TCP Packets
The general format of a tcp protocol line is:

#+begin_src sh

  src > dst: flags data-seqno ack window urgent options

#+end_src

Src and dst are the source and destination IP addresses and ports. Flags are some combination of S (SYN), F (FIN), P (PUSH) or R (RST) or a single `.' (no flags). Data-seqno describes the portion of sequence space covered by the data in this packet (see example below). Ack is sequence number of the next data expected the other direction on this connection. Window is the number of bytes of receive buffer space available the other direction on this connection. Urg indicates there is `urgent' data in the packet. Options are tcp options enclosed in angle brackets (e.g., <mss 1024>).

Src, dst and flags are always present. The other fields depend on the contents of the packet's tcp protocol header and are output only if appropriate.

Here is the opening portion of an rlogin from host rtsg to host csam.

#+begin_src sh

  rtsg.1023 > csam.login: S 768512:768512(0) win 4096 <mss 1024>
  csam.login > rtsg.1023: S 947648:947648(0) ack 768513 win 4096 <mss 1024>
  rtsg.1023 > csam.login: . ack 1 win 4096
  rtsg.1023 > csam.login: P 1:2(1) ack 1 win 4096
  csam.login > rtsg.1023: . ack 2 win 4096
  rtsg.1023 > csam.login: P 2:21(19) ack 1 win 4096
  csam.login > rtsg.1023: P 1:2(1) ack 21 win 4077
  csam.login > rtsg.1023: P 2:3(1) ack 21 win 4077 urg 1
  csam.login > rtsg.1023: P 3:4(1) ack 21 win 4077 urg 1

#+end_src

The first line says that tcp port 1023 on rtsg sent a packet to port login on csam. The S indicates that the SYN flag was set. The packet sequence number was 768512 and it contained no data. (The notation is `first:last(nbytes)' which means `sequence numbers first up to but not including last which is nbytes bytes of user data'.) There was no piggy-backed ack, the available receive window was 4096 bytes and there was a max-segment-size option requesting an mss of 1024 bytes.

Csam replies with a similar packet except it includes a piggy-backed ack for rtsg's SYN. Rtsg then acks csam's SYN. The `.' means no flags were set. The packet contained no data so there is no data sequence number. Note that the ack sequence number is a small integer (1). The first time tcpdump sees a tcp `conversation', it prints the sequence number from the packet. On subsequent packets of the conversation, the difference between the current packet's sequence number and this initial sequence number is printed. This means that sequence numbers after the first can be interpreted as relative byte positions in the conversation's data stream (with the first data byte each direction being `1'). `-S' will override this feature, causing the original sequence numbers to be output.

On the 6th line, rtsg sends csam 19 bytes of data (bytes 2 through 20 in the rtsg -> csam side of the conversation). The PUSH flag is set in the packet. On the 7th line, csam says it's received data sent by rtsg up to but not including byte 21. Most of this data is apparently sitting in the socket buffer since csam's receive window has gotten 19 bytes smaller. Csam also sends one byte of data to rtsg in this packet. On the 8th and 9th lines, csam sends two bytes of urgent, pushed data to rtsg.

**** UDP Packets
UDP format is illustrated by this rwho packet:

#+begin_src sh

  actinide.who > broadcast.who: udp 84

#+end_src

This says that port who on host actinide sent a udp datagram to port who on host broadcast, the Internet broadcast address. The packet contained 84 bytes of user data.

Some UDP services are recognized (from the source or destination port number) and the higher level protocol information printed. In particular, Domain Name service requests (RFC-1034/1035) and Sun RPC calls (RFC-1050) to NFS.

*** traceroute
``Traceroute'' is a network debugging utility that attempts to trace the path a packet takes through the network - its route. A key word here is ``attempts'' - by no means does traceroute work in all cases.

If you've been paying attention, you already know that the only facilities TCP/IP provide for tracing packet routes are IP packet options (record route and its variants) that are poorly specified, rarely implemented in a useful way, and often disabled for security reasons. Traceroute does not depend on any of these facilities. Traceroute, to put it simply, is a hack.

*How Traceroute Works*:
Traceroute transmits packets with small TTL values. Recall that the TTL (Time To Live) is an IP header field that is designed to prevent packets from running in loops. Every router that handles a packet subtracts one from the packet's TTL. If the TTL reaches zero, the packet has expired and is discarded. Traceroute depends on the common router practice of sending an ICMP Time Exceeded message, documented in RFC 792, back to the sender when this occurs. By using small TTL values which quickly expire, traceroute causes routers along a packet's normal delivery path to generate these ICMP messages which identify the router. A TTL value of one should produce a message from the first router; a TTL value of two generates a message from the second; etc.

#+begin_src

      +--------+                                          +--------+
      | SENDER |                                          | TARGET |
      +--------+                                          +--------+
          |                                                   ^|
       [============( Router )=====( Router )=====( Router )==|====]
                   ^              ^              ^            |
                   | TTL=1        | TTL=2        | TTL=3      | TTL=4
  Traceroute       |              |              |            |
  shows these -----+--------------+--------------+------------/
  IP addresses

#+end_src

In a typical traceroute session, a group of packets with TTL=1 are sent. A single router should respond, using the IP address of the interface it transmits the ICMP Timeout messages on, which should be the same as the interface it received the original packets on. The user is told this IP address, and DNS is used to convert this into a symbolic domain address. Also, round trip times are reported for each packet in the group. Traceroute reports any additional ICMP messages (such as destination unreachables) using a rather cryptic syntax - !N means network unreachable, !H means host unreachable, etc. Once this first group of packets has been processed (this can take 10 seconds or no time at all), the second group (TTL=2) begins transmitting, and the whole process repeats.

*** Wireshark
**** Filter packets
***** Filter by protocol
1. On the bar "Apply a display filter.."
2. Protocol name (for example: dns)

1. On the bar "Apply a display filter.."
2. dns || icmp
Note: only show dns and icmp packages

***** Filter by IP address
1. On the bar "Apply a display filter.."
2. ip.addr == 192.168.1.1

***** Filter by source IP address
1. On the bar "Apply a display filter.."
2. ip.src == 192.168.1.1

***** Filter by destination IP address
1. On the bar "Apply a display filter.."
2. ip.dst == 192.168.1.1

**** Save filter packets
***** Filter by IP address
1. Clicking on the plus button to add a new display filter (+ sign at the end of the bar "Apply a display filter..")
2. ip.addr == 192.168.1.1

***** Filter by source IP address
1. Clicking on the plus button to add a new display filter (+ sign at the end of the bar "Apply a display filter..")
2. ip.src == 192.168.1.1

***** Filter by destination IP address
1. Clicking on the plus button to add a new display filter (+ sign at the end of the bar "Apply a display filter..")
2. ip.dst == 192.168.1.1

**** More expressions
https://www.wireshark.org/docs/wsug_html_chunked/ChWorkBuildDisplayFilterSection.html

** Keyboards
*** QMK
**** Brief
[[https://docs.qmk.fm/][QMK]] (Quantum Mechanical Keyboard) is an open source community centered around developing computer input devices. The community encompasses all sorts of input devices, such as keyboards, mice, and MIDI devices. A core group of collaborators maintains QMK Firmware, QMK Configurator, QMK Toolbox, qmk.fm, and this documentation with the help of community members like you.

Your computer keyboard has a processor inside of it, similar to the one inside your computer. This processor runs software that is responsible for detecting button presses and informing the computer when keys are pressed. QMK Firmware fills the role of that software, detecting button presses and passing that information on to the host computer. When you build your custom keymap, you are creating an executable program for your keyboard.

QMK tries to put a lot of power into your hands by making easy things easy, and hard things possible. You don't have to know how to program to create powerful keymaps — you only have to follow a few simple syntax rules.

**** Mod-Tap
The Mod-Tap key ~MT(mod, kc)~ acts like a modifier when held, and a regular keycode when tapped. In other words, you can have a key that sends Escape when you tap it, but functions as a Control or Shift key when you hold it down.

The modifiers (~mod~) argument to the ~MT()~ macro are prefixed with ~MOD_~, not ~KC_~:

| Modifier | Description                              |
|----------+------------------------------------------|
| MOD_LCTL | Left Control                             |
| MOD_LSFT | Left Shift                               |
| MOD_LALT | Left Alt                                 |
| MOD_LGUI | Left GUI (Windows/Command/Super key)     |
| MOD_RCTL | Right Control                            |
| MOD_RSFT | Right Shift                              |
| MOD_RALT | Right Alt (AltGr)                        |
| MOD_RGUI | Right GUI (Windows/Command/Super key)    |
| MOD_HYPR | Hyper (Left Control, Shift, Alt and GUI) |
| MOD_MEH  | Meh (Left Control, Shift, and Alt)       |

You can combine these by ORing them together like so:

#+begin_src conf

  MT(MOD_LCTL | MOD_LSFT, KC_ESC)

#+end_src

This key would activate Left Control and Left Shift when held, and send Escape when tapped.

**** Sticky keys
Sticky keys are known as 'one shot mods'.

*** VIA
**** Brief
VIA offers extensive customization capabilities, these functions enable efficient and flexible keyboard layouts by facilitating temporary and permanent layer changes, single-action switching, toggling between layers, and sustained layer alterations.

Layers allow the ability to change the functionality of the entire keyboard based on what “layer” it’s currently on. It’s best to visualize layers stacked on top of each other.

*Example of layer behavior*: if you press ~TG(1)~ to go to layer ~1~ from layer ~0~, then to return to layer ~0~ you will need to press ~TG(1)~ again to turn off layer ~1~ and go to the previous activated layer ~0~. You can also press ~TG(3)~ to go to layer ~3~ from layer ~1~, then if you press ~TG(3)~ again you'd go to layer ~1~ not layer ~2~ or ~0~, since layer ~2~ is turned off and layer ~0~ would be overshadowed by layer ~1~.

*Note*: this behavior occurs using these layer keys [[MO(n)]], [[TG(n)]], [[OSL(n)]], [[TT(n)]] and [[DF(n)]] except [[TO(n)]].

#+begin_src text

               _______                                   _______
              /      /                                  /      /
             /______/ TG(3)                            /______/ TG(3)
            _______
           /      /
          /______/ TG(2)
        _______                                  _______
       /      /                                 /      /
      /______/ TG(1)                           /______/ TG(1)
    _______                                  _______
   /      /                                 /      /
  /______/  TG(0)                          /______/  TG(0)

  TG(0),TG(1),TG(2),TG(3) on               TG(0),TG(1),TG(3) on

#+end_src

[[https://epomaker.com/blogs/guides/how-to-use-via-for-beginners][Epomaker beginner's guide]]

[[https://epomaker.com/blogs/guides/via-usage-guide][Epomaker advance guide]]

**** Layers
***** MO(n)
When the MO key is *held down*, it *temporarily* activates the designated layer specified by the number n in parentheses.

For example, if I set the Fn key on Layer 0 to MO(1) and assign the Z key on Layer 1 to the number 1, pressing and holding the Fn (MO1) key while pressing the Z (1) key will result in the input of the number 1, reverting to the base layer once the key is released.

***** TG(n)
The TG(n) function stands for "*toggle*," which facilitates a *permanent switch* to Layer n without reverting automatically to Layer 0 (*on/off*). This function allows toggling between two layers, but it does not support switching among multiple layers.

For instance, if you configure the Fn key on Layer 0 to TG(3) and assign the Z key on Layer 3 to input the number 9, pressing the Fn key once will permanently switch the keyboard to Layer 3. Consequently, subsequent presses of the Z (9) key will consistently result in the input of the number 9, regardless of how many times it is pressed and released. This toggle functionality provides a reliable means of maintaining a persistent layer switch for specialized tasks.

Once the keyboard is permanently switched to Layer 3 using TG(3), it remains on this layer until manually switched back. To return to Layer 0, you can employ one of the following methods:

1. KC_TRNS: Place KC_TRNS (transparent key) in the same key position as TG(3). This key is represented by an inverted triangle on the basic layer. Pressing this key will revert the keyboard to the base layer, Layer 0.

2. TO(0): Configure any key position to TO(0), which explicitly transfers the keyboard back to Layer 0 when pressed.

Both methods provide a straightforward way to revert to the default layer after toggling to a different layer with TG(n), ensuring flexible and efficient layer management.

***** TT(n)
Normally acts like [[MO(n)]] unless it's tapped multiple times (5x) which toggles [[TG(n)]] layer n on.

***** OSL(n)
The OSL(n) function, which stands for *One Shot Layer*, provides a temporary switch to a specified layer (Layer n). When OSL(n) is activated, the keyboard momentarily transitions to the indicated layer, allowing for a single key action. After pressing any key on this layer, the keyboard automatically reverts to the default layer (Layer 0). This feature is particularly useful for executing a single action on a different layer without permanently changing layers.

For instance, if you configure the Fn key on Layer 0 to OSL(2) and assign the Z key on Layer 2 to input the number 3, pressing the Fn (OSL(2)) key briefly, followed by the Z (3) key, will result in the input of the number 3. Thus, the OSL function enables a temporary switch to Layer n with a brief key press, and after a key on Layer n is triggered, it seamlessly returns to Layer 0.

***** TO(n)
The TO(n) function is designed to switch to Layer n with a short press, *maintaining that layer until another layer switch is manually initiated*. This function supports permanent toggling between multiple layers, enhancing the flexibility of your keyboard layout.

*Note*: it deletes the previous layers activated, unlike [[MO(n)]], [[TG(n)]], [[OSL(n)]], [[TT(n)]]  which keeps the previous layers, that is, if you have the layers ~1~ and ~2~ activated and you press ~TO(3)~ and then press ~TG(3)~ to turn it off, you return to the layer ~0~ not to the ~2~.

For example, if you configure TO(3) on Layer 0 and TO(5) on Layer 3, you can seamlessly transition from Layer 0 to Layer 5 with just two short presses. First, pressing TO(3) switches from Layer 0 to Layer 3. Then, pressing TO(5) switches from Layer 3 to Layer 5.

***** DF(n)
Sets the default layer ~0~.

*Note*: overrides previous layers, i.e. if you press ~DF(2)~ it will take it as default and override layers ~0~ and ~1~, to get layers ~0~ and ~1~ back you need to press ~DF(1)~ or ~DF(0)~.

***** Fn1 (Fn3)
The Fn1 (Fn3) key offers *dual functionality*, enhancing the versatility of your keyboard layer management. The first function operates similarly to MO(1): *pressing and holding* Fn1 temporarily switches the keyboard to Layer 1, reverting to Layer 0 upon release. The second function is activated by *simultaneously pressing and holding both* Fn1 and Fn2, which switches the keyboard to Layer 3. This dual capability allows for flexible and efficient layer switching, providing a robust solution for advanced keyboard customization.

***** Space Fn(n)
The Space Fn(n) function enables *dual functionality* for enhanced keyboard efficiency. A *short press* of the key outputs a single space, while a *long press temporarily* switches the keyboard to Layer n. Upon release, the keyboard automatically reverts to Layer 0. This behavior mirrors the functionality of MO(n), offering a convenient method for temporary layer switching while maintaining the primary function of the space key.

***** Transparency
The Triangle symbols represent Transparency. Those mean the action is the same as the layer below it. This is useful so that keys from different layers can be pressed at the same time.

**** Any key
***** LT
Using LT for *Layer Switching*: The LT function, which stands for "Layer Tap," is a common method for layer switching. The syntax is LT(layer, kc), where *holding the key switches* to the specified layer, and *tapping the key triggers a keycode*.

For example, LT(3, KC_D) means it temporarily switches to Layer 3 when held down and outputs the key value D when tapped.

***** MT
The term MT(mod, kc) stands for "*Modifier Tap*." This functionality allows a key to serve *dual purposes*, acting as a *modifier when held down* and registering a *different keypress when tapped* quickly. The setting method is the same as LT.

    Long Press: When you hold down the key, it functions as the specified modifier.
    Short Press: When you tap the key quickly, it triggers the specified keycode.

The code MT(MOD_LCTL, KC_CAPS). In this instance:

    Long Press: When you press and hold the key, it functions as the Control (Ctrl) key.
    Short Press: When you tap the key quickly, it toggles the Caps Lock on and off.

***** Modifier key
The *modifier* key combination is mod+kc. In VIA, modifier key combinations are used to trigger specific actions or functions. The syntax for these combinations follows a pattern:

- Shift + KC: Represented as S(KC)
- Ctrl + KC: Represented as C(KC)
- Alt + KC: Represented as A(KC)
- GUI + KC: Represented as G(KC)

Unlike the MT (Modifier Tap) function, which activates different actions based on whether the key is long-pressed or short-pressed, modifier key combinations involve simultaneously holding down the modifier key (mod) and the specified key (KC) to achieve the desired effect.

For example, let's see the 'Alt + A' shortcut for taking a screenshot (APP: iShot). This combination can be set using a macro as {KC_LALT, KC_A}, or using the Any key. First, we need to find the syntax for the alt key combination, which is A(KC) according to the QMK documentation. Therefore, the keycodes for "alt+a" would be represented as A(KC_A). Now, let's set this combination in VIA. We'll assign it to the "a" key on the second layer. Once configured, when we hold down the "d" key to switch to the second layer, pressing "a" while holding the "alt" key will trigger the screenshot function.

Similar syntax applies to the other three modifier keys (Shift, Ctrl, and GUI), and nested combinations can be achieved for multiple modifier keys. For example, macOS has a built-in screenshot shortcut "shift+command+4." The keycodes for this combination in VIA would be S(G(KC_4)), ensuring the parentheses match the nested structure. VIA will automatically adjust nested modifiers. When you confirm the combination and then click "any," you can see that VIA has automatically modified it.

*** Vial
**** Install
https://get.vial.today/manual/

1. Download Vial

Hop over to the [[https://get.vial.today/download/][download]] page to make sure you have the latest build.

2. Run Vial

Make sure that your downloaded file can be executed. You can either do this through your graphical file explorer or use the following command:

#+begin_src sh

  chmod +x ~/Downloads/Vial*.AppImage

#+end_src

After that simply launch the application by either double clicking on it or executing the following command:

#+begin_src sh

  ~/Downloads/Vial*.AppImage

#+end_src

3. Program your new keyboard

Using Vial is extremely simple, just click on a key you would like to change the value of in the top palette then select your replacement key or macro in the bottom palette.

Your new key will be automatically saved onto the keyboard and immediately ready for use.

**** Layers
Layers allow the ability to change the functionality of the entire keyboard based on what “layer” it’s currently on. It’s best to visualize layers stacked on top of each other. Keys can be configured to switch between layers as needed similar to function keys on a laptop.

*Moving Between Layers*
Switching between what layer is active can be done in a few ways. In the lower palette, select the layer tab to view all the different options.

- MO(layer) :: momentarily activates the layer. As soon as you let go of the key, the layer is deactivated.
- DF(layer) :: changes which layer is the default layer. This stays until the device loses power.
- TG(layer) :: toggles the layer, activating it if it’s inactive and vice versa.
- TT(layer) :: If you hold the key down, the layer is activated, and then is de-activated when you let go (like MO). If you repeatedly tap it, the layer will be toggled on or off (like TG). It needs 5 taps to do this.
- OSL(layer) :: momentarily activates the layer until the next key is pressed.
- LT layer (kc) :: momentarily activates the layer when it is held, sends a keycode when pressed. The keycode can be defined like all the other buttons, Just select the smaller box inside.

*Transparency*
The Triangle symbols represent Transparency. Those mean the action is the same as the layer below it. This is useful so that keys from different layers can be pressed at the same time.

**** Macros
Macros can be used to perform a pre-programmed sequence of actions or steps.

*Configure Macros*
Click the /Macros/ tab. All the macros that can be configured will be displayed as seperate tabs. Select a Macro you would like to configure.

The next step is to add actions. In the bottom right corner, you can

- Add action :: Manually adds an action to the list. Configure it to do exactly what you want. select from different options - hold, tap, release, and delay.
- Tap Enter :: Lots of macros end with an “Enter” key so this button makes it easier to add that.
- Record macro :: Lets you record the macro directly from your keyboard. It’s not the most reliable but it’s a great place to start.

After the macro has been configured, it can be used. Just click on a key you would like to use for a macro in the top palette then select your macro in the bottom palette. (under the macro tab)

**** Tap Dance
Tap Dance allows you to configure different actions depending on how a button is used. For example, the same button could be configured to active a macro when the button is tapped or activate something else when the button is held.

*Note*: The /tapping term/ represents the timing on how the firmware distinguishes between the actions. If a button is held for longer than the tapping term it’s considered a hold. If a button is held for shorter than the tapping term it’s considered a tap.

**** Combos
Combos allow you to add custom actions when a certain combination of keys is pressed. for example, hitting A and S within the combo term would hit ESC instead, or have it perform even more complex tasks.

**** Alt Repeat Key
The Alt Repeat Key is a key whose behavior depends on the last pressed key. You can find it under the Quantum tab.

By default, Alt Repeat is defined for navigation keys to navigate in the reverse direction. For instance, if the last pressed key was Ctrl + Shift + Right Arrow, the Alternate Repeat Key behaves as Ctrl + Shift + Left Arrow.

**** Mod Tap or Quantum vs Tap Dance
In fact, it’s all too powerful for some simple functions. Like holding a key for a modifier instead of the normal character, is entirely overkill to implement with a Tap Dance. Especially when that basic function is already there in QMK/Vial in a much simpler format called Mod Tap.

Under the Keymap -> Quantum tab you can find a long list of keycodes that create the exact scenario above, where you have a modifier on hold, and then select your desired ‘normal’ key-code for tapping.

Please read more in QMK Documentation: [[Mod-Tap]] (or [[https://docs.qmk.fm/mod_tap#mod-tap][online]])

*** logkeys
**** Brief
logkeys  is a linux keylogger.

*en_US_debian_1204.map*
keymap file that acts as an input keymap for processing pressed keys.

*logkeys-keylogger.service*
Service unit used to automatically activate the Keylogger at system startup.

- Type=forking :: systemd considers the service started up once the process forks and the parent has exited.

If a service is of type ~forking~, the ~ExecStart=~ command is expected to be a short-lived init process that forks of a main process. The service is considered to be active as soon as the ~ExecStart=~ command terminates successfully, and it remains in this state until the forked off process terminates.

*Note*: it's necessary to change the service type to forking ~Type=forking~ otherwise the service would be activated and immediately deactivated automatically.

**** Script that displays statistics
The following script displays statistics for the characters typed. It's generated by ChatGPT; it's unverified, but it appears to work with some minor glitches.

#+begin_src python

  #!/usr/bin/env python3
  from collections import Counter
  import re
  import sys

  logfile = sys.argv[1] if len(sys.argv) > 1 else '/var/log/logkeys.log'

  with open(logfile, 'r', encoding='utf-8', errors='ignore') as f:
      text = f.read()

  # Match special keys like <Enter>, <Tab>, or single characters
  keys = re.findall(r'<[^>]+>|.', text)

  # Count frequency
  c = Counter(keys)

  # Display sorted results
  for k, v in c.most_common():
      print(f'{repr(k):>10} : {v}')

#+end_src

To use it:

#+begin_src

  chmod +x key_stats.py

  python3 key_stats.py

#+end_src

*** keyd
**** Brief
[[https://github.com/rvaiya/keyd][Keyd]] is a key remapping daemon for Linux that allows users to customize keyboard layouts and key functions system-wide or per application. It supports features like layering, *oneshot modifiers*, and macros, making it a flexible tool for managing keyboard input across different environments.

xmodmap and friends are display server level tools with limited functionality. keyd is a system level solution which implements advanced features like layering and oneshot modifiers. While some X tools offer similar functionality I am not aware of anything that is as flexible as keyd.

[[https://www.emacswiki.org/emacs/StickyModifiersKeyboardConfigDaemon][Sticky Modifiers Keyboard Config Daemon (Emacswiki)]]
These options (keyd, [[kmonad]]) translate keypresses before they’re received by the desktop environment. And they work regardless of whether X, Wayland or some other GUI framework applies. In addition to *StickyKeys*, these programs allow nearly any keyboard customization you can think of.

**** Config
Configuration files loosely follow an INI style format consisting of headers of the form ~[section_name]~ followed by a set of /bindings/. Lines beginning with a hash are ignored.

Config files are stored in ~/etc/keyd/~ and loaded upon initialization. The reload command can be used to update the working set of config files (e.g ~sudo keyd.rvaiya reload~).

A valid config file has the extension ~.conf~ and *must* begin with an ~[ids]~ section that has one of the following forms:

Note that you can obtain the ~<vendor id>:<product id>~ specifiers from the monitor command.

#+begin_src conf

  # Specifies a list of ids to be explicitly matched.
  [ids]

  <vendor id 1>:<product id 1> # obtained via keyd monitor
  <vendor id 2>:<product id 2>
  ...

  # or
  # Matches any id which has not been explicitly excluded.
  [ids]

  ,*

  -<vendor id 1>:<product id 1>
  -<vendor id 2>:<product id 2>
  ...

#+end_src

**** Layers
A layer is a collection of /bindings/, each of which specifies the behaviour of a particular key. Multiple layers may be active at any given time, forming a stack of occluding keymaps consulted in activation order. The default layer is called ~main~ and is where common bindings should be defined.

**** Actions
A key may optionally be bound to an action which accepts zero or more arguments.

- layer(<layer>) :: activate the given layer for the duration of the keypress.

- oneshot(<layer>) :: if tapped, activate the supplied layer for the duration of the next keypress.

- overload(<layer>, <action>) :: activates the given layer while held and executes <action> on tap.

- noop ::  do nothing.

- command(<shell command>) :: execute the given shell command. (e.g. ~command(brightness down)~) *Note*: Commands are executed by the user running the keyd process (probably root), use this feature judiciously.

- swap(<layer>) :: swap the currently active layer with the supplied one. If the current layer is toggled, it is deactivated and the supplied layer is toggled instead. Otherwise, the active layer is deactivated and the supplied layer remains active for the duration of the depression of the activating key.

#+begin_src conf

  [control]

  x = swap(xlayer)

  [xlayer]

  s = C-s
  b = S-insert

#+end_src

*Note*: you probably don’t need to use this unless you are trying to do something quite involved. Think hard about whether or not what you are trying to achieve can be done by other means, as it is easy to end up in states which are impossible to exit.

**** Macros
A macro expression has one of the following forms:

1. ~macro(<exp>)~
2. ~[<modifier 1>[-<modifier 2>...]-<key>~
3. ~<char>~

Where ~<char>~ is a valid unicode character and ~<exp>~ has the form ~<token1> [<token2>...]~ and each token is one of:

• A valid key code.
• A type 2 macro.
• A contiguous group of unicode characters.
• A group of key codes delimited by + to be depressed as a unit.
• A timeout of the form ~<time>ms~ (where <time> < 1024).

Splitting into smaller tokens serves as an escaping mechanism: ~macro(space)~ inserts a =space= but ~macro(s pace)~ writes ~space~. Likewise, ~macro(3+5)~ depresses the =3= and =5= keys as a unit while ~macro(3 + 5)~ writes ~3+5~.

The following are all valid macro expressions:

#+begin_src conf

  C-a
  macro(C-a)
  # simultaneously taps the left meta and left control keys
  macro(leftcontrol+leftmeta)
  A-M-x
  macro(Hello space World)
  macro(h e l l o space w o r ld) (identical to the above)
  macro(C-t 100ms google.com enter)

#+end_src

**** Chording
/Chords/ are groups of keys which are treated as a unit when simultaneously depressed. A chord can be defined by using a group of ~+~ delimited key names as a left hand value. The corresponding action will be activated if all keys are struck within the chording interval (~chord_timeout~).

This will cause =esc= to be produced if both =j= and =k= are simultaneously depressed.

#+begin_src conf

  j+k = esc

#+end_src

*Note*: It may be desirable to change the default chording interval (50ms) to account for the physical characteristics of your keyboard.

**** Examples
***** Layer that creates aditional arrow keys (layer)
Defines a layer called ~nav~ and creates a toggle for it in the ~main~ layer. When =capslock= is held, the ~nav~ layer occludes the ~main~ layer causing =hjkl= to function as the corresponding arrow keys.

#+begin_src conf

  [ids]

  ,*

  [main]

  capslock = layer(nav)

  [nav]

  h = left
  k = up
  j = down
  l = right

#+end_src

***** Make esc+q toggle the dvorak letter layout (toggle)

#+begin_src conf
  [ids]
  ,*

  [main]
  esc = layer(esc)

  [dvorak]

  a = a
  s = o
  ...

  [esc]

  q = toggle(dvorak)

#+end_src

***** One key to switch other layers (oneshoot, toggle)
One key press to switch layers, as well as other useful buttons.

In this example, press =insert= + =<letter>= to switch layers (e.g. =insert= + =e= will switch to the english layer), or useful buttons (e.g. =insert= + =l= is equivalent to the left mouse button).

When you switch to other layer to return to the main layer, press the same combination (e.g.  =insert= + =e= will return to the main layer again from the english layer).

*Note*: keys can be pressed in sequence or holding down the leader key, which in this case is the =insert= key. For example, =insert= =e=, is the same as, =insert= + =e=.

#+begin_src conf

  [main]

  # Switches to other layers and other useful keys
  insert = oneshot(layers)


  # Switches to other layers and other useful keys
  [layers]

  n = toggle(numbers)
  e = toggle(english)

  # Enable the menu key (menu = compose)
  m = compose
  l = leftmouse
  2 = f2


  # Enable numbers using the right hand
  [numbers]

  u = 7
  i = 8
  o = 9
  j = 4
  k = 5
  l = 6
  m = 1
  , = 2
  . = 3
  space = 0
  n = space


  # To practice with Google Translate
  [english]

  # Google Translate speaks
  . = A-x
  # Google Translate listens
  , = A-z

  # Copy a sentence from GoogleTranslate and paste in a new Emacs Org header
  0 = macro(f5 100ms C-enter 50ms C-y 50ms f12 100ms A-c 50ms f5 100ms C-c C-b 50ms C-c C-f 50ms C-right 50ms C-left 50ms C-y 50ms enter)

  # GoogleTranslate speaks a sentence, switches to Emacs, folds the header,
  # moves to the next, folds it, and places the cursor at the beginning
  1 = macro(A-x 50ms f5 100ms tab 50ms C-c C-n 50ms tab 50ms C-right 50ms C-left)

  # GoogleTranslate speaks a sentence, and switches to Emacs and fold the header
  2 = macro(A-x 50ms f5 100ms tab)

#+end_src

***** Layers with aditional modifier (layer)
Unlike most other remapping tools, keyd provides first class support for modifiers. A layer name may optionally end with a ~:~ followed by a set of modifiers to emulate in the absence of an explicit mapping.

These layers play nicely with other modifiers and preserve existing stacking semantics.

Formally, each layer heading has the following form:

~"[" <layer name>[:<modifier set>] "]"~

Where <modifier_set> has the form:

~<modifier1>[-<modifier2>]...~

This will cause =capslock= to behave as =control=, except in the case of =control+j=, which will emit =down=. This makes it trivial to define custom modifiers which don’t interfere with one another.

Note that bindings are not affected by the modifiers of the layer in which they are defined. Thus =capslock+j= will produce an unmodified =down= keypress, while =shift+control+j= will produce =shift+down= as expected.

#+begin_src conf

  [main]

  capslock = layer(capslock)

  [capslock:C]
  j = down

#+end_src

***** CapsLock into sticky key an additional Ctrl key (oneshot, toggle)
Here is a Keyd configuration that implements *StickyKeys* and turns CapsLock into an additional “Ctrl” key.

#+begin_src conf

  [ids]

  ,*

  # Latching (first press activates modifier's layer for next key)
  [main]
  capslock = oneshot(control)

  # Locking (second press locks modifier until pressed again)
  [control]
  capslock = toggle(control)

#+end_src

***** Enable and disable sticky keys (different layers)
Enable sticky keys and disable it on a different layer.

#+begin_src conf

  [ids]
  ,*

  [main]
  esc = layer(esc)


  # Enable sticky keys
  control = oneshot(control)
  shift = oneshot(shift)

  [control]

  control = toggle(control)

  [shift]

  shift = toggle(shift)


  # esc + m enables the layer with modifier by default
  [esc]

  m = toggle(modifiers)

  [modifiers]

  control = layer(controlmod)
  shift = layer(shiftmod)

  [controlmod:C]

  [shiftmod:S]

#+end_src

***** Enables the sticky keys behavior for x seconds
Oneshot_timeout enables the sticky keys behavior for x seconds

#+begin_src conf

  [ids]
  ,*

  [global]
  #900 millisecond timeout for sticky keys,
  # works good for keyboard but if you use mouse then
  # it's a bit problematic.
  oneshot_timeout = 900ms


  [main]

  control = oneshot(control)
  meta = oneshot(meta)
  shift = oneshot(shift)
  rightalt = oneshot(altgr)
  leftalt = oneshot(alt)
  capslock = backspace

#+end_src

***** Act as a control and another key (overload)
Causes =capslock= to behave as =control= when pressed and escape when tapped. Note that =control= is what keyd regards as a 'modifier layer'.

#+begin_src conf

  [ids]

  ,*

  [main]

  capslock = overload(control, esc)

#+end_src

***** Capslock to scp, hold control, and shift + capslock to capslock ([shift:S])
Holding =CapsLock= is =Ctrl=, tapping =CapsLock= once is =Esc=, and doing =Shift+CapsLock= is =CapsLock=.

#+begin_src conf

  [ids]

  ,*

  [main]

  # Maps capslock to escape when pressed and control when held.
  capslock = overload(control, esc)

  # Shift layer
  [shift:S]

  ## shift+capslock is capslock
  capslock = capslock

#+end_src

***** Vi like navigation layer ([key:C], [key+shift])
To create a vi like navigation layer accessible by holding =capslock= while holding =capslock= otherwise behave like the =control= key (when held) and =escape= when tapped.

In essence we want:

 - capslock+h = left
 - capslock+j = down
 - capslock+k = up
 - capslock+l = right

 - capslock+shift+h = C-left
 - capslock+shift+l = C-right
 - capslock+shift+j = C-down
 - capslock+shift+k = C-up

 - capslock = escape (when tapped)

In all other instances capslock should behave as normal control:

 - capslock+a = C-a
 - capslock+b = C-b
 - capslock+c = C-c
 ...
 - capslock+shift+i = C-S-i
 - capslock+shift+k = C-S-k
 ....

This is achieved by creating a composite layer called ~[capslock+shift]~ which is activated whenever capslock and shift are simultaneously held.

#+begin_src conf

  [ids]
  ,*

  [main]
  capslock = overload(capslock, esc)

  [capslock:C]
  h = left
  j = down
  k = up
  l = right

  # Activates when both capslock and shift is pressed
  [capslock+shift]
  h = C-left
  j = C-down
  k = C-up
  l = C-right

#+end_src

***** Advanced use of capslock as a triple function key (overload)

- when =capslock= is tapped (pressed + released), it behaves as =ESC=
  key

- when =capslock= is held, and used in combination with another key,
  it behaves like a =ctrl= key modifier (just like xcape)

- when =capslock= is held, and the =space= key is tapped, we enter a
  3rd state for "VIM mode" where hjkl keys become arrow keys until
  capslock is released.

#+begin_src conf

  [ids]
  ,*

  [main]

  capslock = overload(ctrl_vim, esc)

  # ctrl_vim modifier layer; inherits from 'Ctrl' modifier layer

  [ctrl_vim:C]

  space = swap(vim_mode)

  # vim_mode modifier layer; also inherits from 'Ctrl' modifier layer

  [vim_mode:C]

  h = left
  j = down
  k = up
  l = right
  # forward word
  w = C-right
  # backward word
  b = C-left

#+end_src

***** The space bar as a shift key
This example demostrates how to use the space-bar as a shift key by:

1. Holding shift
2. Holding space
3. Releasing shift

The space-bar will keep the shift layer active until your release it. The advanteage is that you can type using the fingers of both hands freely. Consider it as an alternative to caps-word mode.

#+begin_src conf

  [ids]

  ,*

  [shift]

  space = overloadt(shift, space, 200)

#+end_src

***** Tapping both shift keys will activate capslock

#+begin_src conf

  [shift]

  leftshift = capslock
  rightshift = capslock

#+end_src

***** Create modifier sticky keys
Tapping control once causes it to apply to the next key, tapping it twice activates it until it is pressed again, and holding it produces expected behaviour.

#+begin_src conf

  [ids]
  ,*

  [main]

  control = oneshot(control)
  shift = oneshot(shift)
  meta = oneshot(meta)
  alt = oneshot(alt)

  [control]

  control = toggle(control)

  [shift]

  shift = toggle(shift)

  [meta]

  meta = toggle(meta)

  [alt]

  alt = toggle(alt)

#+end_src

***** No action within a times (timeout)
This example demonstrates what is sometimes called a simlayer: when the layer is activated, if no action is taken within a specified period of time, another action takes place. This can be used, for example, to preserve the repeat action of the key that is being overloaded (=backspace= in this case).

#+begin_src conf

  [ids]

  ,*

  [main]

  backspace = timeout(overload(shortcuts, backspace), 500, backspace)

  [shortcuts]

  1 = M-pageup
  2 = M-pagedown
  # Other shortcuts here.

#+end_src

***** Carousel layer
This example demostrates how to implement a layer carousel which behaves as follows:

- Hold space (numbers layer)
- Tap right shift (functions layer)
- Tap left shift (numbers layer)
- Tap left shift (symbols layer)
- Release space (main layer)

By using swap in our secondary layers, the layer held by space (as determined in the main layer) is replaced and remains active until another swap takes place or the space is eventually released, which brings us back to the main layer.

#+begin_src conf

  [ids]

  ,*

  [main]
  space = overloadt(numbers, space, 200)

  [numbers]
  a = 1
  # Other numbers here.
  leftshift = overload(shift, swap(symbols))
  rightshift = overload(shift, swap(functions))

  [functions]
  a = f1
  # Other functions here.
  leftshift = overload(shift, swap(numbers))
  rightshift = overload(shift, swap(symbols))

  [symbols]
  a = grave
  # Other symbols here.
  leftshift = overload(shift, swap(functions))
  rightshift = overload(shift, swap(numbers))

#+end_src

***** Complete z layer

#+begin_src conf

  [ids]

  ,*

  [main]

  # Maps capslock to escape when pressed and control when held.
  capslock = overload(control, esc)

  # Maps z to a custom layer, but just 'z' when pressed.
  z = overload(z, z)

  # Shift layer
  [shift:S]

  ## shift+capslock is capslock
  capslock = capslock

  # Custom z layer
  [z]

  ## Escape common patterns
  i = macro(zi)
  o = macro(zo)

  ## Emails
  e = macro(hi@ethanppl.com)
  g = macro(hi@ethanppl.com)

  ## Simplify complicated shortcut keys
  v = C-S-v
  n = C-S-a
  x = A-f4
  s = command(systemctl suspend)

  ## Arrows
  j = down
  k = up
  h = left
  l = right

  ## Media / modifier
  m = oneshot(media)

  [media]

  j = previoussong
  k = playpause
  l = nextsong

  a = macro({ enter 10ms "Aut 10ms hor 10ms iza 10ms tio 10ms n": space "Bea 10ms rer space C-v)

#+end_src

***** Disables the esc and end keys

#+begin_src conf

  [main]
  esc = noop
  end = noop

#+end_src

***** Triple click
The Enter key on the keypad to triple-click.

#+begin_src conf

  # Triple click
  kpenter = macro(rightmouse 100ms rightmouse 100ms rightmouse)

#+end_src

***** Invert the behaviour of number and symbols
Invert the behaviour of the shift key without breaking modifier behaviour.

#+begin_src conf

  [ids]
  ,*

  [main]
  1 = !
  2 = @
  3 = #
  4 = $
  5 = %
  6 = ^
  7 = &
  8 = *
  9 = (
  0 = )

  [shift]
  0 = 0
  0 = 0
  1 = 1
  2 = 2
  3 = 3
  4 = 4
  5 = 5
  6 = 6
  7 = 7
  8 = 8
  9 = 9

#+end_src

***** Compose key to display international glyphs
Uses the compose key functionality of the display server to generate international glyphs.

*NOTE*: it doesn't work well in some window apps. Instead, it triggers the menu key (e.g. in the gnome terminal, text editor, ...), and sometimes it doesn't work at all.

#+begin_src conf

  [ids]
  ,*


  [main]

  rightalt = oneshot(dia)

  [dia]

  # For this to work 'setxkbmap -option compose:menu' must
  # be run after keyd has started.

  # If you want to revert this behavior replace all
  # previously specified options, use the -option flag
  # with an empty argument first 'setxkbmap -option'

  # Map a to á
  a = macro(compose a ')

  # Map e to é
  e = macro(compose ' e)

  # Map i to í
  i = macro(compose i ')

  # Map o to ó
  o = macro(compose o ')

  # Map u to ú
  u = macro(compose u ')

  # Map n to ñ
  n = macro(compose n ~)

#+end_src

**** Personal examples
***** default.conf
Maps all available devices except those that have their own configuration file.

#+begin_src conf

  [ids]

  ,*

  [main]

  # To use as a fix key (pause), and bind it to a gnome command (Show the
  # overview), and also use as a modifier key that enables an extra layer
  capslock = overload(extralayer, pause)

  # Change the insert key to the menu key (menu = compose)
  insert = compose


  # Switch to last window
  f1 = M-grave
  # Cut (works almost universally like C-x)
  f2 = S-delete
  # Copy (works almost universally like C-c)
  f3 = C-insert
  # Paste (works almost universally like C-v)
  f4 = S-insert

  # Cancel
  f6 = C-g
  # Undo
  f7 = C-z
  # New tab
  f8 = C-t

  # Switch to previous tab
  f10 = C-pageup
  # Switch to next tab
  f11 = C-pagedown


  # Close windows
  numlock = C-w

  # Click
  kpenter = rightmouse


  # Enable sticky keys
  # Latching (first press activates modifier's layer for next key)
  control = oneshot(control)
  shift = oneshot(shift)
  meta = oneshot(meta)
  alt = oneshot(alt)

  # Locking (second press locks modifier until pressed again)
  [control]

  control = toggle(control)

  [shift]

  shift = toggle(shift)

  [meta]

  meta = toggle(meta)

  [alt]

  alt = toggle(alt)


  # Extra layer for numbers and other tricks
  [extralayer]

  u = 7
  i = 8
  o = 9
  j = 4
  k = 5
  l = 6
  m = 1
  , = 2
  . = 3
  space = 0
  n = space

  # Function keys
  2 = f2

  # Toggle to numbers layer (capslock+tab)
  tab = toggle(numbers)

  # To practice with Google Translate
  e = toggle(english)

  # To use design programs with default modifier behavior
  d = toggle(design)

  [numbers]

  u = 7
  i = 8
  o = 9
  j = 4
  k = 5
  l = 6
  m = 1
  , = 2
  . = 3
  space = 0
  n = space

  # Function keys
  2 = f2

  [english]

  # Google Translate speaks
  x = A-x
  # Google Translate listens
  z = A-z

  # Copy a sentence from GoogleTranslate and paste in a new Emacs Org header
  kp0 = macro(f5 100ms C-enter 50ms C-y 50ms f12 100ms A-c 50ms f5 100ms C-c C-b 50ms C-c C-f 50ms C-right 50ms C-left 50ms C-y 50ms enter)

  # GoogleTranslate speaks a sentence, switches to Emacs, folds the header,
  # moves to the next, folds it, and places the cursor at the beginning
  kp1 = macro(A-x 50ms f5 100ms tab 50ms C-c C-n 50ms tab 50ms C-right 50ms C-left)

  # GoogleTranslate speaks a sentence, and switches to Emacs and fold the header
  kp2 = macro(A-x 50ms f5 100ms tab)

  [design]

  control = layer(controlmod)
  shift = layer(shiftmod)

  [controlmod:C]

  [shiftmod:S]

#+end_src

***** AT-Translated.conf
Maps only the specific device, in this case the laptop keyboard AT-Translated ~0001:0001~, the ~id~ is obtained using the command ~keyd.rvaiya monitor~.

*Warning*: being the main device sometimes creates problems with the mouse (INS).

#+begin_src conf

  [ids]

  0001:0001

  [main]

  # Copy a sentence from GoogleTranslate and paste in a new Emacs Org header
  kp0 = macro(f5 100ms C-enter 50ms C-y 50ms f12 100ms A-c 50ms f5 100ms C-c C-b 50ms C-c C-f 50ms C-right 50ms C-left 50ms C-y 50ms enter)

  # GoogleTranslate speaks a sentence, switches to Emacs, folds the header,
  # moves to the next, folds it, and places the cursor at the beginning
  kpdot = macro(A-x 50ms f5 100ms tab 50ms C-c C-n 50ms tab 50ms C-right 50ms C-left)

  # GoogleTrans speaks a sentence, and switches to Emacs and fold the header
  kp3 = macro(A-x 50ms f5 100ms tab)

#+end_src

***** Nordic-2.4G-Wireless-Receiver-Mouse

#+begin_src conf

  [ids]

  25a7:fa10

  [main]

  # Toggle dwell command (enable/disable auto-click)
  mouse2 = M-A-S-z

  # First focus the window and then close it.
  middlemouse = macro(rightmouse C-w)

#+end_src

*** input-remapper
**** Brief
Input device button mapping tool. ~input-remapper~ allows users to map buttons on all input devices (keyboards, mice, gamepads...) in X11 and Wayland. It also supports combined buttons and programmable macros.

input-remapper includes a UI to configure button mappings, per device, and configuration to automatically apply button mappings at boot and on device connection.

A ~key + key + ... + key~ syntax can be used to trigger key combinations. for example ~Control_L + a~.

Writing ~disable~ as a mapping disables a key.

Macros allow multiple characters to be written with a single key-press.

See [[https://github.com/sezanzeb/input-remapper/blob/HEAD/readme/usage.md][usage.md]] for more info.
See [[https://github.com/sezanzeb/input-remapper/blob/HEAD/readme/examples.md][examples.md]] for more examples.
See [[https://github.com/sezanzeb/input-remapper/blob/HEAD/readme/macros.md][macros.md]] for a list of all macros.

**** Usage
Look into your applications menu and search for Input Remapper to open the UI. You should be prompted for your sudo password as special permissions are needed to read events from ~/dev/input/~ files. You can also start it via ~input-remapper-gtk~.

First, select your *device* (like your keyboard), then create a new *preset*, and add a mapping.

Changes are saved automatically. Press the "Apply" button to activate (inject) the mapping you created.

If you later want to modify the Input of your mapping you need to use the "Stop" button, so that the application can read your original input. It would otherwise be invisible since the daemon maps it independently of the GUI.

*Note*: you could map =Super= or other key to ~disable~. Now =Super= won't do anything anymore, and therefore pressing the combination won't have any side effects.

*Combinations*
You can use combinations of different inputs to trigger a mapping: While you record the input (Record - Button) press multiple keys and/or move axis at once. The mapping will be triggered as soon as all the recorded inputs are pressed.

**** Writing Combinations
You can write ~Control_L + a~ as mapping, which will inject those two keycodes into your system on a single key press. An arbitrary number of names can be chained using ~+~.

**** Key Names
Check the autocompletion of the GUI for possible values. You can also obtain a complete list of possiblities using ~input-remapper-control --symbol-names~.

Input-remapper only recognizes symbol names, but not the symbols themselves. So for example, input-remapper might (depending on the system layout) know what a ~minus~ is, but it doesn't know ~-~.

Key names that start with ~KEY_~ are keyboard layout independent constants that might not result in the expected output. For example using ~KEY_Y~ would result in "z" if the layout of the environment is set to german. Using ~y~ on the other hand would correctly result in "y" to be written.

It is also possible to map a key to ~disable~ to stop it from doing anything.

*Note*: the =KEY_COMPOSE= and =Multi_ke= keys make the ~Show the overview~ (show all current windows) System command in debian gnome.

**** Configuration Files
If you don't have a graphical user interface, you'll need to edit the configuration files. All configuration files need to be valid json files, otherwise the parser refuses to work.

The default configuration is stored at ~~/.config/input-remapper/config.json~, which doesn't include any mappings, but rather other parameters that are interesting for injections.

#+begin_src json

  {
      "autoload": {
          "Logitech USB Keyboard": "preset name"
      },
      "version": "1.6"
  }

#+end_src

~preset name~ refers to ~~/.config/input-remapper/presets/device name/preset name.json~. The device name can be found with ~sudo input-remapper-control --list-devices~.

The preset files are a collection of mappings. Here is an example: ~~/.config/input-remapper/presets/gamepad/a.json~

**** Macros
***** Brief
input-remapper comes with an optional custom macro language with support for cross-device variables, conditions and named parameters.

Multiple functions are chained using ~.~ a dot.

***** Key
Acts like a pressed key. All names that are available in regular mappings can be used here.

#+begin_src js

  key(symbol: str)

  key(symbol=KEY_A)
  key(b).key(space)

#+end_src

***** key_down and key_up
Inject the press/down/1 and release/up/0 events individually with those macros.

#+begin_src javascript

  key_down(symbol: str)
  key_up(symbol: str)

  key_down(KEY_A)
  key_up(KEY_B)

#+end_src

***** wait
Waits in milliseconds before continuing the macro. If the max_time argument is provided, it will randomize the time between time and max_time.

#+begin_src javascript

  wait(time: int, max_time: int | None)

  wait(time=100)
  wait(500)
  wait(10, 1000)

#+end_src

***** repeat
Repeats the execution of the second parameter a few times

#+begin_src javascript

  repeat(repeats: int, macro: Macro)

  repeat(1, key(KEY_A))
  repeat(repeats=2, key(space))

#+end_src

***** modify
Holds a modifier while executing the second parameter

#+begin_src javascript

  modify(modifier: str, macro: Macro)

  modify(Control_L, key(a).key(x))

#+end_src

**** Troubleshooting
If your key is hanging due to a macro, unplug your device, and then plug it back in. This should reset the key.

If stuff doesn't work, check the output of ~input-remapper-gtk -d~.

If ~input-remapper~ or your presets prevents your input device from working at all due to autoload, please try to unplug and plug it in twice. No injection should be running anymore.

*UI Shortcuts*
- ~ctrl + del~ stops the injection (only works while the gui is in focus)
- ~ctrl + q~ closes the application
- ~ctrl + r~ refreshes the device list

**** Short Macro Examples

- ~key(BTN_LEFT)~ a single mouse-click
- ~key(1).key(2)~ 1, 2
- ~wheel(down, 10)~ ~wheel(up, 10)~ Scroll while the input is pressed.
- ~mouse(left, 5)~ ~mouse(right, 2)~ ~mouse(up, 1)~ ~mouse(down, 3)~ Move the cursor while the input is pressed.
- ~repeat(3, key(a).w(500))~ a, a, a with 500ms pause
- ~modify(Control_L, key(a).key(x))~ CTRL + a, CTRL + x
- ~key(1).hold(key(2)).key(3)~ writes 1 2 2 ... 2 2 3 while the key is pressed
- ~event(EV_REL, REL_X, 10)~ moves the mouse cursor 10px to the right
- ~mouse(right, 4)~ which keeps moving the mouse while pressed
- ~wheel(down, 1)~ keeps scrolling down while held
- ~set(foo, 1)~ set [[https://en.wikipedia.org/wiki/Metasyntactic_variable]["foo"]] to 1
- ~if_eq($foo, 1, key(x), key(y))~ if "foo" is 1, write x, otherwise y
- ~hold()~ does nothing as long as your key is held down
- ~hold_keys(a)~ holds down "a" as long as the key is pressed, just like a regular non-macro mapping
- ~if_tap(key(a), key(b))~ writes a if the key is tapped, otherwise b
- ~if_tap(key(a), key(b), 1000)~ writes a if the key is released within a second, otherwise b
- ~if_single(key(a), key(b))~ writes b if another key is pressed, or a if the key is released
  and no other key was pressed in the meantime.
- ~if_tap(if_tap(key(a), key(b)), key(c))~ "a" if tapped twice, "b" if tapped once and "c" if
  held down long enough
- ~key_up(a).wait(1000).key_down(a)~ keeps a pressed for one second
- ~hold_keys(Control_L, a)~ holds down those two keys
- ~key(BTN_LEFT).wait(100).key(BTN_LEFT)~ a double-click

**** Personal examples
***** A triple-click
To select text in browsers and other apps.

#+begin_src javascript

  key(BTN_LEFT).wait(100).key(BTN_LEFT).wait(100).key(BTN_LEFT)

#+end_src

***** Key macro on a mouse
Put a keybinding to one of the extra mouse buttons to toggle the auto-click feature. In this case, it's the Toggle dwell command, which is bound to ~Super + Shift + Meta z~.

#+begin_src javascript

  // Button EXTRA   ->
  key(Super_L).key(Shift_L).key(Alt_L).key(z)

#+end_src

***** Action, switch app, and action
Google Translate speaks, then switches to Emacs and fold the header

#+begin_src javascript

  modify(Alt_L, key(x)).
  wait(50).
  key(KEY_F5).
  wait(50).
  key(KEY_TAB)

#+end_src

***** Switch apps and do something
This macro works to copy the translation of a sentence from Google Translate in Chromium and paste it into Emacs as a org heading.

#+begin_src javascript

  // First select text in Google Translate in Chromium
  // Switch to Emacs
  modify(Super_L, key(1)).
  wait(50).
  // org-insert-heading-respect-content
  modify(Control_L, key(Return)).
  wait(50).
  // org-yank
  modify(Control_L, key(y)).
  wait(50).
  // Switch to Cheomium
  modify(Super_L, key(3)).
  wait(50).
  // Copy translation
  modify(Alt_L, key(c)).
  wait(50).
  // Switch to Emacs
  modify(Super_L, key(1)).
  wait(50).
  // org-backward-heading-same-level
  modify(Control_L, key(c).key(b)).
  wait(50).
  // org-forward-heading-same-level
  modify(Control_L, key(c).key(f)).
  wait(50).
  // right-word
  modify(Control_L, key(Right)).
  wait(50).
  // left-word
  modify(Control_L, key(Left)).
  wait(50).
  // org-yank
  modify(Control_L, key(y)).
  wait(50).
  // org-return
  key(Return)

#+end_src

***** Swap buttons on the touchpad
Palce the middle button in the position of the left button. It's useful to be able to use the middle click on the touch pad.

#+begin_src javascript

  Button LEFT   ->   BTN_MIDDLE

#+end_src

***** Holding down the key + type another key
The =NumLock= key is set to =C-w= (close window), and we want to set the =Shift= + =NumLock= macro to =C-S-t= (reopen last tab).

This command would work  ~key(Shift_L).key(Control_L).key(t)~, but since we would already be holding down the =Shift= key, we remove it ~key(Control_L).key(t)~. Now since we would be holding down two modifiers together (=Shift= and =Ctrl=) they are locked, so we need to release them by pressing each one again, so the command would look like this:

#+begin_src javascript

  key(Control_L).key(t).key(Control_L).key(Shift_L)

#+end_src

***** Compose key sequence to write the ñ in Spanish
To type ~ñ~ using the compose key, we need to type the compose key (=Multi_key=), then ~~~, and then ~n~.

*Note*: the =Multi_Key= (Compose key) is already set to =Multi_key + KEY_APOSTROPHE=, maybe that's why in some apps we need to hold the =Multi_key= and quickly press ~n~ twice to type ~ñ~.

#+begin_src javascript

  key(Multi_key).key(Shift_L).key(grave).key(n)

#+end_src

* References
- [[https://archive.org/details/howlinuxworkswha0000ward][How Linux Works by Brian Ward]]
- [[https://www.debian.org/doc/manuals/debian-reference/][Debian Reference]]
- [[https://www.gnu.org/manual/manual.html][GNU Manuals Online]]
- [[https://www.wikipedia.org/][Wikipedia]]
- [[https://linuxcommand.org/tlcl.php][The Linux Command Line by William Shotts]]
- [[https://www.man7.org/linux/man-pages/index.html][Linux man pages]]