Categories
Bash

safer Bash scripts

TL;DR

You should write the following at the beginning of every script:

set -euo pipefail
shopt -s lastpipe

Long version

Bash is a common used script language, if it comes to write some shell scripts for Unix. To make your scripts safer for common mistakes you should set some options of Bash in your script to prevent unexpected behavior and to enable some stricter rules.

set -e

If you set this option, Bash will immediately stop execution of your script if an error occurs.

The script:

#!/bin/bash

unknowncommand
echo "my second line"

will output with an return code of 0:

./test.sh: line 3: unknowncommand: command not found
my second output

If you add set -e at the beginning:

#!/bin/bash

set -e

unknowncommand
echo "my second line"

you will get the following output with an return code of 127.

./test.sh: line 5: unknowncommand: command not found

So you can trust that if a command like mkdir or cd for example will fail, your script will stop.

If you want to allow a command to fail, you have to add || true, for example:

#!/bin/bash

set -e

unknowncommand || true
echo "my second line"

set -u

This option will prevent you from using undefined variables.

The script:

#!/bin/bash

echo "Hello $NAME"

will output:

Hello

Bash handle undefined variables as empty variables per default. If you add set -u at the beginning of your script:

#!/bin/bash

set -u

echo "Hello $NAME"

you will get the following output:

./test.sh: line 5: NAME: unbound variable

If you are not sure, if the variable exists, you can use the syntax for default values:

#!/bin/bash

set -u

echo "Hello ${NAME:-World}"

set -o pipefail

If you are using pipes, only the last command will define the result of the whole pipe. So if you run this script:

#!/bin/bash

set -eu

unknowncommand | echo "other command"
echo "after command"

you will get the following output:

./test.sh: line 5: unknowncommand: command not found
other command
after command

If you add set -o pipefail at the beginning of your script:

#!/bin/bash

set -euo pipefail

unknowncommand | echo "other command"
echo "after command"

the script will stop after the failed pipe and you get the output:

./test.sh: line 5: unknowncommand: command not found

shopt -s lastpipe

If you run a script in a pipe and want to set variables, this won’t work, because your command is run in a sub shell and then you cannot change variables.

The following script:

#!/bin/bash

set -euo pipefail

VAR=Hello
echo "some" | { while read -r file ; do VAR=World; done }
echo $VAR

would output:

Hello

If you add shopt -s lastpipe at the beginning of your script:

#!/bin/bash

set -euo pipefail
shopt -s lastpipe

VAR=Hello
echo "some" | { while read -r file ; do VAR=World; done }
echo $VAR

the script would output:

World
Categories
Linux Networking

use SSH agent sockets with tmux

Tmux is a nice helper for having multiple parallel shells running, especially using SSH connections. Additionally your shells remain running after connection lost. But if you also using SSH agents for using your SSH keys also on other servers, you have a problem with reconnecting to such tmux sessions.

If you reconnect, the unix socket to your SSH agent will change and because the shell is reused respectively reattached, your SSH agent will no operate as expected.

So you have to change the environment variable after reconnect to a new value. This is quite not easy, especially for already running programs. So it is easier to symlink your current unix socket to a specific file and update only the symlink on reconnect.

To achieve this, you should add the following code to your .bashrc (or whatever shell you are using).

if [[ ! -z "$SSH_AUTH_SOCK" ]] ; then
    if [[ -S "$SSH_AUTH_SOCK" && $(basename "$SSH_AUTH_SOCK") != "localauthsock" ]] ; then
        ln -sf "$SSH_AUTH_SOCK" ~/.ssh/localauthsock
    fi
fi
export SSH_AUTH_SOCK=$HOME/.ssh/localauthsock
Categories
Linux

use mount in udev rule with systemd-udevd

If you write your own scripts called by udevd, for example after plugin a specific USB stick, you maybe want to mount a partition, which could fail. I got often return code 32 from mount, which means “mount failure” but without any additional error message.

This happens, because systemd-udevd is started in its own namespace for mounting. So the filesystem of udevd differs from the main system. You should be aware, that running udevd in an own namespace increases security. But if you want to disable this to run your scripts, do the following steps.

First verify the current MountFlags settings of udevd service. It should be “slave”.

systemctl show systemd-udevd | grep -F MountFlags

Now edit the udevd service

systemctl edit systemd-udevd

and set MountFlags to shared:

[Service]
MountFlags=shared

At last you just have to reload systemd settings and restart the udevd daemon:

systemctl daemon-reload
systemctl restart systemd-udevd

You then also can verify that udevd didn’t use the namespace anymore and list all current mount namespaces in use:

lsns -t mnt
Categories
Linux

Preventing window resizing after resume

I have a laptop with high DPI display (HiDPI). I use scaling factor 100%, but login and lock screen of GDM uses 200 % because of auto detection. This is the second error concerning GDM and HiDPI displays. After unlock my system (here Manjaro) I got the following screen:

Then I had to minimize and restore the window to get a correctly rendered window.

The solution is to set the scaling factor in GDM to the same as in your user session. You can check the current setting with:

gsettings get org.gnome.desktop.interface scaling-factor

0 means auto scaling, 1 means 100%, 2 means 200% and so on.

So you can change this value from auto detection to a static value, in my case 1 for 100% scaling factor:

gsettings set org.gnome.desktop.interface scaling-factor 1
Categories
Linux

output systemd daemon to console

Per default setting all services log only to journal and not to console. You can check the settings of a service with:

sudo systemctl show MyService.service | grep -Fi Standard

And you should get something like.

StandardInput=null
StandardInputData=
StandardOutput=journal
StandardError=inherit

If you want to change it, you have to set StandardOutput to journal+console.

The full options are documented in the man pages. You can look into man page of systemd.directives:

man systemd.directives

Search for StandardOutput and you will find a reference to the man page of systemd.exec. There you will find every option of this directive.

To change it you should call:

sudo systemctl edit MyService.service

and then add or change the line of StandardOutput to

[Service]
StandardOutput=journal+console

After all you should reload systemd configuration:

sudo systemctl daemon-reload
Categories
Linux Virtualization

analyze / mount virtual images (VDI) from VirtualBox

Sometimes you would like to look into a virtual image (VDI file) of VirtualBox without starting the VM. One solution could be to setup another VM and attach the VDI to the helper VM. An easier solution would be the analysis with standard Linux tools.

You can use the QEMU tools to mount your virtual disks. First you have to install these tools if not already present (here for Ubuntu):

apt install qemu-utils

Then you should create a snapshot of your virtual image, so that modifications are not written back to your image. If you need to manipulate a virtual image you can skip this step.

qemu-img create -f qcow2 -b VIRTUALDISK.vdi snapshot.qcow2

Then you will use qemu-nbd to create a block device of your virtual image. To use qemu-nbd you need to have to load the kernel module nbd.

sudo modprobe nbd
lsmod | grep -Fi nbd 

The last command should output something like:

nbd                    40960  0

Then you will connect the blockdevice /dev/nbd0 with your virtual image:

qemu-nbd -c /dev/nbd0 snapshot.qcow2

Then you can use it like every other disc. For example with fdisk or lsblk or you can mount a partition of the virtual image.

fdisk -lu /dev/nbd0
lsblk -i /dev/nbd0
mount /dev/nbd0p1 /mnt

After you finished your analysis, you have to disconnect the virtual image:

qemu-nbd -d /dev/nbd0

If you created a snapshot you can easily remove this file.

Categories
Linux

using sytem clipboard with tmux

If you are using tmux to handle your multiwindow console, you should be familiar with the copy text function (CTRL+A + [ then cursors and SPACE to select text then ENTER and insert with CTRL+Aa + ]) .

If you are also using the vim mode for navigating in copy mode you can set your Y key for yanking text into the system clipboard. So you can use ENTER for tmux internal copy and Y for copy to system clipboard.

First you should check that you have xclip installed, or you need to install it (here Ubuntu):

apt install xclip

Now you have to check your tmux version:

tmux -V

If you have tmux < 2.5, you have to add this to your tmux.conf (or enter it in live tmux session with CTRL+A + :)

bind -t vi-copy y copy-pipe 'xclip -in -selection clipboard'

If you have tmux >= 2.5, you have to add this to your tmux.conf:

bind -T copy-mode-vi y send-keys -X copy-pipe-and-cancel 'xclip -in -selection clipboard'

Of course this works by default only using a local tmux session.

Categories
Docker

configure docker network size

Docker uses by default /16 network for every network you create with it. This means you have 65534 available IPs in this network. Especially if you use docker-compose, you will create a network for every project / docker-compose file you run.

If you create to many networks, docker will use ip addresses from 192.168.0.0/16 or from 10.0.0.0/8. The chance this interference with your given network is then high. So you should think about the needed network size and adjust the default settings.

Because I mostly did not run many containers, but many projects and so many networks, I use a /27 network for every network. This means I have 30 available IPs. Because I typically have not more than 10 containers in one network, this is enough. You can also choose another network size depending on your needs:

Network SizeIPsavailable IPs
/273230
/266462
/25128126
/24256254

Now change your docker default config in /etc/docker/daemon.json. You have to create the file, if it didn’t exists. And add the setting default-address-pools:

{
    "default-address-pools":[
        {
            "scope": "local",
            "base":"172.17.0.0/16",
            "size":27
        }
    ]
}

You can choose as base everything you want, but please use only IPs reserved for private networks.