Categories
Bash

safer Bash scripts

TL;DR

You should write the following at the beginning of every script:

set -euo pipefail
shopt -s lastpipe

Long version

Bash is a common used script language, if it comes to write some shell scripts for Unix. To make your scripts safer for common mistakes you should set some options of Bash in your script to prevent unexpected behavior and to enable some stricter rules.

set -e

If you set this option, Bash will immediately stop execution of your script if an error occurs.

The script:

#!/bin/bash

unknowncommand
echo "my second line"

will output with an return code of 0:

./test.sh: line 3: unknowncommand: command not found
my second output

If you add set -e at the beginning:

#!/bin/bash

set -e

unknowncommand
echo "my second line"

you will get the following output with an return code of 127.

./test.sh: line 5: unknowncommand: command not found

So you can trust that if a command like mkdir or cd for example will fail, your script will stop.

If you want to allow a command to fail, you have to add || true, for example:

#!/bin/bash

set -e

unknowncommand || true
echo "my second line"

set -u

This option will prevent you from using undefined variables.

The script:

#!/bin/bash

echo "Hello $NAME"

will output:

Hello

Bash handle undefined variables as empty variables per default. If you add set -u at the beginning of your script:

#!/bin/bash

set -u

echo "Hello $NAME"

you will get the following output:

./test.sh: line 5: NAME: unbound variable

If you are not sure, if the variable exists, you can use the syntax for default values:

#!/bin/bash

set -u

echo "Hello ${NAME:-World}"

set -o pipefail

If you are using pipes, only the last command will define the result of the whole pipe. So if you run this script:

#!/bin/bash

set -eu

unknowncommand | echo "other command"
echo "after command"

you will get the following output:

./test.sh: line 5: unknowncommand: command not found
other command
after command

If you add set -o pipefail at the beginning of your script:

#!/bin/bash

set -euo pipefail

unknowncommand | echo "other command"
echo "after command"

the script will stop after the failed pipe and you get the output:

./test.sh: line 5: unknowncommand: command not found

shopt -s lastpipe

If you run a script in a pipe and want to set variables, this won’t work, because your command is run in a sub shell and then you cannot change variables.

The following script:

#!/bin/bash

set -euo pipefail

VAR=Hello
echo "some" | { while read -r file ; do VAR=World; done }
echo $VAR

would output:

Hello

If you add shopt -s lastpipe at the beginning of your script:

#!/bin/bash

set -euo pipefail
shopt -s lastpipe

VAR=Hello
echo "some" | { while read -r file ; do VAR=World; done }
echo $VAR

the script would output:

World
Categories
Linux Networking

use SSH agent sockets with tmux

Tmux is a nice helper for having multiple parallel shells running, especially using SSH connections. Additionally your shells remain running after connection lost. But if you also using SSH agents for using your SSH keys also on other servers, you have a problem with reconnecting to such tmux sessions.

If you reconnect, the unix socket to your SSH agent will change and because the shell is reused respectively reattached, your SSH agent will no operate as expected.

So you have to change the environment variable after reconnect to a new value. This is quite not easy, especially for already running programs. So it is easier to symlink your current unix socket to a specific file and update only the symlink on reconnect.

To achieve this, you should add the following code to your .bashrc (or whatever shell you are using).

if [[ ! -z "$SSH_AUTH_SOCK" ]] ; then
    if [[ -S "$SSH_AUTH_SOCK" && $(basename "$SSH_AUTH_SOCK") != "localauthsock" ]] ; then
        ln -sf "$SSH_AUTH_SOCK" ~/.ssh/localauthsock
    fi
fi
export SSH_AUTH_SOCK=$HOME/.ssh/localauthsock
Categories
HTML/CSS

automatically count headers via css

CSS 2.1 introduced counters which can be used to auto number your headings or other elements. You can define a counter, which will incremented for every element of a specific class or it will reset an other counter.

A simple example would be to automatic prefix all h2 and h3 tags with automatic numbering. The counter will be used in a :before pseudo element for output.

h1 {
  counter-reset: h2counter;
}

h2:before {
  content: counter(h2counter) ".\0000a0\0000a0";
  counter-increment: h2counter;
  counter-reset: h3counter;
}

h3:before {
  content: counter(h2counter)"."counter(h3counter) ".\0000a0\0000a0";
  counter-increment: h3counter;
  counter-reset: h4counter;
}

“\0000a0” is the NO-BREAK-SPACE Unicode Character.

Categories
Linux

use mount in udev rule with systemd-udevd

If you write your own scripts called by udevd, for example after plugin a specific USB stick, you maybe want to mount a partition, which could fail. I got often return code 32 from mount, which means “mount failure” but without any additional error message.

This happens, because systemd-udevd is started in its own namespace for mounting. So the filesystem of udevd differs from the main system. You should be aware, that running udevd in an own namespace increases security. But if you want to disable this to run your scripts, do the following steps.

First verify the current MountFlags settings of udevd service. It should be “slave”.

systemctl show systemd-udevd | grep -F MountFlags

Now edit the udevd service

systemctl edit systemd-udevd

and set MountFlags to shared:

[Service]
MountFlags=shared

At last you just have to reload systemd settings and restart the udevd daemon:

systemctl daemon-reload
systemctl restart systemd-udevd

You then also can verify that udevd didn’t use the namespace anymore and list all current mount namespaces in use:

lsns -t mnt
Categories
Java

velocity with custom loader

Velocity is a feature rich template engine for Java applications. This engine is mainly designed to have template files which then will be rendered into OutputStreams using a context. But you can also extend velocity to use in memory or database stored templates.

First you have to register your own ResourceLoader at the VelocityEngine:

final VelocityEngine ve = new VelocityEngine();
ve.setProperty("yamltemplates.resource.loader.class", YamlTemplateResourceLoader.class.getName());
ve.setProperty("yamltemplates.resource.loader.templatefile", templateFile);
ve.setProperty(VelocityEngine.RESOURCE_LOADER, "yamltemplates");
ve.setProperty(VelocityEngine.INPUT_ENCODING, "UTF-8");
ve.init();

Then you can implement your ResourceLoader:

public class YamlTemplateResourceLoader extends ResourceLoader {

    private Map<String, Object> templateData;

    @Override
    public void init(final ExtProperties configuration)
    {
        try
        {
            try (InputStream data = (InputStream) configuration.get("templatefile"))
            {
                Yaml yaml = new Yaml();
                templateData = yaml.load(data);
            }
        }
        catch (IOException e)
        {
            throw new RuntimeException(e);
        }
    }

    @Override
    public Reader getResourceReader(final String source, final String encoding) throws ResourceNotFoundException
    {
        try
        {
            if ("velocimacros.vtl".equals(source))
            {
                return buildReader(new ByteArrayInputStream(new byte[0]), encoding);
            }
            Object template = templateData.get(source);
            if (template == null)
            {
                throw new ResourceNotFoundException(source);
            }
            if (template instanceof String)
            {
                return new StringReader((String) template);
            }
            throw new RuntimeException("cannot handle " + template);
        }
        catch (IOException e)
        {
            throw new RuntimeException(e);
        }
    }

    @Override
    public boolean isSourceModified(final Resource resource)
    {
        return false;
    }

    @Override
    public long getLastModified(final Resource resource)
    {
        return 0;
    }
}

Last but not least you can render your templates like any other templates with Velocity:

StringWriter stringWriter = new StringWriter();
VelocityContext context = new VelocityContext();
context.put("myparam", "Hello World!");
contextSetup.setupContext(context);
ve.getTemplate(templateName).merge(context, stringWriter);
String result = stringWriter.toString();

}

Categories
Linux

Preventing window resizing after resume

I have a laptop with high DPI display (HiDPI). I use scaling factor 100%, but login and lock screen of GDM uses 200 % because of auto detection. This is the second error concerning GDM and HiDPI displays. After unlock my system (here Manjaro) I got the following screen:

Then I had to minimize and restore the window to get a correctly rendered window.

The solution is to set the scaling factor in GDM to the same as in your user session. You can check the current setting with:

gsettings get org.gnome.desktop.interface scaling-factor

0 means auto scaling, 1 means 100%, 2 means 200% and so on.

So you can change this value from auto detection to a static value, in my case 1 for 100% scaling factor:

gsettings set org.gnome.desktop.interface scaling-factor 1
Categories
Linux

output systemd daemon to console

Per default setting all services log only to journal and not to console. You can check the settings of a service with:

sudo systemctl show MyService.service | grep -Fi Standard

And you should get something like.

StandardInput=null
StandardInputData=
StandardOutput=journal
StandardError=inherit

If you want to change it, you have to set StandardOutput to journal+console.

The full options are documented in the man pages. You can look into man page of systemd.directives:

man systemd.directives

Search for StandardOutput and you will find a reference to the man page of systemd.exec. There you will find every option of this directive.

To change it you should call:

sudo systemctl edit MyService.service

and then add or change the line of StandardOutput to

[Service]
StandardOutput=journal+console

After all you should reload systemd configuration:

sudo systemctl daemon-reload
Categories
Java

JPA / JQL + QueryDSL string replace

Sometimes you need a string replace in SQL query, for example to compare multiple values each other which need to manipulate before comparing.

Unfortunately JPA did not support any replace function, so you have to use database specific functions. You should read the documentation of your database server how you can replace strings via SQL. For PostgreSQL you can use the replace function.

To use custom functions with QueryDSL, you have to use templates as described in a previous blog post about difference of dates in JPA.

final StringTemplate convertedField1 = Expressions.stringTemplate("function('replace', {0}, '/', '-')", model.field1);

Then you can use this template in your query (for example compare replaced field1 with field2 of your model):

new JPAQuery<>()
…
.where(convertedField1.eq(model.field2))