Shells starting very slow on my ubuntu and mac

My shells were starting very slow which is really bad as a dev/admin. I found out that all those version managers I installed were to blame for that. So I decided to get rid of nvm and rvm by simply removing the directories and cleaning up my .bashrc and .profile in $HOME.

#nvm
rm -rf ~/.nvm
rm -rf ~/.npm
rm -rf ~/.bower
rm .zshrc # i am using bash

#rvm
rm -rf .rvm

# cleanup configs
vim ~/.bashrc ~/.profile

After those simple steps my shell is up within an instand instead of needing some thinking time.

sdkman and brew seem to be ok and are allowed to stay installed for now.

Uninstall snapd from ubuntu 20.04

In my last post I found out who was to blame for filling my complete disk. It was this new and fu**** slow snapd added to ubuntu in the last few moments before release to get into the app market business. I decided to live without this feature and uninstalled it:

snap list | awk '{print $1}' | xargs -rn1 sudo snap remove

After letting this run a few times i tried to uninstall the remaining snaps manually with

sudo snap remove <name>

Some refused to uninstall but I continued with unmounting its volume and getting rid of snapd from the system:

sudo umount /snap/core/<replace_with_number_in_your_folder>
sudo apt purge snapd

In the end i cleaned up any stuff left from snapd:

rm -rf ~/snap
sudo rm -rf /snap
sudo rm -rf /var/snap
sudo rm -rf /var/lib/snapd

In the end removing all snaps freed about 20GB of data. I will reinstall everything based on apt as usual and will then be back at maybe 15GB of saved space without snapd.

Next to this space problem all the apt installed applications start seconds faster.

Thanks for the long years of great operating system but that’s a way I won’t go with you.

Remove old/disabled snaps from the system – ubuntu why…

The new snap system starts eating my system partition. Next to being slow it also holds old backups of upgraded snaps. Ubuntu itself has no option to disable that. You can only lower the number of old versions by using

sudo snap set system refresh.retain=2

which will limit the number to one live and one backup copy. But what if you also don’t need or want that backup copy? refresh.retain doesn’t allow 1:

sudo snap set system refresh.retain=1

error: cannot perform the following tasks:
- Run configure hook of "core" snap (run hook "configure": retain must be a number between 2 and 20, not "1")

I didn’t further check how to remove that but used the following one-liner to get rid of old versions:

sudo su
snap list --all | awk '$6~"disabled"{print $1" --revision "$3}' | xargs -rn3 snap remove

Next to those backups also the /var/lib/snapd/cache folder holds a massive amount of space on my machine. I considered it to be safe to delete:

sudo rm -f /var/lib/snapd/cache/*

This cleanup of snapd folder freed around 12GB of data on my root partition.

Stop firefox from changing your URLs automatically to https

As an admin I have to work with http and https sites quite often. Firefox started to put https in front of everything automatically and clearing the cache all the time is not what I wanted to continue. I found a quick workaround that avoids this issue. Open a new tab and enter about:config in the address bar and hit enter. Confirm the warning and the look for browser.urlbar.autoFill and disable it. With this set to false Firefox stopped that annoying behavior for me.

Get script directory in Bash

If you need to get the script directory of an executed bash script you can use the following snippet to cd to the script directory:

#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR" || exit 1

Add docker to unattended-upgrades in ubuntu 20.04

If you want your server to install docker updates automatically then you need to add the docker repository to the list of allowed origins. Otherwise docker updates will stay untouched by unattended-update which will looks similar to this when you login and check which updates need to be installed. Had that quite some times until I decided to look into this.

sudo apt dist-upgrade 
...
The following packages will be upgraded:
  containerd.io docker-ce docker-ce-cli

As you can see all docker packages (coming from https://download.docker.com/linux/ubuntu) weren’t updated. This happens because this origin isn’t in the allowed origin list of unattended-upgrade. You can see metadata of the repository by running

apt-cache policy

There you will find an entry like this

 500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     release o=Docker,a=focal,l=Docker CE,c=stable,b=amd64
     origin download.docker.com

The important part is the o and a in this definition. Those state the origin and the archive.

To allow the docker repo as an origin you need to open /etc/apt/apt.conf.d/50unattended-upgrades with root access and add

"Docker:${distro_codename}";

to the list in Unattended-Upgrade::Allowed-Origins. The syntax is short for “origin:archive”.

Here the allowed origins list from my file as an example:

Unattended-Upgrade::Allowed-Origins {
        "${distro_id}:${distro_codename}";
        "${distro_id}:${distro_codename}-security";
        "${distro_id}ESMApps:${distro_codename}-apps-security";
        "${distro_id}ESM:${distro_codename}-infra-security";
        "${distro_id}:${distro_codename}-updates";
        "Docker:${distro_codename}";
};

I replaced the archive name focal with the variable distro_codename like the existing examples also did. This will help in a future dist upgrade if you plan to upgrade your os for example with the next lts version.

With those changes in place unattended-upgrade should also install updates for docker from now on.

The same procedure can be followed to add other repositories to this list as well.

Teamspeak server

If you want to host your own teamspeak server this can be done quite easily with a server and docker-compose. Here is the example I tried:

version: '3.1'
services:
  teamspeak:
    image: teamspeak
    restart: always
    ports:
      - 9987:9987/udp
      - 10011:10011
      - 30033:30033
    environment:
      TS3SERVER_DB_PLUGIN: ts3db_mariadb
      TS3SERVER_DB_SQLCREATEPATH: create_mariadb
      TS3SERVER_DB_HOST: db
      TS3SERVER_DB_USER: root
      TS3SERVER_DB_PASSWORD: tsmysqlpass
      TS3SERVER_DB_NAME: teamspeak
      TS3SERVER_DB_WAITUNTILREADY: 30
      TS3SERVER_LICENSE: accept
  db:
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: tsmysqlpass
      MYSQL_DATABASE: teamspeak

You will need a dns entry or the ip of the server to connect with your teamspeak client. With the client you can then add a password to protect the server. I didn’t find a docker config to set it.

ATTENTION: If you don’t secure the server with credentials it will be open for everybody.

Nextcloud fix installation

After some updates the installation/the database needs to be fixed. This can be done with the command line tool on a debian system where the installation belongs to the apache user www-data:

cd <installdir>
sudo -u www-data php occ db:add-missing-indices
sudo -u www-data php occ db:add-missing-columns

Resize mp4 with ffmpeg

Zoom didn’t want to accept a background video so I tried to find a way to shrink its size to 1080p. There is a simple command to do that with ffmpeg:

ffmpeg -i original.mp4 -vf scale=1920:1080 new_1080p.mp4

DaVinci Resolve 16 Linux mp4/h.264 encode/decode

DaVinci Resolve doesn’t support import or export of mp4 format on Linux. It is working on Windows 10, but I really don’t want to switch to boot into Windows for rendering.

So i looked around and found a workflow for using Resolve under Linux: ffmpeg is my tool of choice to change formats so that Resolve is able to load and write my videos.

My recordings from OBS are saved as mkv as proposed by an OBS warning in the recording settings.

mkv to mov

ffmpeg -i input.mkv -map 0:0 -map 0:1 -map 0:2 -vcodec dnxhd -acodec:0 pcm_s16le -acodec:1 pcm_s16le -s 1920x1080 -r 30000/1001 -b:v 36M -pix_fmt yuv422p -f mov output.mov

After converting the recording to the mov format DaVinci Resolve 16 is able to use the file with all audio tracks.

mp4 to mov

If I get videos from other sources most of the time its in mp4 format. With the following command I convert the mp4 to DaVinci Resolve usable mov:

ffmpeg -i input.mp4 -vcodec dnxhd -acodec pcm_s16le -s 1920x1080 -r 30000/1001 -b:v 36M -pix_fmt yuv422p -f mov output.mov

mov to mp4

In DaVinci Resolve 16 I export my videos with mov format and then convert them to youtube ready mp4 with this command:

ffmpeg -i input.mov -vf yadif -codec:v libx264 -crf 1 -bf 2 -flags +cgop -pix_fmt yuv420p -codec:a aac -strict -2 -b:a 384k -r:a 48000 -movflags faststart output.mp4

TLDR

DaVinci Resolve 16 does NOT support h.264 en-/decoding on Linux and you have to transcode your videos for example with ffmpeg as shown above. Transcoding is quite fast also on my 6 year old machine.