9.2 KiB
9.2 KiB
onyx-dev - Container Server of nbit Informatik GmbH
onyx-dev.nbit.ch is used to run rootless Podman containers, using Traefik as a Reverse Proxy
It is the Development Server, the main server is onyx.nbit.ch
Additionally it acts as a primary nameserver ns1.nbit.ch
In this directory (/data/onyx-dev_pods), you will find all configuration files to run the containers (with Podman and Kubernetes YAML files)
Specs:
- Rocky Linux 10
- Hetzner Cloud Server CX21
- 2 vCPUs
- 4 GB RAM
- 40 GB Disk
- Location Nuremberg
Persistent data is stored in /data
Create Server
Name: onyx-dev.nbit.ch
Set Root-Password (by hand)
`
dnf update
init 6 (Reboot)
groupadd containers
useradd -m -g containers containers
passwd containers
For containers User, add the following to ~/.bash_profile:
if [ -z "$XDG_RUNTIME_DIR" ] && [ -d "/run/user/$(id -u)" ]; then export XDG_RUNTIME_DIR=/run/user/$(id -u) fi
hostnamectl set-hostname onyx-dev.nbit.ch
timedatectl set-timezone Europe/Zurich
dnf install glibc-langpack-de
localectl set-locale LANG=en_US.UTF-8
localectl set-locale LC_TIME=de_CH.UTF-8
Set Journalctl-Config to persistent Storage: cp /usr/lib/systemd/journald.conf /etc/systemd /etc/systemd/journald.conf: Storage=persistent
Enable Selinux in Enforcing Mode: change /etc/selinux/config
dnf install setroubleshoot-server
Disable kdump: systemctl disable --now kdump grubby --update-kernel=ALL --remove-args="crashkernel" reboot `
enable EPEL Repo (for instance because of fail2ban): `
dnf install epel-release
`
Firewall
# dnf install firewalld
# systemctl enable --now firewalld
# firewall-cmd --add-service={http,https} --permanent
# firewall-cmd --remove-service=cockpit --permanent
# firewall-cmd --reload
List Rules:
# firewall-cmd --list-all
fail2ban on Host for ssh
# dnf install fail2ban
# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
edit /etc/fail2ban/jail.local:
enabled = true => below [sshd]
# systemctl enable fail2ban
# systemctl restart fail2ban
command to check who is banned:
# fail2ban-client status sshd
Install Software
# dnf install passt
# dnf install git
# dnf install podman
# dnf install jq
# dnf install sysstat
# dnf install lftp
# dnf install telnet
# dnf install binutils
Setup Mail
# dnf install s-nail procmail msmtp
# cp /usr/share/doc/msmtp/msmtprc-system.example /etc/msmtprc
Backup Server
# dnf install restic
# mkdir /backup
# mkdir /backup-restic
# restic init --repo /backup-restic/restic-repo-$(hostname --short) # Passwort in Keepass
SSH Keypaar fuer User root erstellen und auf Hetzner Storagebox hinterlegen:
# ssh-keygen
Restic Script:
/usr/local/bin/backup-to-disk.sh
#!/bin/bash
# Backup der wichtigsten Verzeichnisse nach einem Verzeichnis
#
# Es wird restic verwendet.
#
PATH=$PATH:/usr/local/bin
export RESTIC_PASSWORD="$(hostname --short)7355"
restic backup --quiet --repo /backup-restic/restic-repo-$(hostname --short) /home /etc /var /opt /data /usr/local/bin /backup --exclude=/var/log
if [ $? -eq 0 ]; then
restic forget --quiet --repo /backup-restic/restic-repo-$(hostname --short) --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 20 --prune
else
>&2 echo "Problem with restic Backup $(hostname --short)"
fi
/etc/cron.d/backup-to-disk:
#
# Backup important Files to Disk
#
55 6 * * * root /usr/local/bin/backup-to-disk.sh >/dev/null
Backup auf Storag Box:
# cat > /etc/cron.d/rsync-backup-to-other-host <<HERE
#
# Rsync /backup-restic to backup space
#
20 7 * * * root /usr/bin/rsync -avzH --delete --numeric-ids -e 'ssh -p23' /backup-restic u152662@u152662.your-storagebox.de:onyx-backup-restic-rsync >/dev/null
HERE
List Usage on Storage Box:
# ssh -p 23 u152662@u152662.your-storagebox.de du -h -d 1
261G ./arq-backup-mbp
8.3G ./moby-backup-restic-rsync
650M ./onyx-dev-backup-restic-rsync
1.6G ./mb2-backup-rsync
1.2G ./wobisch2-backup-rsync
1.6G ./mb1-backup-rsync
11G ./mail-backup-restic-rsync
20G ./onyx-backup-restic-rsync
44G ./cinnamon.nbit.ch
10K ./.ssh
348G .
Setup Env for Podman
# mkdir /data
# chown containers:containers /data
Set Defaults:
containers$ cat ~/.config/containers/containers.conf
[network]
network_backend = "netavark"
Credentials for Docker Registry should be available after Reboot
# echo 'net.ipv4.ip_unprivileged_port_start=80' >> /etc/sysctl.d/containers.conf
# loginctl enable-linger containers
[containers@onyx-dev ~]$ podman login git.nbit.ch
Username: jlehmann
Password:
Login Succeeded!
[containers@onyx-dev ~]$ mkdir ~/.config/containers
[containers@onyx-dev ~]$ cp /run/user/1000/containers/auth.json ~/.config/containers/auth.json
Setup Traefik
Traefik will be started with podman play kube (with yaml file) and attach to HostNetwork (hostNetwork: true in yaml). Backends will map there port and will be accessed on localhost:
we do not use the dynamic config using docker provider, but rather the file provider (one file per backend)
Example Backend Service File:
[containers@onyx onyx_pods]$ cat traefik/configuration/nbitwebsite.yml
http:
routers:
nbitwebsite:
entrypoints:
- websecure
tls:
certresolver: "myresolver"
domains:
- main: "www.linux-freelancer.ch"
sans: "linux-freelancer.ch"
rule: "Host(`linux-freelancer.ch`,`www.linux-freelancer.ch`)"
service: nbitwebsite
services:
nbitwebsite:
loadBalancer:
servers:
- url: http://127.0.0.1:9000/
passHostHeader: false
# echo 'net.ipv4.ip_unprivileged_port_start=80' >> /etc/sysctl.d/containers.conf
# loginctl enable-linger containers
# The following fixes "Failed to connect to bus: No medium found"
export XDG_RUNTIME_DIR=/run/user/$(id -u)
containers$ touch /data/traefik/acme.json
containers$ chmod 0600 /data/traefik/acme.json
Important: set SELinux Context on /data for containers:
[root@onyx-dev named]# semanage fcontext -a -s unconfined_u -t container_file_t "/data(/.*)?"
[root@onyx-dev named]# restorecon -r -v /data
Check:
find /data ! -context '*:container_file_t:*'
/etc/subgid:containers:100000:65536
/etc/subuid:containers:100000:65536
!!!!!!!!!!!!!!
IMPORTANT: we changed the startup Method to Quadlets, put into
~containers/.config/containers/systemd/
See also https://www.redhat.com/sysadmin/quadlet-podman
Example of such a quadlet config (Network=pasta is not to be used for Traefik):
[containers@onyx-dev ~]$ more ~containers/.config/containers/systemd/onyx-dev-mqtt.kube
[Unit]
Description=MQTT
Before=local-fs.target
[Kube]
Yaml=/home/containers/onyx-dev_pods/mqtt/mqtt.yaml
LogDriver=journald
Network=pasta
[Service]
Restart=always
RestartSec=15
TimeoutStartSec=300
[Install]
# Start by default on boot
WantedBy=multi-user.target default.target
!!!!!!!!!!!!!!
Setup Backend Services
Create Kubernetes YAML File:
backendservice=nbitwebsite
containers$ mkdir ~/onyx_pods/${backendservice}
Create File ~/onyx_pods/${backendservice}/${backendservice}.yaml
Postfix
# dnf install postfix
We do not want to accept emails from outside, so we do not change the config (we only listen on localhost)
# systemctl enable postfix
# systemctl start postfix
Creat DNS Records:
mx1.nbit.ch and mx2.nbit.ch (A/AAAA Records to the same IPs as onyx and onyx-dev)
Change SPF Records to include mx1 and mx2
Primary Nameserver
# dnf install bind bind-utils
# systemctl enable --now named
# firewall-cmd --add-service=dns --permanent
# firewall-cmd --reload
Adopt /etc/named.conf:
Important => recursion no
[root@onyx-dev etc]# diff named.conf named.conf.6nov2026
11,12c11,12
< listen-on port 53 { any; };
< listen-on-v6 port 53 { any; };
---
> listen-on port 53 { 127.0.0.1; };
> listen-on-v6 port 53 { ::1; };
19,20c19
< allow-query { any; };
< version "You are too curious";
---
> allow-query { localhost; };
32c31
< recursion no;
---
> recursion yes;
61,124d59
< acl slaves {
< 95.216.148.212;
< 2a01:4f9:c010:5dd::1;
< 23.88.33.113;
< 2a01:4f8:c010:90e1::1;
< 5.75.251.111;
< 2a01:4f8:c012:c21c::1;
< 127.0.0.1;
< ::1;
< };
<
< zone "nbit.ch" IN {
< type master;
< file "master/nbit.ch.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
<
< zone "mini-beieli.ch" IN {
< type master;
< file "master/mini-beieli.ch.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
<
< zone "wo-bisch.ch" IN {
< type master;
< file "master/wo-bisch.ch.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
<
< zone "ch-wirth.ch" IN {
< type master;
< file "master/ch-wirth.ch.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
<
< zone "cmoag.com" IN {
< type master;
< file "master/cmoag.com.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
<
< zone "acmoag.com" IN {
< type master;
< file "master/acmoag.com.zone";
< allow-update { none; };
< allow-transfer {
< slaves;
< };
< };
on master:
[root@onyx-dev named]# semanage fcontext -a -t named_cache_t "/var/named/master(/.*)?"
[root@onyx-dev named]# restorecon -r -v /var/named/master
...