6.9 KiB

Raw Blame History

onyx - Container Server of nbit Informatik GmbH

In this directory (/home/containers/onyx_pods), you will find all configuration files to run the containers (with Podman and Kubernetes YAML files)

Persistent data is stored in /data

Specs:

Rocky Linux 9
Hetzner Cloud Server CX 31
- 2 vCPUs
- 8 GB RAM
- 80 GB Disk

Create Server

Name: onyx.nbit.ch

Set Root-Password (by hand)

dnf update

groupadd containers

useradd -m -g containers containers

passwd containers

hostnamectl set-hostname onyx.nbit.ch

enable EPEL Repo: `

dnf install epel-release

Firewall

# dnf install firewalld
# firewall-cmd --add-service={http,https} --permanent
# TODO

fail2ban on Host for ssh

# dnf install fail2ban
# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
edit /etc/fail2ban/jail.local:
enabled = true  => below [sshd]

# systemctl enable fail2ban
# systemctl restart fail2ban

command to check who is banned:
# fail2ban-client status sshd

Install Software

# dnf install git
# dnf install podman
# dnf install jq

Backup Server

TODO

Backup MySQL-DBs:

/usr/local/bin/backup-mysql-dbs.sh (sinngemaess, eine Zeile pro Container):
root@moby:/usr/local/bin# more backup-mysql-dbs.sh
#!/bin/bash
# Backup der MySQL DBs (Docker)
#
for container_name in $(docker ps --format "{{.Image}} {{.Names}}" |grep mysql  |awk '{print $2}'); do
  if [ -f /usr/local/bin/${container_name}.pwd ]; then
    # im pwd-File muss "PWD=XXXX" (root) gesetzt werden
    . /usr/local/bin/${container_name}.pwd
    docker exec ${container_name} /usr/bin/mysqldump -u root --password=${PWD} --all-databases > /backup/mysql-databases-${container_name}-$(date +%Y%m%W).sql 2>/dev/null
  else
    >&2 echo "Password must be set as PWD=XXXX in /usr/local/bin/${container_name}.pwd"
  fi
done

# Cleanup Old Backups
find /backup -type f -mtime +30 -exec rm {} \;

/etc/cron.d/backup-mysql-dbs:
# Backup MySQL DBs
#
45 5 * * * root /usr/local/bin/backup-mysql-dbs.sh >/dev/null


Restore: just in case:
cat backup.sql | docker exec -i CONTAINER /usr/bin/mysql -u root --password=root DATABASE


# dnf install restic
# mkdir /backup
# mkdir /backup-restic
# restic init --repo /backup-restic/restic-repo-$(hostname --short)      # Passwort in Keepass

SSH Keypaar fuer User root erstellen und auf Hetzner Storagebox hinterlegen:

# ssh-keygen



Restic Script:

/usr/local/bin/backup-to-disk.sh
#!/bin/bash
# Backup der wichtigsten Verzeichnisse nach einem Verzeichnis
#
# Es wird restic verwendet.
#
PATH=$PATH:/usr/local/bin
export RESTIC_PASSWORD="$(hostname --short)7355"
restic backup --quiet --repo /backup-restic/restic-repo-$(hostname --short) /home /etc /var /opt /var/lib/docker/volumes /usr/local/bin /backup --exclude=/var/log --exclude=/var/lib/docker/overlay2

if [ $? -eq 0 ]; then
  restic forget --quiet --repo /backup-restic/restic-repo-$(hostname --short) --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 20 --prune
else
  >&2 echo "Problem with restic Backup $(hostname --short)"
fi

/etc/cron.d/backup-to-disk:
#
# Backup important Files to Disk
#
55 6 * * * root /usr/local/bin/backup-to-disk.sh >/dev/null

Backup auf Storag Box:

# cat > /etc/cron.d/rsync-backup-to-other-host <<HERE
#
# Rsync /backup-restic to backup space
#
20 7 * * * root /usr/bin/rsync -avzH --delete --numeric-ids -e 'ssh -p23' /backup-restic u152662@u152662.your-storagebox.de:onyx-backup-restic-rsync >/dev/null
HERE

Wordpress behind Traefik

following needs to be inserted in wp-config.php (on top of PHP Code):

if (strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false)
   $_SERVER['HTTPS']='on';

DNS Server Tests

TODO => REVIEW

# cat /etc/cron.d/checkdnsserver
#
# Check DNS Server
#
*/15 * * * * root /usr/local/bin/checkdnsserver.sh >/dev/null


# cat /usr/local/bin/checkdnsserver.sh
#!/bin/bash
#
# Check my DNS servers and report to CloudRadar
#
# Joerg Lehmann, 17.8.2021
#

for dnsserver in ns1.nbit.ch ns2.nbit.ch ; do
  dig +short ${dnsserver} @${dnsserver} >/dev/null 2>&1
  if [ $? -eq 0 ]; then
    csender -t 6tZlIPoM7OQb \
      -u https://hub.cloudradar.io/cct/ \
      -n checkdnsserver \
      -s 1
  else
    csender -t 6tZlIPoM7OQb \
      -u https://hub.cloudradar.io/cct/ \
      -n checkdnsserver \
      -s 0 \
      -a "DNS Test @${dnsserver} failed"
  fi
  sleep 10
done

Setup Traefik

see https://gerov.eu/posts/traefik-for-podman/

# echo 'net.ipv4.ip_unprivileged_port_start=80' >> /etc/sysctl.d/containers.conf
# loginctl enable-linger containers
# The following fixes "Failed to connect to bus: No medium found"
export XDG_RUNTIME_DIR=/run/user/$(id -u)
containers$ systemctl --user enable --now podman.socket
containers$ podman network create web
containers$ touch acme.json
containers$ chmod 0600 acme.json

containers$ podman run --name=traefik -d \
  --security-opt label=type:container_runtime_t \
  -v /run/user/1000/podman/podman.sock:/var/run/docker.sock:z \
  -v ./acme.json:/acme.json:z \
  -v ./configuration/:/configuration/:z \
  --net web \
  -p 80:80 \
  -p 443:443 \
  --restart always \
  docker.io/library/traefik:latest \
  --entrypoints.web.address=":80" \
  --entrypoints.web.http.redirections.entryPoint.to=websecure \
  --entrypoints.web.http.redirections.entryPoint.scheme=https \
  --entrypoints.websecure.address=":443" \
  --providers.docker=true \
  --certificatesresolvers.myresolver.acme.email=postmaster@nbit.ch \
  --certificatesresolvers.myresolver.acme.storage=/acme.json \
  --certificatesresolvers.myresolver.acme.tlschallenge=true \
  --certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory \
  --certificatesresolvers.myresolver.acme.httpChallenge.entrypoint=web \
  --providers.file.directory=/configuration/ \
  --providers.file.watch=true \
  --accesslog=true \
  --log.level=DEBUG


containers$ mkdir -p ~/.config/systemd/user/
containers$ cd ~/.config/systemd/user/
containers$ podman generate systemd -f -n traefik
containers$ systemctl --user enable container-traefik.service

containers$ podman stop traefik
containers$ systemctl --user start container-traefik.service

# mkdir /data
# chown containers:containers /data

containers$ escaped=$(systemd-escape ~/onyx_pods/traefik/traefik.yaml)
containers$ systemctl --user start podman-kube@$escaped.service
containers$ systemctl --user enable podman-kube@$escaped.service


Hack until Podman supports network in Play YAML:

[containers@onyx default.target.wants]$ pwd
/home/containers/.config/systemd/user/default.target.wants
[containers@onyx default.target.wants]$ ls -l
total 0
lrwxrwxrwx. 1 containers containers 42 Dec  5 19:16 podman-kube@-home-containers-onyx_pods-nbit_websites-nbit_websites.yaml.service -> /usr/lib/systemd/user/podman-kube@.service
lrwxrwxrwx. 1 containers containers 42 Dec  3 16:06 podman-kube@-home-containers-onyx_pods-traefik-traefik.yaml.service -> /usr/lib/systemd/user/podman-kube@.service

6.9 KiB Raw Blame History