[{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/alertmanager/","section":"Tags","summary":"","title":"Alertmanager","type":"tags"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/authors/brettonvine/","section":"Authors","summary":"","title":"Brettonvine","type":"authors"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/freebsd/","section":"Tags","summary":"","title":"FreeBSD","type":"tags"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/mastodon/","section":"Tags","summary":"","title":"Mastodon","type":"tags"},{"content":" Mastodon and Related Infrastructure with Podman Containers # Introduction # We previously documented running Mastodon using Pot containers in this blog post.\nIn that post we used ansible to provision a host using a series of tasks in a playbook. You would clone the git repo of the playbook, configure some settings in an inventory file, and run the ansible playbook to provision your server.\nSince then podman has become available for FreeBSD, along with podman compose, making dockerisation of jails possible.\nThere is still a way to go with developing podman on FreeBSD before it becomes as user-friendly as docker or podman containers on other systems, but it\u0026rsquo;s definitely usable now.\nThis post covers some of the adjustments required, and decisions made, to run Mastodon and related services with podman containers on FreeBSD.\nBackground: Available FreeBSD Container Images # First some background on the different FreeBSD container images, aka OCI-format images.\nThe FreeBSD Foundation releases formal images for every release, and these are also available on Docker Hub and Github Container Registry.\nAs written by Doug Rabson in his FreeBSD Foundation blog post:\nOCI container engines such as containerd or podman need images. A container image is a read-only directory tree which typically contains an application with supporting files and libraries. Running this image on a container engine makes a writable clone of the image and executes the application in some kind of isolation environment such as a jail.\nImages are distributed via registries which store the image data and provide a simple REST API to access images and their metadata. The registry APIs, image formats and metadata are standardised by the Open Container Initiative which largely replaces earlier docker formats.\nFor example, with FreeBSD 15.0, the following images are available:\nFreeBSD-15.0-RELEASE-amd64-container-image-static.txz FreeBSD-15.0-RELEASE-amd64-container-image-dynamic.txz FreeBSD-15.0-RELEASE-amd64-container-image-runtime.txz FreeBSD-15.0-RELEASE-amd64-container-image-notoolchain.txz FreeBSD-15.0-RELEASE-amd64-container-image-toolchain.txz A detailed explanation for the types of FreeBSD OCI images is paraphrased below, sourced from this pull request for the FreeBSD handbook.\nfreebsd-static # The static image is intended as a base image, for a workload which is entirely statically linked. It contains no libraries, nor binaries, just the supporting files that most applications of this nature require. There is no UNIX shell, no command-line tools, no dynamic libraries, nor package manager.\nfreebsd-dynamic # The dynamic image uses the static image as a parent layer, and supports using shared libraries, including libc. It doesn\u0026rsquo;t have a shell, rc system, nor a package manager.\nfreebsd-runtime # Again, runtime builds on the preceding dynamic layer, and finally adds the minimum that a user would expect - a UNIX shell, rc system, and the package manager pkg.\nfreebsd-notoolchain # This base image contains almost all tools one would expect on a typical FreeBSD system, excluding those that are directly hardware-related, and thus not generally useful within a container, and the compiler and related toolchain, as it is quite large.\nfreebsd-toolchain # The Toolchain base image is the sum of all preceding images, including a full compiler and toolchain.\nReal world usage notes # Contrary to the proposed handbook documentation, we\u0026rsquo;ve found that the most useful OCI images are the freebsd-notoolchain and freebsd-toolchain ones.\nWhile initial development and experimentation with the freebsd-runtime container image proved useful, there are coverage gaps better suited by the freebsd-notoolchain and freebsd-toolchain images.\nFor example, to build a varnish container you will need the freebsd-toolchain base container, because varnish compiles its VCL at runtime. This needs a compiler.\nIt\u0026rsquo;s a labourious exercise to install compiler tools on a leaner container base, and far simpler to use the applicable OCI image which has a compiler already installed.\nAlthough a larger base image to download, the build process for custom containers completes much faster, which means a quicker live service.\nBackground: Podman Container Hooks # Not all FreeBSD services will run automatically with podman containers. For example, if you try to build a postgresql container, then regardless of which OCI image you use, when you run it you will get complaints about shared memory, and postgresql will refuse to start.\nThe solution is to edit jail parameters to configure sysvshm, sysvsem and sysvmsg along with some others. Not an obvious solution, but there is a way to automate it.\nPodman has the ability to run scripts via a container hooks subsystem, which can modify containers at various stages of the container lifecycle.\nThis is first discussed in this github issue from March 2025, along with a Mastodon thread from David Chisnall in May 2025.\nThe podman container hooks concept is also explored in a series of blog posts by Dave Cottlehuber:\nUsing Podman hooks to mount persistent ZFS datasets into ephemeral Containers Using Podman hooks to attach Nebula mesh networking to containers Extending OCI container networks with Hooks We cover our implementation of automatic container hooks, specifically for postgresql and opensearch containers, and annotations in podman-compose.yml, in the setup of micropod-mastodon-suite below.\nWhat is required to run Mastodon? # At the most basic level, all you need to run Mastodon is a database, redis, and a webserver.\nYou might also want S3 storage for media, as well as elasticsearch or opensearch integration for search.\nThe setup we cover below is a bit more involved, inclusive of database backups, haproxy and varnish cache, separate containers for the different sidekiq queues, and mastodon-streaming and mastodon-web processes. Serving media from S3 is a separate nginx container from the one which provides a reverse proxy to the mastodon-web process.\nOn a ~100 user system you will need a host or virtual machine with at least 16 vCPU and 24GB memory, and at least 80GB disk space when media storage is in separate S3, or 300GB or more if not.\nMicropod-mastodon-suite # Micropod-mastodon-suite is a podman compose recipe for running a suite of services to support a Mastodon instance.\nIt assumes you have external S3 storage for media files, along with a separate firewall/proxy frontend to handle SSL certificates and haproxy rules to the internal host.\nIt uses included shell scripts for starting and stopping the containers in a staggered order, rather than the conventional podman compose up -d or podman compose down.\nSystem Preparation # You will need to prepare your FreeBSD 15.0 (or higher) host to run podman containers, along with additional preparation for podman container hooks to be able to run the postgresql and opensearch containers.\nAll the following commands are run by the root user. Podman on FreeBSD doesn\u0026rsquo;t support rootless containers.\nCreate ZFS datasets # Podman will by default store containers in /var/db/containers. Make sure there is a ZFS dataset as follows\nzfs create -o mountpoint=/var/db/containers zroot/containers It is also recommended to create a ZFS dataset to clone the repo to, and retain stored data.\nzfs create -o mountpoint=/mnt/data zroot/data /mnt/data could also be an NFS mount to a host server if running on a vm-bhyve virtual machine.\nCreate a container logs directory # Make sure there is a directory /var/log/containers for container logs.\nmkdir -p /var/log/containers Most containers in Micropod-mastodon-suite will write log files here, with the exception of postgresql which writes internally and is limited to 10 MB in size. Failure to limit postgresql container logs will quickly use up all available disk space on a Mastodon instance.\nSet package stream to \u0026ldquo;latest\u0026rdquo; # Make sure you\u0026rsquo;re on the latest package stream as follows:\nmkdir -p /usr/local/etc/pkg/repos nano /usr/local/etc/pkg/repos/FreeBSD.conf and add contents\nFreeBSD-ports: { url: \u0026#34;pkg+https://pkg.FreeBSD.org/${ABI}/latest\u0026#34;, mirror_type: \u0026#34;srv\u0026#34;, signature_type: \u0026#34;fingerprints\u0026#34;, fingerprints: \u0026#34;/usr/share/keys/pkg\u0026#34;, enabled: yes } Then run\npkg update -f pkg upgrade -yf Install podman, podman-compose, git, jq # Install podman-suite, py311-podman-compose, git and jq as follows\npkg install podman-suite py311-podman-compose git jq Configure PF firewall # The following pf ruleset differs from the documented one for Podman on FreeBSD as it supports IPv6.\nCreate /etc/pf.conf with the following contents, making sure to set vtnet0 to the interface name of your host or virtual machine if it differs:\nv4egress_if = \u0026#34;vtnet0\u0026#34; v6egress_if = \u0026#34;vtnet0\u0026#34; cni_if0 = \u0026#34;cni-podman0\u0026#34; cni_if1 = \u0026#34;cni-podman1\u0026#34; table \u0026lt;cni-nat\u0026gt; persist { 10.88.0.0/16, 10.89.0.0/24, fdca:1db3:f328:2d7c::/64 } nat on $v4egress_if inet from \u0026lt;cni-nat\u0026gt; to any -\u0026gt; ($v4egress_if) nat on $v6egress_if inet6 from \u0026lt;cni-nat\u0026gt; to !ff00::/8 -\u0026gt; ($v6egress_if) rdr-anchor \u0026#34;cni-rdr/*\u0026#34; nat-anchor \u0026#34;cni-rdr/*\u0026#34; nat-anchor \u0026#34;cni-nat/*\u0026#34; anchor \u0026#34;cni-nat/*\u0026#34; # Pass rules so packets aren’t blocked pass quick on $cni_if0 inet from ($cni_if0:network) to any keep state pass quick on $cni_if1 inet from ($cni_if1:network) to any keep state pass quick on $cni_if0 inet6 from ($cni_if0:network) to any keep state pass quick on $cni_if1 inet6 from ($cni_if1:network) to any keep state pass out quick on $v4egress_if inet from ($cni_if0:network) to any keep state pass out quick on $v4egress_if inet from ($cni_if1:network) to any keep state pass out quick on $v6egress_if inet6 from ($cni_if0:network) to any keep state pass out quick on $v6egress_if inet6 from ($cni_if1:network) to any keep state # ICMP/ICMPv6 (ping + PMTU) pass inet proto icmp from ($cni_if0:network) to any keep state pass inet proto icmp from ($cni_if1:network) to any keep state pass inet6 proto icmp6 from ($cni_if0:network) to any keep state pass inet6 proto icmp6 from ($cni_if1:network) to any keep state Configure the following sysctl parameter:\nsysctl net.pf.filter_local=1 echo \u0026#34;net.pf.filter_local=1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf Enable pf and pflog services, and start pf:\nservice pf enable service pflog enable service pf start Enable gateway services for correct routing # Configure the following gateway services for the host to ensure correct routing:\nsysrc gateway_enable=\u0026#34;YES\u0026#34; sysrc ipv6_gateway_enable=\u0026#34;YES\u0026#34; service routing restart Configure fdescfs mount # Make sure fdescfs is mounted, and will mount on reboot. This is required for Podman.\nmount -t fdescfs fdesc /dev/fd echo \u0026#34;fdesc /dev/fd fdescfs rw 0 0\u0026#34; \u0026gt;\u0026gt; /etc/fstab Enable and start the podman_service and podman services # Enable and start the podman_service service as follows:\nservice podman_service enable service podman_service start Enable and start the podman service as follows:\nservice podman enable service podman start Both services must be enabled. The podman service will ensure containers restart automatically on reboot.\nSetup source containers with official FreeBSD OCI images # Use podman to load the relevant source FreeBSD OCI images:\npodman pull docker.io/freebsd/freebsd-notoolchain:15.0 podman pull docker.io/freebsd/freebsd-toolchain:15.0 You only need to do this once, it doesn\u0026rsquo;t need to be repeated for every container build.\nCreate custom podman hooks # The postgresql container needs the following jail parameters enabled\nsysvmsg=1 sysvsem=1 sysvshm=1 The opensearch container needs the following jail parameters enabled\nallow.mount=1 allow.mount.fdescfs=1 allow.mount.procfs=1 enforce_statfs=1 allow.mlock=1 We will automate this by creating custom hooks for podman.\nCreate the file /usr/local/etc/containers/hooks.d/customattributes.json with contents:\n{ \u0026#34;version\u0026#34;: \u0026#34;1.0.0\u0026#34;, \u0026#34;hook\u0026#34;: { \u0026#34;path\u0026#34;: \u0026#34;/usr/local/etc/containers/hooks.d/customattributes.sh\u0026#34; }, \u0026#34;when\u0026#34;: { \u0026#34;annotations\u0026#34;: { \u0026#34;^com\\\\.micropod\\\\.jail\\\\.profile$\u0026#34;: \u0026#34;^(postgresql|opensearch)$\u0026#34; } }, \u0026#34;stages\u0026#34;: [ \u0026#34;createRuntime\u0026#34; ] } Then create the file /usr/local/etc/containers/hooks.d/customattributes.sh with contents:\n#!/bin/sh set -eu PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin STATE=\u0026#34;$(cat)\u0026#34; CONTAINER_ID=\u0026#34;$(printf \u0026#39;%s\u0026#39; \u0026#34;$STATE\u0026#34; | jq -r \u0026#39;.id\u0026#39;)\u0026#34; PROFILE=\u0026#34;$(printf \u0026#39;%s\u0026#39; \u0026#34;$STATE\u0026#34; | jq -r \u0026#39;.annotations[\u0026#34;com.micropod.jail.profile\u0026#34;] // empty\u0026#39;)\u0026#34; JAIL_STATE=\u0026#34;/var/run/ocijail/${CONTAINER_ID}/state.json\u0026#34; JAIL_ID=\u0026#34;$(jq -r \u0026#39;.jid\u0026#39; \u0026lt; \u0026#34;$JAIL_STATE\u0026#34;)\u0026#34; case \u0026#34;$PROFILE\u0026#34; in postgresql) jail -m jid=\u0026#34;$JAIL_ID\u0026#34; \\ sysvmsg=1 \\ sysvsem=1 \\ sysvshm=1 logger -t oci \u0026#34;custom jail profile=postgresql container=${CONTAINER_ID} jid=${JAIL_ID}\u0026#34; ;; opensearch) jail -m jid=\u0026#34;$JAIL_ID\u0026#34; \\ allow.mount=1 \\ allow.mount.fdescfs=1 \\ allow.mount.procfs=1 \\ enforce_statfs=1 \\ allow.mlock=1 logger -t oci \u0026#34;custom jail profile=opensearch container=${CONTAINER_ID} jid=${JAIL_ID}\u0026#34; ;; *) logger -t oci \u0026#34;unknown or missing jail profile=\u0026#39;${PROFILE}\u0026#39; container=${CONTAINER_ID}\u0026#34; exit 0 ;; esac Make it executable:\nchmod +x /usr/local/etc/containers/hooks.d/customattributes.sh Then make sure the file /usr/local/etc/containers/containers.conf has the hooks section enabled (uncommented) and a custom directory location set as follows:\n[engine] hooks_dir = [ \u0026#34;/usr/local/etc/containers/hooks.d/\u0026#34;, ] Finally, restart the podman_service service:\nservice podman_service restart In the podman-compose.yml file in the repo below, the annotations are set for the postgresql image as\npostgresql: ... container_name: postgresql annotations: com.micropod.jail.profile: \u0026#34;postgresql\u0026#34; and for the opensearch image as\nopensearch: ... container_name: opensearch annotations: com.micropod.jail.profile: \u0026#34;opensearch\u0026#34; These annotations will trigger the custom podman container hooks and set the required jail parameters as specified in /usr/local/etc/containers/hooks.d/customattributes.sh.\nSystem preparation is now complete. You can move onto setting up Micropod-mastodon-suite.\nCloning the git repo and configuring .env file # Clone the Micropod-mastodon-suite repo # Clone the Micropod-mastodon-suite repo, in the ZFS dataset or NFS mounted directory, as follows:\ncd /mnt/data git clone https://codeberg.org/Honeyguide/micropod-mastodon-suite.git cd micropod-mastodon-suite/ Configure the .env file with your specific parameters # Copy the sample.env file to .env and edit according to your needs.\nYou will need to set IPv4 and IPv6 addresses for the host, as well as IPv4 and IPv6 addresses of the upstream firewall/proxy host.\nSet your domain name and CDN domain name.\nMake sure to include the IPv4 address of your S3 host, along with the applicable ports. Include SMTP server and credentials as well.\n# Host specific variables # This is your VM\u0026#39;s IP address IP4_ADDRESS=\u0026#34;\u0026#34; # make sure to have square brackets around IPv6 address here IP6_ADDRESS=\u0026#34;[]\u0026#34; # External proxy IP addresses, usually upstream Haproxy\u0026#39;s IP addresses, used for real IP forwarding in nginx # Do not set square brackets on v6 address here! PROXY_IP_V4=\u0026#34;\u0026#34; PROXY_IP_V6=\u0026#34;\u0026#34; # Application specific variables # database DB_HOST=\u0026#34;postgres\u0026#34; DB_PORT=\u0026#34;5432\u0026#34; DB_NAME=\u0026#34;mastodon_production\u0026#34; DB_USER=\u0026#34;mastodon\u0026#34; DB_PASSWORD=\u0026#34;mastodon\u0026#34; # mastodon variables RAILS_ENV=\u0026#34;production\u0026#34; DOMAIN=\u0026#34;example.com\u0026#34; SECRET_KEY=\u0026#34;\u0026#34; OTP_SECRET=\u0026#34;\u0026#34; VAPID_PRIVATE_KEY=\u0026#34;\u0026#34; VAPID_PUBLIC_KEY=\u0026#34;\u0026#34; ACTIVE_PRIMARY_KEY=\u0026#34;\u0026#34; ACTIVE_DETERMINISTIC_KEY=\u0026#34;\u0026#34; ACTIVE_KEY_DERIVATION_SALT=\u0026#34;\u0026#34; MY_MAIL_HOST=\u0026#34;\u0026#34; MY_MAIL_PORT=\u0026#34;\u0026#34; MY_MAIL_USERNAME=\u0026#34;\u0026#34; MY_MAIL_PASSWORD=\u0026#34;\u0026#34; MY_MAIL_FROM_ADDRESS=\u0026#34;\u0026#34; S3_BUCKET=\u0026#34;mastodon\u0026#34; S3_REGION=\u0026#34;garage\u0026#34; S3_USER=\u0026#34;\u0026#34; S3_PASSWORD=\u0026#34;\u0026#34; S3_CDN_HOSTNAME=\u0026#34;\u0026#34; # leave elastic user and password empty for now ELASTIC_USERNAME=\u0026#34;\u0026#34; ELASTIC_PASSWORD=\u0026#34;\u0026#34; DEEPL_API_KEY=\u0026#34;\u0026#34; DEEPL_API_PLAN=\u0026#34;\u0026#34; # for haproxy container to S3 storage # Add IP address of your S3 storage and set ports if different from the garage web port used here S3_IP_ADDRESS_1=\u0026#34;\u0026#34; S3_READ_PORT_NUMBER=\u0026#34;3902\u0026#34; S3_WRITE_PORT_NUMBER=\u0026#34;3900\u0026#34; If you do not have existing values for secret key, OTP secret, vapid keys, active keys, leave these blank as they will be auto-generated and stored for future use on container restarts.\nIt is not necessary to set a username or password for elasticsearch (which is actually opensearch).\nBuilding and running the containers # We\u0026rsquo;ve adopted a build local approach to podman containers, as opposed to pulling prebuilt containers from a container registry. This is an opinionated choice.\nFor some podman compose setups we\u0026rsquo;ve also adopted custom shell scripts to start/stop containers in a defined build and run order, as opposed to using podman compose up -d or podman compose down directly.\nFor Micropod-mastodon-suite you can initiate building and running the containers with:\n./start-mastodon-suite.sh This step can take 30-45 minutes to complete, and the containers will be running automatically.\nThe script will build all containers and store images locally. Then it will start them in a staggered fashion, where some containers do specific things and exit, while others serve as templates for service-specific containers.\nFor example, the maintaindb will perform administrative functions on the postgresql server before any Mastodon containers are running.\nThe mastodon container will perform all the necessary steps to build a Mastodon-capable container image, but won\u0026rsquo;t run directly. The various sidekiq processes, and mastodon-streaming and mastodon-web, are duplicates of the source mastodon container with different entrypoints configured for each.\nWhen done, check the output of the following command to see if all images are live:\npodman ps -a To stop the containers you can run:\n./stop-all-containers.sh Setting up a new instance # There are many scripts in the scripts/ subdirectory that can perform different functions. On a freshly installed host we recommend the following:\nSetup Admin User # To setup an admin user, run the script scripts/run-create-admin-user.sh as follows:\n./scripts/run-create-admin-user.sh --username name --email email@address View the credentials in the file data/mastodon/private/mastodon.owner.credentials to login as first user.\nSetup opensearch index # To setup opensearch indexes run\n./scripts/run-setup-es-index.sh Enable query_stats in postgresql # To enable pg_stat_statements in the pgdata tool for administrators, run\n./scripts/run_enable_postgresql_query_stats.sh External monitoring and restarting of sidekiq queues # Podman on FreeBSD doesn\u0026rsquo;t have podman container healthcheck monitoring like it does on Linux, as this is a systemd service.\nInstead you have to build your own monitoring solution via cron jobs.\nIf you want to do external monitoring of the sidekiq queues, and trigger automatic container restarts on wedged queues, create a cron job for the root user with crontab -e.\nPaste the following in, making sure to configure the INSTANCE variable to match the IP address of the host running Micropod-mastodon-suite, and setting the MONITORHOST variable to be the IP address of an alertmanager server, if available.\nMAILTO=\u0026#34;\u0026#34; INSTANCE=\u0026#34;1.2.3.4\u0026#34; MONITORHOST=\u0026#34;1.2.3.5\u0026#34; */2 * * * * QUEUES=\u0026#34;ingress:mastodon-sidekiq-ingress-1 ingress:mastodon-sidekiq-ingress-2 pull:mastodon-sidekiq-pull push:mastodon-sidekiq-push\u0026#34; /mnt/data/micropod-mastodon-suite/scripts/cron-sidekiq-healthcheck.sh This will monitor the 2 sidekiq ingress queues, and sidekiq pull and push queues.\nYou can list all the sidekiq queues to be monitored if desired, see the source for scripts/cron-sidekiq-healthcheck.sh\nDo not change the timing, it is expected to run every 2 minutes.\nAn alertmanager server is available as part of micropod-monitoring\nHappy tooting!\n","date":"June 20, 2026","externalUrl":null,"permalink":"/blog/2026-06-20/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eMastodon and Related Infrastructure with Podman Containers \n    \u003cdiv id=\"mastodon-and-related-infrastructure-with-podman-containers\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#mastodon-and-related-infrastructure-with-podman-containers\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\u003ch3 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003eWe previously documented running Mastodon using Pot containers in this \u003ca\n  href=\"https://honeyguide.eu/blog/2024-02-16/\"\n    target=\"_blank\"\n  \u003eblog post\u003c/a\u003e.\u003c/p\u003e","title":"Mastodon and Related Infrastructure with Podman Containers","type":"blog"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/podman/","section":"Tags","summary":"","title":"Podman","type":"tags"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/podman-compose/","section":"Tags","summary":"","title":"Podman Compose","type":"tags"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/podman-hooks/","section":"Tags","summary":"","title":"Podman Hooks","type":"tags"},{"content":"","date":"June 20, 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"September 9, 2025","externalUrl":null,"permalink":"/tags/minio/","section":"Tags","summary":"","title":"Minio","type":"tags"},{"content":"","date":"September 9, 2025","externalUrl":null,"permalink":"/tags/nbd/","section":"Tags","summary":"","title":"NBD","type":"tags"},{"content":"","date":"September 9, 2025","externalUrl":null,"permalink":"/tags/s3/","section":"Tags","summary":"","title":"S3","type":"tags"},{"content":"","date":"September 9, 2025","externalUrl":null,"permalink":"/tags/scalable-storage/","section":"Tags","summary":"","title":"Scalable Storage","type":"tags"},{"content":"","date":"September 9, 2025","externalUrl":null,"permalink":"/tags/zerofs/","section":"Tags","summary":"","title":"ZeroFS","type":"tags"},{"content":" ZeroFS: S3-backed ZPOOL Via NBD Service on FreeBSD (Proof of Concept, Part 2) # Introduction # ZeroFS turns S3 storage into a high-performance, always-encrypted, globally accessible, and scalable file system—no FUSE drivers required. There\u0026rsquo;s also an official ZeroFS github repo.\nZeroFS makes S3 storage feel like a real filesystem. Built on SlateDB, it provides file-level access via NFS and 9P and block-level access via NBD. Fast enough to compile code on, with clients already built into your OS. No FUSE drivers, no kernel modules, just mount and go. This post walks through a FreeBSD proof of concept using ZeroFS to serve network block devices (NBD), then creating a ZPOOL with those devices with the help of the FreeBSD GEOM NBD Client. It is part 2 in a short series.\nIt differs from part 1 in the use of Minio as S3 provider, because Garage doesn\u0026rsquo;t support conditional writes and the latest versions of ZeroFS generate an error with a Garage backend.\nAdditionally, ZeroFS has a rapid pace of development, with updates to configuration, and part 1 may be considered obsolete already.\nIf you want to experiment with ZeroFS on FreeBSD to provide NBD devices for a zpool, follow along.\nAcknowledgements # This proof of concept was made possible with input from Ryan Moeller, author of the FreeBSD GEOM NBD Client, and Pierre Barre, author of ZeroFS, in making the necessary changes to get things working correctly.\nPrerequisites # A FreeBSD 14.3 virtual host with ZFS. We’ll install a temporary S3 service using Minio and run the ZeroFS binary. In this guide, this host’s IP is 10.0.0.10.\nA second host on the same network to access NFS and NBD shares. For this guide, this host’s IP is 10.0.0.20.\nOutline # Install a S3 service on the first virtual host, using Minio as S3 provider, configure credentials and bucket with access rights.\nThen install ZeroFS, adjust configuration file, and enable a shell script to start the service.\nOn the other host, install the FreeBSD GEOM NBD Client, mount the NFS share to create NBD files, unmount NFS share, connect to NBD shares with the FreeBSD GEOM NBD Client, and create a zpool using the mounted devices.\nPreparation # Use the latest pkg repository on your virtual host:\nmkdir -p /usr/local/etc/pkg/repos echo \u0026#39;FreeBSD: { url: \u0026#34;pkg+http://pkg.FreeBSD.org/${ABI}/latest\u0026#34; }\u0026#39; \u0026gt; /usr/local/etc/pkg/repos/FreeBSD.conf Update and force-upgrade installed packages:\npkg update pkg upgrade -f Install a few handy tools:\npkg install -y ca_root_nss curl openssl tmux First host: install Minio # Create ZFS datasets for Minio:\nzfs create -o mountpoint=/mnt/data zroot/data zfs create -o mountpoint=/mnt/data/minio zroot/data/minio Install minio and minio-client:\npkg install minio minio-client Set minio as owner on the ZFS dataset:\nchown -R minio:minio /mnt/data/minio Generate a password for the Minio admin user, for example:\nopenssl rand -hex 32 Configure RC entries for minio, setting minio_root_password to the password generated in previous step:\nsysrc minio_disks=\u0026#34;/mnt/data/minio\u0026#34; sysrc minio_address=\u0026#34;:9000\u0026#34; sysrc minio_root_user=\u0026#34;admin\u0026#34; sysrc minio_root_password=\u0026#34;02fc91742b4399b88ec4bd59a1787a6537dba1df65799d5f6e6a79d4568b7c69\u0026#34; sysrc minio_root_access=\u0026#34;on\u0026#34; sysrc minio_logfile=\u0026#34;/var/log/minio.log\u0026#34; Enable and start Minio:\nservice minio enable service minio start Create a directory for minio user config:\nmkdir -p /root/.minio-client Create /root/.minio-client/config.json so minio-client can perform administrative actions:\n{ \u0026#34;version\u0026#34;: \u0026#34;10\u0026#34;, \u0026#34;aliases\u0026#34;: { \u0026#34;gcs\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://storage.googleapis.com\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v2\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;dns\u0026#34; }, \u0026#34;local\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;http://127.0.0.1:9000\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;myminio\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;http://127.0.0.1:9000\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;admin\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;02fc91742b4399b88ec4bd59a1787a6537dba1df65799d5f6e6a79d4568b7c69\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;play\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://play.min.io\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;Q3AM3UQ867SPQQA43P2F\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;s3\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://s3.amazonaws.com\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;dns\u0026#34; } } } Create the files bucket in the myminio instance:\nminio-client --insecure mb \\ --ignore-existing \\ --with-lock \\ myminio/files \\ --config-dir \u0026#34;/root/.minio-client/\u0026#34; Generate a password for the Minio zerofs user, for example:\nopenssl rand -hex 32 Create the zerofs user in the myminio instance, adjusting password to match the one created in previous step:\nminio-client --insecure admin user add \\ myminio \\ zerofs f839343afd282fd1a348450385791b9a664e054e09f4e4c648d920366fe8ee13 \\ --config-dir \u0026#34;/root/.minio-client/\u0026#34; Create a Minio policy file /tmp/zerofs.json with contents:\n{ \u0026#34;Version\u0026#34;: \u0026#34;2012-10-17\u0026#34;, \u0026#34;Statement\u0026#34;: [ { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Action\u0026#34;: \u0026#34;s3:*\u0026#34;, \u0026#34;Resource\u0026#34;: \u0026#34;arn:aws:s3:::files/*\u0026#34; } ] } Add the Minio policy file to the myminio instance:\n/usr/local/bin/minio-client --insecure admin policy create \\ myminio \\ zerofs \\ \u0026#34;/tmp/zerofs.json\u0026#34; \\ --config-dir \u0026#34;/root/.minio-client/\u0026#34; Link the imported policy to the Minio zerofs user:\n/usr/local/bin/minio-client --insecure admin policy attach \\ myminio \\ zerofs \\ --user \u0026#34;zerofs\u0026#34; \\ --config-dir /root/.minio-client/ Finally, apply the policy:\n/usr/local/bin/minio-client --insecure admin user policy \\ myminio \\ zerofs \\ --config-dir /root/.minio-client/ Minio should be setup correctly with full access to the myminio/files bucket by the Minio zerofs user.\nFirst host: install ZeroFS # ZeroFS is in the ports tree. You must be using version 0.15.1 or higher for this guide.\nTo install:\npkg install zerofs To build from ports:\npkg install git pkgconf gmake rust echo \u0026#34;DEFAULT_VERSIONS+=ssl=openssl\u0026#34; \u0026gt; /etc/make.conf git clone --depth=1 -b main https://git.freebsd.org/ports.git /usr/ports cd /usr/ports/filesystems/zerofs/ make install Building Zerofs will take a little while.\nCreate the ZeroFS configuration file:\ncd /usr/local/etc zerofs init Then edit the generated file /usr/local/etc/zerofs.toml to match the following (includes enabling IPv6):\n[cache] dir = \u0026#34;${HOME}/.cache/zerofs\u0026#34; disk_size_gb = 10.0 memory_size_gb = 1.0 [storage] url = \u0026#34;s3://files/zerofs-data\u0026#34; encryption_password = \u0026#34;${ZEROFS_PASSWORD}\u0026#34; [servers.nfs] addresses = [\u0026#34;0.0.0.0:2049\u0026#34;, \u0026#34;[::1]:2049\u0026#34;] [servers.ninep] addresses = [\u0026#34;0.0.0.0:5564\u0026#34;, \u0026#34;[::1]:5564\u0026#34;] unix_socket = \u0026#34;/tmp/zerofs.9p.sock\u0026#34; [servers.nbd] addresses = [\u0026#34;0.0.0.0:10809\u0026#34;, \u0026#34;[::1]:10809\u0026#34;] unix_socket = \u0026#34;/tmp/zerofs.nbd.sock\u0026#34; [aws] secret_access_key = \u0026#34;${AWS_SECRET_ACCESS_KEY}\u0026#34; access_key_id = \u0026#34;${AWS_ACCESS_KEY_ID}\u0026#34; # Optional AWS S3 settings (uncomment to use): endpoint = \u0026#34;http://127.0.0.1:9000\u0026#34; # For S3-compatible services default_region = \u0026#34;global\u0026#34; allow_http = \u0026#34;true\u0026#34; # For non-HTTPS endpoints Because there is no rc service/variables yet, use a small wrapper script to start ZeroFS with the right environment.\nCreate /root/bin/start-zerofs.sh with the correct credentials for your system, and set a new value for ZEROFS_PASSWORD:\n#!/bin/sh ZEROFS_PASSWORD=test-secret-test export ZEROFS_PASSWORD AWS_ACCESS_KEY_ID=zerofs export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=f839343afd282fd1a348450385791b9a664e054e09f4e4c648d920366fe8ee13 export AWS_SECRET_ACCESS_KEY /usr/local/bin/zerofs run -c /usr/local/etc/zerofs.toml Make it executable:\nchmod +x /root/bin/start-zerofs.sh Start ZeroFS (a tmux session is handy to keep it running):\n/root/bin/start-zerofs.sh Second host: install FreeBSD GEOM client # On the second host, install git:\npkg install git Then clone the FreeBSD GEOM client git repo, build and install:\ngit clone https://github.com/ryan-moeller/kernel-nbd-client cd kernel-nbd-client make make install Load the kernel module:\nkldload geom_nbd.ko Check if loaded:\nkldstat Id Refs Address Size Name 1 16 0xffffffff80200000 1f41500 kernel 2 1 0xffffffff82142000 5e9328 zfs.ko 3 1 0xffffffff8272c000 7808 cryptodev.ko 4 1 0xffffffff82e10000 58c0 geom_nbd.ko Second host: mount ZeroFS over NFS to create NBD devices # Create a mount point for NFS for temporary step to create NBD files:\nmkdir -p /mnt/files Mount the NFS share (replace the IP with your ZeroFS host):\nmount -t nfs -o nfsv3,nolockd,bg,hard,tcp,port=2049,mountport=2049,rsize=1048576,wsize=1048576 10.0.0.10:/ /mnt/files Create NBD devices inside of /mnt/files/.nbd, which will already exist in the NFS share:\ntruncate -s 2G /mnt/files/.nbd/nbd0 truncate -s 2G /mnt/files/.nbd/nbd1 truncate -s 2G /mnt/files/.nbd/nbd2 truncate -s 2G /mnt/files/.nbd/nbd3 Unmount the NFS share:\numount /mnt/files You may see a benign error because there are no RPC services on the ZeroFS host:\numount /mnt/files umount: 10.0.0.10: MOUNTPROG: RPC: Remote system error - Connection refused Second host: mount NBD devices and create ZPOOL # Initialise gnbd and check the NBD exports on the ZeroFS host:\ngnbd load gnbd exports -p 10809 10.0.0.10 nbd0 nbd1 nbd2 nbd3 Use gnbd to mount each device:\ngnbd connect -n nbd0 10.0.0.10 gnbd connect -n nbd1 10.0.0.10 gnbd connect -n nbd2 10.0.0.10 gnbd connect -n nbd3 10.0.0.10 Create a RAID10 style mirrored zpool over the NBD devices (may be overkill):\nzpool create testpool mirror /dev/nbd0 /dev/nbd1 mirror /dev/nbd2 /dev/nbd3 Check the zpool exists:\nzpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT testpool 3.75G 468K 3.75G - - 0% 0% 1.00x ONLINE - zroot 47.5G 4.07G 43.4G - - 0% 8% 1.00x ONLINE - Check the zpool layout:\nzpool status testpool pool: testpool state: ONLINE config: NAME STATE READ WRITE CKSUM testpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 nbd0 ONLINE 0 0 0 nbd1 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 nbd2 ONLINE 0 0 0 nbd3 ONLINE 0 0 0 errors: No known data errors Create a ZFS dataset on the testpool zpool:\nzfs create -o mountpoint=/mnt/testdata testpool/testdata Add some files to the newly created ZFS dataset:\ncp -a /usr/local/share/doc /mnt/testdata Perform any other testing you need.\nIf finished testing, export the zpool before ending:\nzpool export testpool Then disconnect the NBD devices:\ngnbd disconnect nbd3 gnbd disconnect nbd2 gnbd disconnect nbd1 gnbd disconnect nbd0 Conclusion # If everything went smoothly, you now have a working proof of concept: S3-backed storage serving network block devices, as components of a zpool array, via ZeroFS.\nQuestions, suggestions, or improvements? Reach out on Mastodon.\n","date":"September 9, 2025","externalUrl":null,"permalink":"/blog/2025-09-09/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eZeroFS: S3-backed ZPOOL Via NBD Service on FreeBSD (Proof of Concept, Part 2) \n    \u003cdiv id=\"zerofs-s3-backed-zpool-via-nbd-service-on-freebsd-proof-of-concept-part-2\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#zerofs-s3-backed-zpool-via-nbd-service-on-freebsd-proof-of-concept-part-2\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\u003ch3 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003e\u003ca\n  href=\"https://www.zerofs.net\"\n    target=\"_blank\"\n  \u003eZeroFS\u003c/a\u003e turns S3 storage into a high-performance, always-encrypted, globally accessible, and scalable file system—no FUSE drivers required. There\u0026rsquo;s also an official \u003ca\n  href=\"https://github.com/Barre/ZeroFS\"\n    target=\"_blank\"\n  \u003eZeroFS github repo\u003c/a\u003e.\u003c/p\u003e","title":"ZeroFS: S3-backed ZPOOL via NBD Service on FreeBSD (Proof of Concept, Part 2)","type":"blog"},{"content":"","date":"August 29, 2025","externalUrl":null,"permalink":"/tags/garage/","section":"Tags","summary":"","title":"Garage","type":"tags"},{"content":"","date":"August 29, 2025","externalUrl":null,"permalink":"/tags/nfs/","section":"Tags","summary":"","title":"NFS","type":"tags"},{"content":" ZeroFS: S3-backed NFS Service on FreeBSD (Proof of Concept, Part 1) # Introduction # Please note that we found out while operating the PoC that Garage ignores the conditions in conditional writes. As this feature is required by ZeroFS, this means that ZeroFS is actually not reliably working with Garage at the moment.\nWe strongly suggest that you use Minio as local S3 platform as described in part 2 of this series instead.\nZeroFS turns S3 storage into a high-performance, always-encrypted, globally accessible, and scalable file system—no FUSE drivers required. There\u0026rsquo;s also an official ZeroFS github repo.\nZeroFS makes S3 storage feel like a real filesystem. Built on SlateDB, it provides file-level access via NFS and 9P and block-level access via NBD. Fast enough to compile code on, with clients already built into your OS. No FUSE drivers, no kernel modules, just mount and go. This post walks through a FreeBSD proof of concept using ZeroFS for NFS (with a brief note on 9P). It’s part 1 in a short series.\nPart 2 will follow once FreeBSD has better NBD support, enabling ZFS pools backed by S3.\nIf you want to experiment with ZeroFS on FreeBSD to provide NFS and 9P shares, follow along.\nPrerequisites # A FreeBSD 14.3 virtual host with ZFS. We’ll install a temporary S3 service using Garage and run the ZeroFS binary. In this guide, that host’s IP is 10.0.0.10.\nA second host on the same network for NFS-client testing.\nOutline # Install a S3 service on the virtual host, using Garage as S3 provider, configure credentials and bucket with access rights.\nThen install ZeroFS and configure a shell script to start it, making a NFS share available, with contents stored inside the SlateDB database in the S3 bucket.\nFinally, mount the NFS share on another host, write files, unmount, remount and perform any other testing.\nPreparation # Use the latest pkg repository on your virtual host:\nmkdir -p /usr/local/etc/pkg/repos echo \u0026#39;FreeBSD: { url: \u0026#34;pkg+http://pkg.FreeBSD.org/${ABI}/latest\u0026#34; }\u0026#39; \u0026gt; /usr/local/etc/pkg/repos/FreeBSD.conf Update and force-upgrade installed packages:\npkg update pkg upgrade -f Install a few handy tools:\npkg install -y ca_root_nss curl openssl tmux Install Garage # Create ZFS datasets for Garage:\nzfs create -o mountpoint=/opt zroot/opt zfs create -o mountpoint=/opt/garagemetadata zroot/opt/garagemetadata zfs create -o mountpoint=/mnt/data zroot/data zfs create -o mountpoint=/mnt/data/garage zroot/data/garage Install garage and minio-client:\npkg install garage minio-client Generate three internal passwords using:\nopenssl rand -hex 32 Create /usr/local/etc/garage.toml (replace SECRET1, SECRET2, and SECRET3 with the values generated above):\nmetadata_dir = \u0026#34;/opt/garagemetadata\u0026#34; data_dir = \u0026#34;/mnt/data/garage\u0026#34; db_engine = \u0026#34;lmdb\u0026#34; replication_mode = \u0026#34;none\u0026#34; rpc_bind_addr = \u0026#34;127.0.0.1:3901\u0026#34; rpc_public_addr = \u0026#34;127.0.0.1:3901\u0026#34; rpc_secret = \u0026#34;SECRET1\u0026#34; bootstrap_peers = [] [s3_api] s3_region = \u0026#34;garage\u0026#34; api_bind_addr = \u0026#34;0.0.0.0:3900\u0026#34; root_domain = \u0026#34;.s3.garage\u0026#34; [s3_web] bind_addr = \u0026#34;0.0.0.0:3902\u0026#34; root_domain = \u0026#34;.web.garage\u0026#34; index = \u0026#34;index.html\u0026#34; [admin] api_bind_addr = \u0026#34;0.0.0.0:3903\u0026#34; admin_token = \u0026#34;SECRET2\u0026#34; metrics_token = \u0026#34;SECRET3\u0026#34; Register RC variables:\nsysrc garage_config=\u0026#34;/usr/local/etc/garage.toml\u0026#34; sysrc garage_log_file=\u0026#34;/var/log/garage.log\u0026#34; Enable and start Garage:\nservice garage enable service garage start Verify it’s running:\ntail -f /var/log/garage.log or\ngarage status Garage still needs a layout and a user. Get the Garage node id with:\n/usr/local/bin/garage status | tail -1 | awk \u0026#39;{ print $1 }\u0026#39; Assign a layout (substitute your zone name, capacity, and node ID):\n/usr/local/bin/garage layout assign -z \u0026lt;name\u0026gt; -c \u0026lt;sizeGB\u0026gt; \u0026lt;nodeid\u0026gt; For example:\n/usr/local/bin/garage layout assign -z testzerofs -c 20GB 84d82b9a0310eb03 Apply the layout:\n/usr/local/bin/garage layout apply --version 1 Confirm the layout:\n/usr/local/bin/garage layout show Create an administrative user (the following credentials are an example, do not skip this step by using the same values further on):\n/usr/local/bin/garage key create admin Key name: admin Key ID: GKd998a885674ec7e6eaaa3587 Secret key: 088946b12c01af906fb7e2ab1ee231cbe64dd1ca4728458628516c89c3374a1f Can create buckets: false Key-specific bucket aliases: Authorized buckets: You’ll use the Key ID and Secret Key with ZeroFS.\nCreate a bucket and grant access:\n/usr/local/bin/garage bucket create files /usr/local/bin/garage bucket allow --read --write --owner files --key admin Optionally, create /root/.minio-client/config.json so minio-client can list the bucket contents. Adjust the Garage alias to match your admin user (Key ID → accessKey, Secret key → secretKey):\n{ \u0026#34;version\u0026#34;: \u0026#34;10\u0026#34;, \u0026#34;aliases\u0026#34;: { \u0026#34;gcs\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://storage.googleapis.com\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v2\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;dns\u0026#34; }, \u0026#34;local\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;http://127.0.0.1:9000\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;garage\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;http://127.0.0.1:3900\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;GKd998a885674ec7e6eaaa3587\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;088946b12c01af906fb7e2ab1ee231cbe64dd1ca4728458628516c89c3374a1f\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;play\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://play.min.io\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;Q3AM3UQ867SPQQA43P2F\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;auto\u0026#34; }, \u0026#34;s3\u0026#34;: { \u0026#34;url\u0026#34;: \u0026#34;https://s3.amazonaws.com\u0026#34;, \u0026#34;accessKey\u0026#34;: \u0026#34;YOUR-ACCESS-KEY-HERE\u0026#34;, \u0026#34;secretKey\u0026#34;: \u0026#34;YOUR-SECRET-KEY-HERE\u0026#34;, \u0026#34;api\u0026#34;: \u0026#34;S3v4\u0026#34;, \u0026#34;path\u0026#34;: \u0026#34;dns\u0026#34; } } } Install ZeroFS # ZeroFS is in the ports tree, though a pre-built pkg may or may not be available. This guide is specific to version 0.10.5 and will not be relevant on newer versions due to changes in ZeroFS configuration setup.\nTo install:\npkg install zerofs To build from ports:\npkg install git pkgconf gmake rust echo \u0026#34;DEFAULT_VERSIONS+=ssl=openssl\u0026#34; \u0026gt; /etc/make.conf git clone --depth=1 -b main https://git.freebsd.org/ports.git /usr/ports cd /usr/ports/filesystems/zerofs/ make install Building Zerofs will take a little while.\nBecause there’s no rc service/variables yet, use a small wrapper script to start ZeroFS with the right environment.\nCreate /root/bin/start-zerofs.sh (set AWS_ACCESS_KEY_ID to your Garage admin Key ID and AWS_SECRET_ACCESS_KEY to the Secret key):\n#!/bin/sh SLATEDB_CACHE_DIR=/tmp/zerofs-cache export SLATEDB_CACHE_DIR SLATEDB_CACHE_SIZE_GB=5 export SLATEDB_CACHE_SIZE_GB ZEROFS_ENCRYPTION_PASSWORD=test-secret-test export ZEROFS_ENCRYPTION_PASSWORD AWS_ENDPOINT=http://127.0.0.1:3900 export AWS_ENDPOINT AWS_ACCESS_KEY_ID=GKd998a885674ec7e6eaaa3587 export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=088946b12c01af906fb7e2ab1ee231cbe64dd1ca4728458628516c89c3374a1f export AWS_SECRET_ACCESS_KEY AWS_ALLOW_HTTP=true export AWS_ALLOW_HTTP AWS_DEFAULT_REGION=garage export AWS_DEFAULT_REGION ZEROFS_MEMORY_CACHE_SIZE_GB=1 export ZEROFS_MEMORY_CACHE_SIZE_GB ZEROFS_NFS_HOST=0.0.0.0 export ZEROFS_NFS_HOST ZEROFS_NFS_HOST_PORT=2049 export ZEROFS_NFS_HOST_PORT ZEROFS_9P_HOST=0.0.0.0 export ZEROFS_9P_HOST ZEROFS_9P_PORT=5564 export ZEROFS_9P_PORT /usr/local/bin/zerofs s3://files Make it executable:\nchmod +x /root/bin/start-zerofs.sh Create the cache directory:\nmkdir -p /tmp/zerofs-cache Start ZeroFS (a tmux session is handy to keep it running):\n/root/bin/start-zerofs.sh Test mounting the NFS share # On another host, create a mount point:\nmkdir -p /mnt/files Mount the NFS export (replace the IP with your ZeroFS host):\nmount -t nfs -o nfsv3,nolockd,bg,hard,tcp,port=2049,mountport=2049,rsize=1048576,wsize=1048576 10.0.0.10:/ /mnt/files Create some test files:\ncd /mnt/files echo $(openssl rand -base64 1000) \u0026gt; test1.txt echo $(openssl rand -hex 1000) \u0026gt; test2.txt echo \u0026#34;This is my sample text\u0026#34; \u0026gt; test3.txt (Or copy in larger files such as an ISO, photos, or videos.)\nUnmount:\numount /mnt/files You may see a benign error because there are no RPC services on the ZeroFS host:\numount /mnt/files umount: 10.0.0.10: MOUNTPROG: RPC: Remote system error - Connection refused Re-mount:\nmount -t nfs -o nfsv3,nolockd,bg,hard,tcp,port=2049,mountport=2049,rsize=1048576,wsize=1048576 10.0.0.10:/ /mnt/files Confirm contents:\nls -al /mnt/files For mount details:\nnfsstat -m Umount again:\numount /mnt/files To make the mount persistent, first enable the NFS client:\nsysrc nfs_client_enable=\u0026#34;YES\u0026#34; Then add this to /etc/fstab (use tabs; substitute your host IP):\n# 10.0.0.10:/\t/mnt/files\tnfs\trw,nolockd,bg,hard,tcp,port=2049,mountport=2049,rsize=1048576,wsize=1048576\t0\t0 Reboot:\nshutdown -r now After boot, check /mnt/files to confirm the NFS share is mounted and your files are present.\nWhat about 9P? # This post doesn’t cover mounting the 9P share. We need FreeBSD 15’s virtio_p9fs module for straightforward 9P client support. It may be possible to use vm-bhyve 9P features, but that’s outside this post’s scope.\nConclusion # If everything went smoothly, you now have a working proof of concept: S3-backed storage serving an NFS export via ZeroFS.\nQuestions, suggestions, or improvements? Reach out on Mastodon.\n","date":"August 29, 2025","externalUrl":null,"permalink":"/blog/2025-08-29/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eZeroFS: S3-backed NFS Service on FreeBSD (Proof of Concept, Part 1) \n    \u003cdiv id=\"zerofs-s3-backed-nfs-service-on-freebsd-proof-of-concept-part-1\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#zerofs-s3-backed-nfs-service-on-freebsd-proof-of-concept-part-1\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\u003ch3 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\n  \n\n\n\n\u003cdiv\n  \n    class=\"flex px-4 py-3 rounded-md bg-primary-100 dark:bg-primary-900\"\n  \n  \u003e\n  \u003cspan\n    \n      class=\"text-primary-400 ltr:pr-3 rtl:pl-3 flex items-center\"\n    \n    \u003e\n    \n\n  \u003cspan class=\"relative block icon\"\u003e\n    \u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\"\u003e\u003cpath fill=\"currentColor\" d=\"M506.3 417l-213.3-364c-16.33-28-57.54-28-73.98 0l-213.2 364C-10.59 444.9 9.849 480 42.74 480h426.6C502.1 480 522.6 445 506.3 417zM232 168c0-13.25 10.75-24 24-24S280 154.8 280 168v128c0 13.25-10.75 24-23.1 24S232 309.3 232 296V168zM256 416c-17.36 0-31.44-14.08-31.44-31.44c0-17.36 14.07-31.44 31.44-31.44s31.44 14.08 31.44 31.44C287.4 401.9 273.4 416 256 416z\"/\u003e\u003c/svg\u003e\n\n  \u003c/span\u003e\n\n\n  \u003c/span\u003e\n\n  \u003cspan\n    \n      class=\"dark:text-neutral-300\"\n    \n    \u003e\u003cp\u003ePlease note that we found out while operating the PoC that Garage ignores the conditions in conditional writes. As this feature is required by ZeroFS, this means \u003ca\n  href=\"https://github.com/Barre/ZeroFS/issues/121\"\n    target=\"_blank\"\n  \u003ethat ZeroFS is actually not reliably working with Garage at the moment\u003c/a\u003e.\u003c/p\u003e","title":"ZeroFS: S3-backed NFS Service on FreeBSD (Proof of Concept, Part 1)","type":"blog"},{"content":" Build and Run Your Own FreeBSD-native Containers with Buildah, Containerfiles and Podman # This is an updated version of an earlier blog post. There is now an official FreeBSD container image and the approach of a base image with clones is no longer supported\nIntroduction # Would you like to run a full Docker-style setup under FreeBSD?\nDid you hear it\u0026rsquo;s not possible? Not production-ready? Or a bit iffy?\nWould you like to try anyway?\nBuilding a container with buildah from Containerfiles # On your host, or newly-created virtual machine:\npkg update pkg install podman-suite To properly support Podman\u0026rsquo;s container restart policy, conmon needs fdescfs(5) to be mounted on /dev/fd.\necho \u0026#34;fdesc /dev/fd fdescfs rw 0 0\u0026#34; \u0026gt;\u0026gt; /etc/fstab mount -t fdescfs fdesc /dev/fd Create /etc/pf.conf with the following contents. If the network interface is not vtnet0, adjust to the correct interface name:\n# Change these to the interface(s) with the default route v4egress_if = \u0026#34;vtnet0\u0026#34; v6egress_if = \u0026#34;vtnet0\u0026#34; nat on $v4egress_if inet from \u0026lt;cni-nat\u0026gt; to any -\u0026gt; ($v4egress_if) nat on $v6egress_if inet6 from \u0026lt;cni-nat\u0026gt; to !ff00::/8 -\u0026gt; ($v6egress_if) rdr-anchor \u0026#34;cni-rdr/*\u0026#34; nat-anchor \u0026#34;cni-rdr/*\u0026#34; table \u0026lt;cni-nat\u0026gt; Make sure to enable and start pf:\nservice pf enable service pf start Make sure to enable and start the podman_service and podman services:\nservice podman_service enable service podman enable service podman_service start service podman start To get started, import the FreeBSD Official OCI Image into the local image repo:\npodman load -i=https://download.freebsd.org/releases/OCI-IMAGES/14.2-RELEASE/amd64/Latest/FreeBSD-14.2-RELEASE-amd64-container-image-minimal.txz Then create a minio container. First make a directory and enter it, then create Containerfile and entrypoint.sh as follows:\nmkdir -p mycontainers/minio cd mycontainers/minio nano Containerfile Add to Containerfile:\nFROM localhost/freebsd14-minimal:14.2-RELEASE-amd64 MAINTAINER Your Name \u0026lt;your@email.address\u0026gt; # Set default environment variables ENV MINIO_USER=\u0026#34;admin\u0026#34; ENV MINIO_PASS=\u0026#34;l0ng-c0mpl1c4t3d-p4ssw0rd\u0026#34; # setup pkg source RUN mkdir -p /usr/local/etc/pkg/repos ADD FreeBSD.conf /usr/local/etc/pkg/repos/FreeBSD.conf # bootstrap pkg RUN ASSUME_ALWAYS_YES=yes pkg bootstrap -f RUN ASSUME_ALWAYS_YES=yes pkg update -f # install openssl and ca_root_nss RUN ASSUME_ALWAYS_YES=yes pkg install -y openssl RUN ASSUME_ALWAYS_YES=yes pkg install -y ca_root_nss # Install minio RUN ASSUME_ALWAYS_YES=yes pkg install -y minio RUN ASSUME_ALWAYS_YES=yes pkg clean -ay # Set entrypoint ADD entrypoint.sh /usr/local/bin/entrypoint.sh ENTRYPOINT \u0026#34;/usr/local/bin/entrypoint.sh\u0026#34; Create the entrypoint.sh file with executable permissions:\nnano entrypoint.sh \u0026amp;\u0026amp; chmod +x entrypoint.sh Add to entrypoint.sh:\n#!/bin/sh MINIO_ROOT_USER=\u0026#34;${MINIO_USER}\u0026#34; MINIO_ROOT_PASSWORD=\u0026#34;${MINIO_PASS}\u0026#34; MINIO_PROMETHEUS_AUTH_TYPE=public /usr/local/bin/minio --quiet server --address=\u0026#34;:9000\u0026#34; --console-address :9001 /var/db/minio Create a FreeBSD.conf file with contents:\nFreeBSD: { url: \u0026#34;pkg+http://pkg.FreeBSD.org/${ABI}/latest\u0026#34;, mirror_type: \u0026#34;srv\u0026#34;, signature_type: \u0026#34;fingerprints\u0026#34;, fingerprints: \u0026#34;/usr/share/keys/pkg\u0026#34;, enabled: yes } Create the minio container and publish to localhost repository in one command:\nsudo buildah bud -t minio-0.0.1 . cd .. Persistent storage # For persistent storage, the container will use a mounted in ZFS dataset.\nCreate a ZFS dataset for the persistent data, assumes zroot/data is at /mnt/data.\nsudo zfs create zroot/data/minio Running a Container With podman and Variables # Run the container with podman as follows:\nsudo podman run -dt \\ --ip=10.88.0.2 \\ --volume \u0026#34;/mnt/data/minio:/var/db/minio:rw\u0026#34; \\ -e MINIO_USER=\u0026#34;admin\u0026#34; \\ -e MINIO_PASS=\u0026#34;set-your-own-password\u0026#34; \\ -h minio1 \\ minio-0.0.1:latest Check the container is running with:\nsudo podman ps Minio will be available at 10.88.0.2:9000\nIf you wish to expose minio to rest of network, run podman with the following options, and minio will be avalable at \u0026lt;host-ip\u0026gt;:9000:\nsudo podman run -dt \\ --ip=10.88.0.2 \\ --volume \u0026#34;/mnt/data/minio:/var/db/minio:rw\u0026#34; \\ -e MINIO_USER=\u0026#34;admin\u0026#34; \\ -e MINIO_PASS=\u0026#34;set-your-own-password\u0026#34; \\ -p 9000:9000 \\ -p 9001:9001 \\ -h minio1 \\ minio-0.0.1:latest Conclusion # It\u0026rsquo;s possible to run Docker-style FreeBSD-native, containers on FreeBSD.\nMore complicated setups for virtual data center environments are covered in this blog post.\n","date":"February 28, 2025","externalUrl":null,"permalink":"/blog/2025-02-28/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eBuild and Run Your Own FreeBSD-native Containers with Buildah, Containerfiles and Podman \n    \u003cdiv id=\"build-and-run-your-own-freebsd-native-containers-with-buildah-containerfiles-and-podman\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#build-and-run-your-own-freebsd-native-containers-with-buildah-containerfiles-and-podman\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003cem\u003eThis is an updated version of an earlier blog post. There is now an official FreeBSD container image and the approach of a base image with clones is no longer supported\u003c/em\u003e\u003c/p\u003e","title":"Build and run your own FreeBSD-native containers with Buildah, Containerfiles and Podman (updated!)","type":"blog"},{"content":"","date":"February 28, 2025","externalUrl":null,"permalink":"/tags/buildah/","section":"Tags","summary":"","title":"Buildah","type":"tags"},{"content":"","date":"February 28, 2025","externalUrl":null,"permalink":"/tags/freebsd-containers/","section":"Tags","summary":"","title":"Freebsd Containers","type":"tags"},{"content":" Impact Enterprise \u0026amp; Digital Avisory, operating since 2004 in Europe and Southern Africa.\nWe are a team of engineers and researchers working together to solve difficult problems for ourselves and our clients.\nWe provide independent Digitalisation Advisory Services and Enterprise Architecture concepts that are off the beaten track.\nOur focus is on Digital Sovereignty for Europe and South Africa.\nInstead of force-feeding you with the usual large-cap tech company products, you might end up with a tailor-made solution incorporating open-source software which is exactly what your business needs.\nAnd we know what we are talking about: Our business runs on open-source software, too.\n","date":"January 1, 2025","externalUrl":null,"permalink":"/","section":"","summary":"\u003cdiv class=\"lead text-neutral-500 dark:text-neutral-400 !mb-9 text-xl\"\u003e\n  \u003cp\u003e\u003cb\u003eImpact Enterprise \u0026amp; Digital Avisory\u003c/b\u003e, operating since 2004 in Europe and Southern Africa.\u003c/p\u003e\n\u003cp\u003eWe are a team of engineers and researchers working together to solve difficult problems for ourselves and our clients.\u003c/p\u003e","title":"","type":"page"},{"content":"","date":"August 20, 2024","externalUrl":null,"permalink":"/tags/ansible/","section":"Tags","summary":"","title":"Ansible","type":"tags"},{"content":"","date":"August 20, 2024","externalUrl":null,"permalink":"/tags/micropod/","section":"Tags","summary":"","title":"Micropod","type":"tags"},{"content":" Micropod-sampler: A minimal viable FreeBSD-based container virtual data center # The Micropod-sampler ansible playbook is a minimal viable OCI container-based virtual data center.\nThe host is FreeBSD. The containers are FreeBSD too.\nThe tooling is buildah and podman which provides a Docker-like container experience on FreeBSD.\nThe following playbook will configure everything needed to run a small virtual data center with:\nminio for S3 consul for service orchestration nomad for job management traefik-consul for routing (not in use) nginx-s3 nomad job to load website from S3 bucket in minio Prerequisites # You will need a FreeBSD 14.0+ virtual machine or host.\nThe package stream must be configured to use latest packages:\nmkdir -p /usr/local/etc/pkg/repos/ echo \u0026#39;FreeBSD: { url: \u0026#34;pkg+http://pkg.FreeBSD.org/${ABI}/latest\u0026#34; }\u0026#39; \u0026gt; /usr/local/etc/pkg/repos/FreeBSD.conf sudo pkg update -f sudp pkg upgrade You will need SSH keys configured on the host for automatic access, or pass in --ask-become-pass to the ansible-playbook commands.\nRun playbook on your own computer # Clone the git repo and change into the directory:\ngit clone https://codeberg.org/Honeyguide/micropod-sampler.git cd micropod-sampler Create a python virtual environment:\npython3 -m pip install virtualenv python3 -m venv .venv source .venv/bin/activate (.venv) .venv/bin/python3 -m pip install --upgrade pip (.venv) .venv/bin/python3 -m pip install -r requirements.txt Copy the hosts.sample file to hosts and edit to your needs:\ncp inventory/hosts.sample inventory/hosts Edit the file to change temporary holding values YOUR-USERNAME once, and YOUR-HOST-IP twice.\nIf the network interface is not vtnet0, adjust to the correct interface name:\n[all:vars] my_default_username=YOUR-USERNAME # set to python3 or more specifically python3.9 or python3.12 [local] localhost ansible_connection=local ansible_python_interpreter=\u0026#34;env python3.12\u0026#34; [servers] server1 ansible_host=YOUR-HOST-IP ansible_port=22 [servers:vars] my_network_interface=vtnet0 my_default_ip=YOUR-HOST-IP bsd_packages_stream=latest my_datacenter=micropod external_dns_server=1.1.1.1 pf_ipv6_enable=yes minio_name=myminio minio_user=admin minio_pass=s4mpl3p4ssw0rd minio_bucket=mybucket minio_ip_address=10.88.0.10 consul_ip_address=10.88.0.11 consul_gossip_key=\u0026#34;oSPiRcrbp96JoWYO6SNzpAgItM14zlvZiw2OPT0UGmA=\u0026#34; nomad_ip_address=10.88.0.12 nomad_gossip_key=\u0026#34;oSPiRcrbp96JoWYO6SNzpAgItM14zlvZiw2OPT0UGmA=\u0026#34; traefik_ip_address=10.88.0.13 nginx_s3_ip=10.88.0.20 nginx_s3_port=25000 Run the script to provision your virtual machine or host:\n(.venv) .venv/bin/ansible-playbook -i inventory/hosts site.yml Output of successful completion will look like this:\nTASK [add_nomad_jobs : Debug nomad job status] ************************************************************************* ok: [server1] =\u0026gt; { \u0026#34;nomad_job_status.stdout_lines\u0026#34;: [ \u0026#34;ID Type Priority Status Submit Date\u0026#34;, \u0026#34;nginx-s3 service 50 running 2024-08-04T17:15:46+02:00\u0026#34; ] } TASK [add_nomad_jobs : Check if nginx-s3 is performing correctly] ****************************************************** changed: [server1] TASK [add_nomad_jobs : Debug nginx-s3 status] ************************************************************************** ok: [server1] =\u0026gt; { \u0026#34;nginx_s3_status.stdout_lines\u0026#34;: [ \u0026#34;\u0026lt;html\u0026gt;\u0026#34;, \u0026#34;\u0026lt;head\u0026gt;\u0026#34;, \u0026#34;\u0026lt;title\u0026gt;S3 default page\u0026lt;/title\u0026gt;\u0026#34;, \u0026#34;\u0026lt;/head\u0026gt;\u0026#34;, \u0026#34;\u0026lt;body\u0026gt;\u0026#34;, \u0026#34;\u0026lt;p\u0026gt;blank page\u0026lt;/p\u0026gt;\u0026#34;, \u0026#34;\u0026lt;/body\u0026gt;\u0026#34;, \u0026#34;\u0026lt;/html\u0026gt;\u0026#34; ] } PLAY RECAP ************************************************************************************************************* localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 server1 : ok=92 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 This confirms the nginx container, loaded via nomad job, is able to serve the files in minio S3 bucket.\nWhat next? # Take a look in roles/create_container_files/files/micropod/ and see how Containerfile are setup, with entrypoint.sh and associated configuration files.\nBuild new containers based on how these are configured.\nThe playbook can be adapted to build and run additional containers.\n","date":"August 20, 2024","externalUrl":null,"permalink":"/blog/2024-08-20/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eMicropod-sampler: A minimal viable FreeBSD-based container virtual data center \n    \u003cdiv id=\"micropod-sampler-a-minimal-viable-freebsd-based-container-virtual-data-center\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#micropod-sampler-a-minimal-viable-freebsd-based-container-virtual-data-center\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eThe \u003ccode\u003eMicropod-sampler\u003c/code\u003e ansible playbook is a minimal viable OCI container-based virtual data center.\u003c/p\u003e","title":"Micropod-sampler: A minimal viable FreeBSD-based container virtual data center","type":"blog"},{"content":" Build and run your own FreeBSD-native containers with Buildah, Containerfiles and Podman # This is a deprecated version of the blog post. There is now an official FreeBSD container image and the approach of a base image with clones is no longer supported\nWould you like to run a full Docker-style setup under FreeBSD?\nDid you hear it\u0026rsquo;s not possible? Not production-ready? Or a bit iffy?\nWould you like to try anyway?\nBuilding a container with Buildah from Containerfiles # On your host, or newly-created virtual machine:\npkg update pkg install podman-suite Create /etc/pf.conf with the following contents. If the network interface is not vtnet0, adjust to the correct interface name:\n# Change these to the interface(s) with the default route v4egress_if = \u0026#34;vtnet0\u0026#34; v6egress_if = \u0026#34;vtnet0\u0026#34; nat on $v4egress_if inet from \u0026lt;cni-nat\u0026gt; to any -\u0026gt; ($v4egress_if) nat on $v6egress_if inet6 from \u0026lt;cni-nat\u0026gt; to !ff00::/8 -\u0026gt; ($v6egress_if) rdr-anchor \u0026#34;cni-rdr/*\u0026#34; nat-anchor \u0026#34;cni-rdr/*\u0026#34; table \u0026lt;cni-nat\u0026gt; Make sure to enable and start pf:\nservice pf enable service pf start Make sure to enable and start the podman_service service:\nservice podman_service enable service podman_service start Create a working directory, with a base directory inside:\nmkdir -p mycontainers/base cd mycontainers Start off by adding the Containerfile file, which uses an image pulled from dougrabson/freebsd14.0-minimal.\ncd base nano Containerfile This still will inform buildah to update the package sources, and install pkg and ca_root_nss, along with setting the entrypoint.\nAdd to Containerfile:\nFROM quay.io/dougrabson/freebsd14.0-minimal RUN pkg update -f RUN pkg install -y pkg RUN pkg install -y ca_root_nss ADD entrypoint.sh /usr/local/bin/entrypoint.sh ENTRYPOINT \u0026#34;/usr/local/bin/entrypoint.sh\u0026#34; Create the entrypoint.sh file with executable permissions:\nnano entrypoint.sh \u0026amp;\u0026amp; chmod +x entrypoint.sh Add to entrypoint.sh as below. No actual commands are included, because this is a base image:\n#!/bin/sh # echo \u0026#34;micropod base image\u0026#34; Create the base container, and publish to the localhost repository in one command:\nsudo buildah bud -t micropod-base-0.0.1 . cd .. When done, verify the base image is there with:\nsudo buildah images Now create an additional container using this one as a base. This elminates steps for pkg updating and makes things faster with additional containers.\nCreate a minio container. First make a directory, change to it, then create Containerfile and entrypoint.sh as follows:\ncd ~mycontainers mkdir minio; cd minio nano Containerfile Add to Containerfile:\nFROM localhost/micropod-base-0.0.1 # Set default environment variables ENV MINIO_USER=\u0026#34;admin\u0026#34; ENV MINIO_PASS=\u0026#34;l0ng-c0mpl1c4t3d-p4ssw0rd\u0026#34; # Install minio RUN pkg install -y minio # Set entrypoint ADD entrypoint.sh /usr/local/bin/entrypoint.sh ENTRYPOINT \u0026#34;/usr/local/bin/entrypoint.sh\u0026#34; Create the entrypoint.sh file with executable permissions:\nnano entrypoint.sh \u0026amp;\u0026amp; chmod +x entrypoint.sh Add to entrypoint.sh:\n#!/bin/sh MINIO_ROOT_USER=\u0026#34;${MINIO_USER}\u0026#34; MINIO_ROOT_PASSWORD=\u0026#34;${MINIO_PASS}\u0026#34; MINIO_PROMETHEUS_AUTH_TYPE=public /usr/local/bin/minio --quiet server --address=\u0026#34;:9000\u0026#34; --console-address :9001 /var/db/minio Create the minio container and publish to localhost repository in one command:\nsudo buildah bud -t minio-0.0.1 . cd .. Persistent storage # For persistent storage, the container will use a mounted in ZFS dataset.\nCreate a ZFS dataset for the persistent data, assumes zroot/data is at /mnt/data.\nsudo zfs create zroot/data/minio Running a container with podman and variables # Run the container with podman as follows:\nsudo podman run -dt \\ --ip=10.88.0.2 \\ --volume \u0026#34;/mnt/data/minio:/var/db/minio:rw\u0026#34; \\ -e MINIO_USER=\u0026#34;admin\u0026#34; \\ -e MINIO_PASS=\u0026#34;set-your-own-password\u0026#34; \\ -h minio1 \\ minio-0.0.1:latest Check the container is running with:\nsudo podman ps Minio will be available at 10.88.0.2:9000\nIf you wish to expose minio to rest of network, run podman with the following options, and minio will be avalable at \u0026lt;host-ip\u0026gt;:9000:\nsudo podman run -dt \\ --ip=10.88.0.2 \\ --volume \u0026#34;/mnt/data/minio:/var/db/minio:rw\u0026#34; \\ -e MINIO_USER=\u0026#34;admin\u0026#34; \\ -e MINIO_PASS=\u0026#34;set-your-own-password\u0026#34; \\ -p 9000:9000 \\ -p 9001:9001 \\ -h minio1 \\ minio-0.0.1:latest Conclusion # It\u0026rsquo;s possible to run Docker-style FreeBSD-native, containers on FreeBSD.\nMore complicated setups for virtual data center environments will be covered in the next blog post.\n","date":"August 19, 2024","externalUrl":null,"permalink":"/blog/2024-08-19/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eBuild and run your own FreeBSD-native containers with Buildah, Containerfiles and Podman \n    \u003cdiv id=\"build-and-run-your-own-freebsd-native-containers-with-buildah-containerfiles-and-podman\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#build-and-run-your-own-freebsd-native-containers-with-buildah-containerfiles-and-podman\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003cem\u003eThis is a deprecated version of the blog post. There is now an official FreeBSD container image and the approach of a base image with clones is no longer supported\u003c/em\u003e\u003c/p\u003e","title":"Build and run your own FreeBSD-native containers with Buildah, Containerfiles and Podman","type":"blog"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/fediverse/","section":"Tags","summary":"","title":"Fediverse","type":"tags"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/jails/","section":"Tags","summary":"","title":"Jails","type":"tags"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/postgresql/","section":"Tags","summary":"","title":"Postgresql","type":"tags"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/potluck/","section":"Tags","summary":"","title":"Potluck","type":"tags"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/redis/","section":"Tags","summary":"","title":"Redis","type":"tags"},{"content":" Introduction # Have you heard of Mastodon?\nMastodon is a self-hosted social networking service, where independently run servers federate content to each other using ActivityPub.\nEach Mastodon server has its own set of users, own code of conduct, own terms and moderation policies.\nUsers post short messages (called \u0026ldquo;toots\u0026rdquo;) to the world, or to select people. They can also subscribe to other users\u0026rsquo; feeds on any server.\nActivityPub is a standard for the Internet in the Social Web Networking Group of the World Wide Web Consortium (W3C).\nIt is an open, decentralized social networking protocol which provides an API for managing content, and federated server-to-server notifications and content distribution.\nThe collective term for federated services which use ActivityPub is \u0026ldquo;the fediverse\u0026rdquo;, a portmanteau of \u0026ldquo;federation\u0026rdquo; and \u0026ldquo;universe\u0026rdquo;.\nMastodon is one part of the fediverse, where Wikipedia maintains a listing of others.\nWhat is Involved in Your Own Mastodon Instance? # Your own Mastodon instance will require a few things:\nDomain name and fixed IP postgresql redis Webserver with support for ruby, node S3-compatible file store (see How To Set Up a Minio Cluster From Potluck, Complete With Nextcloud, Monitoring And Alerting) The setup itself is mainly configuring a number of parameters as described in this article and running the ansible playbook described in this article which will pull the container images, configure and start them.\nEasy Setup on FreeBSD With Potluck # There is an easy install using an ansible playbook for FreeBSD systems, that will provide a complete environment with monitoring and alerting.\nYou will need a FreeBSD 13.2 or 14.0 server with the following prerequisites:\nYou must configure the server\u0026rsquo;s default network interface to be labelled \u0026lsquo;untrusted\u0026rsquo;. For example, with interface em0:\nifconfig_em0_name=\u0026#34;untrusted\u0026#34; ifconfig_untrusted=\u0026#34;inet 192.168.1.1 netmask 255.255.255.0\u0026#34; You will also need a domain name, with the subdomains mastodon.domain.name and files.domain.name pointing at the same IP.\nThe ansible script below will provision a version of Mastodon with a patch to allow 5000 character posts, alternatively you can also pull the official Mastodon versions from https://github.com/mastodon/mastodon.\nObtaining The Playbook And Setting Up Environment # Clone the git repo and configure the environment as follows:\ngit clone https://codeberg.org/Honeyguide/mastodon-sampler.git cd mastodon-sampler Set Up a Python Virtual Environment to Ensure Correct Version Of ansible # Set up a python virtual environment to ensure correct version of ansible as below.\nIf not already installed, install python virtualenv:\npython3 -m pip install virtualenv Then create a virtual environment and activate it, before installing the required dependencies:\npython3 -m venv .venv source .venv/bin/activate (.venv) .venv/bin/python3 -m pip install --upgrade pip (.venv) .venv/bin/python3 -m pip install -r requirements.txt It\u0026rsquo;s important to note that you will run the ansible-playbook executable from the virtual environment created, but do not do this yet!\n.venv/bin/ansible-playbook -i [inventory] [playbook] Update The Inventory Parameters # Update the inventory parameters to reflect all your specific details:\n(.venv) cd inventory (.venv) cp hosts.sample hosts (.venv) nano hosts Set your default username on the remote server in the following field. This is the username you ssh to your server as:\nmy_default_username=REPLACE-WITH-USERNAME Set the hostname or IP address of your server in the following field, do not change other fields on this line:\n[servers] server1 ansible_host=REPLACE-WITH-IP-HOSTNAME ... If your server has IPv6 enabled, make sure to set the following field from no to yes:\npf_ipv6_enable=yes Update the following fields with a long complicated string of characters to secure your minio instance. This is the minio admin user:\nminio_user=REPLACE-WITH-LONG-MINIO-USERNAME minio_pass=REPLACE-WITH-LONG-MINIO-PASSWORD Update the following fields to set your email address and domain names for certificate registration:\nmy_acme_email=REPLACE-WITH-EMAIL-ADDRESS my_acme_domain=REPLACE-WITH-DOMAIN-NAME my_acme_alias1=mastodon.REPLACE-WITH-DOMAIN-NAME my_acme_alias2=files.REPLACE-WITH-DOMAIN-NAME Update the haproxy frontend IP address to match your server\u0026rsquo;s public IPv4 address:\nhaproxy_public_ip=REPLACE-FRONTEND-IP Scroll down and set a grafana admin user and password:\npot_beast_grafana_user=REPLACE-GRAFANA-USER pot_beast_grafana_pass=REPLACE-GRAFANA-PASSWORD Also include mail server parameters for alertmanager notices:\npot_beast_smtphostport=localhost:25 pot_beast_smtp_from=notices@REPLACE-WITH-DOMAIN-NAME pot_beast_alertaddress=REPLACE-WITH-EMAIL pot_beast_smtp_user= pot_beast_smtp_pass= Further down the file is the following setting, configure it as files.your.domain:\npot_nomad_s3_domain_name=REPLACE-WITH-FILES-DOMAIN-NAME Scroll further down and modify the zincsearch admin user and password:\npot_zincsearch_user=SET-ZINC-USER pot_zincsearch_pass=SET-ZINC-PASSWORD Page down some more and set a postgresql_exporter password:\npot_postgresql_exporter_pass=REPLACE-PGEXPORTER-PASS You can also set the Mastodon user password for postgresql:\npot_postgresql_mastodon_password=REPLACE-WITH-MASTODON-DB-PASSWORD A little further down are parameters for the Mastodon instance.\nSet the domain name of your mastodon instance, for the purposes of this playbook make it mastodon.your.domain:\npot_mastodon_domain=REPLACE-WITH-DOMAIN-NAME Configure email notifications for Mastodon with the following fields:\npot_mastodon_to_email=YOUR-EMAIL pot_mastodon_from_email=YOUR-EMAIL pot_mastodon_email_host=YOUR-EMAIL-HOST pot_mastodon_email_port=465 pot_mastodon_smtp_user=YOUR-SMTP-USER pot_mastodon_smtp_pass=YOUR-SMTP-PASS Use the same postgresql password as above:\npot_mastodon_db_pass=REPLACE-WITH-MASTODON-DB-PASSWORD Then update the following field to match the files.your.domain configuration:\npot_mastodon_public_s3_host=files.REPLACE-WITH-DOMAIN-NAME For the purposes of this playbook we don\u0026rsquo;t set any secret_key or otp_key and related, we\u0026rsquo;ll allow the image to generate them. Leave these fields blank to have them autogenerated in the resulting image.\nFinally, scroll to the very bottom and update the following field with a long complicated password for the Mastodon minio user. This must be different from the admin password:\nmastodon_minio_pass=REPLACE-WITH-LONG-MINIO-MASTODON-PASSWORD Run The playbook # Change back to the parent directory and run the playbook as follows:\n(.venv) cd .. (.venv) .venv/bin/ansible-playbook -i inventory/hosts site.yml The playbook will take approximately 60-90mins to provision your Mastodon server.\nWhen complete, a QR code for a wireguard link will be displayed. Make sure to get a screen capture of this, then load it into a QR Code viewer to extract the text.\nCopy the extracted text into your wireguard client and then connect.\nYou can now connect to the IP address displayed in the final task in the playbook.\nOpen http://10.1.2.10 (or whichever IP is assigned) to get an index page to the various internal services.\nFrom here you can access the links, for example if you want to see charts you can access grafana at http://10.1.2.250:3000 with the user/pass you configured in the inventory file.\nAll the monitoring services are on the same fixed internal IP 10.1.2.250, but the port changes between services.\nYou can also access dashboards for consul, nomad and traefik which are on different IP addresses.\nIf you need to access minio, you can do so directly at https://10.1.1.1:9000. Make sure to accept the self-signed certificate TWICE because the port changes automatically and you need to accept it for each port.\nConfigure First Mastodon User # Open mastodon.your.domain and setup your first user.\nIn Case Of Problems # If you have any problems, you can clean the system and try over.\nIf you misconfigured your inventory, or something went wrong, you can run the clean.yml playbook to clean your system of all images and data.\nImportant: make sure to disconnect your wireguard session before running clean.yml.\n(.venv) .venv/bin/ansible-playbook -i inventory/hosts clean.yml All data will be wiped too so don\u0026rsquo;t run this on a system in regular use. It\u0026rsquo;s for cleaning a bad initial install.\nThen make changes to the playbook or inventory and run site.yml again to provision.\n","date":"February 16, 2024","externalUrl":null,"permalink":"/blog/2024-02-16/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eHave you heard of \u003ca\n  href=\"https://joinmastodon.org/\"\n    target=\"_blank\"\n  \u003eMastodon\u003c/a\u003e?\u003c/p\u003e\n\u003cp\u003eMastodon is a self-hosted social networking service, where independently run servers federate content to each other using ActivityPub.\u003c/p\u003e","title":"Run Your Own Mastodon Server on FreeBSD in a Potluck Container","type":"blog"},{"content":"","date":"February 16, 2024","externalUrl":null,"permalink":"/tags/social-media/","section":"Tags","summary":"","title":"Social Media","type":"tags"},{"content":"","date":"February 7, 2024","externalUrl":null,"permalink":"/tags/dedicated-server/","section":"Tags","summary":"","title":"Dedicated Server","type":"tags"},{"content":" Introduction # Lets walk through the steps of installing FreeBSD 14.0 on a dedicated server from Xneelo.\nXneelo is a South African hosting provider with dedicated servers and a rescue system, similar to Hetzner.\nFreeBSD is not available as an operating system to install, however, it can installed from the rescue console using the depenguin.me installer.\nPrepare ssh Public Key # Setup your ssh public key in a web-accessible location, such as a text file in a webserver root.\nThis would be the contents of id_rsa.pub, or the contents of .ssh/authorized_keys from an existing server.\nFor example: https://your.host/keys.txt\nSet Up Your Xneelo User # Register on the Xneelo control panel, and configure 2FA. Consult the Xneelo documentation for more infomation.\nIf assisting someone on their account, login as your newly created user, then switch account to access servers and rescue console.\nOrder a Server # Login to the Xneelo control panel and view \u0026ldquo;Self-managed servers\u0026rdquo;.\nMake a selection from the list for the dedicated servers.\nFor OS install, choose \u0026lsquo;Linux\u0026rsquo; and then \u0026lsquo;Debian\u0026rsquo;. This will be overwritten, however, you\u0026rsquo;ll first boot into it and gather some useful information.\nSelect RAM and drive specifications and add any custom comments.\nContinue and complete the process to order a server, providing any necessary payment information.\nLogin to Provisioned Server # A password-protected PDF will be sent via email when the server is provisioned.\nThe PDF contains important information, such as:\nIP addressing Remote console info Initial passwords If you don\u0026rsquo;t already have the necessary customer number, login to the Xneelo control panel, switch to the right account, then get the customer number to use as password to open PDF.\nOn the first ssh connection to the Debian instance as root user, login with the password provided.\nYou will immediately be asked to change your password. This involves inputting the password a second time, and then setting a new one.\nSince you won\u0026rsquo;t be keeping Debian, set any password and continue.\nCheck a few things before rebooting, as below.\nCheck Disks # Check disks and capacities with lsblk. Determine which two drives will form your system mirror array.\nIt might be sda, sdb, or it might be a sdb, sdc from a mix of drive capacities over sda, sdb, sdc, sdd.\nImportant: This might change between reboots!\nCheck Interfaces # Check the interfaces with ip addr and make a note of the MAC address in link/ether for the connected interface.\nAlthough IP addressing is provided via DHCP, you\u0026rsquo;ll manually configure addresses later on. You need the IP, netmask and MAC address.\nCreate a Private IPv6 Address to Use # Use this tool to create a local IPv6 range with the MAC address from the step above.\nFrom the fdxx:xxxx:xxxx:0:/64 range that comes out, jot down fdxx:xxxx:xxxx::1 to use for a local IPv6 address later on.\nGet DNS Servers # Get the DNS servers in /etc/resolv.conf because they need to be included later on.\ncat /etc/resolv.conf Select IPv6 Name Servers # IPv6 is not enabled by default at Xneelo, however we\u0026rsquo;ll include IPv6 nameservers in the setup later. Cloudflare\u0026rsquo;s 1.1.1.1 IPv6 nameservers will do for now:\n2606:4700:4700::1111 2606:4700:4700::1001 Prepare to Reboot # You\u0026rsquo;re done with checking out the information that may be needed. You will reboot after first activating the rescue console.\nActivate Rescue Console # Please refer to the Xneelo rescue docs for latest information!\nOpen the Xneelo Control Panel and log in.\nUnder \u0026lsquo;Products\u0026rsquo; in the side menu, select \u0026lsquo;Self-Managed Servers\u0026rsquo;.\nClick on the server you would like to access.\nOn the \u0026lsquo;Product Overview\u0026rsquo; screen, click on the \u0026lsquo;Linux Rescue System\u0026rsquo; icon.\nSelect \u0026lsquo;Activate\u0026rsquo;.\nImportant: Make a note of the one time password, you will need this to login.\nNow you can reboot the Debian instance at the console. You must do this within 10 minutes of activating the rescue console.\nOnce rebooted, your server will PXE boot and load the rescue system.\nLog Into Rescue Console # On your own computer, make sure the old server keys are removed from your .ssh/known_hosts with the following command:\nssh-keygen -R \u0026lt;xneelo-server-ip\u0026gt; Then connect to your server, now in rescue console mode.\nLog in with the username and password provided at activation time, either from the console or via ssh access.\nInstallation # Unattended Installation With depenguin.me Script # Download And Run The Installer Script # The Xneelo rescue system doesn\u0026rsquo;t like the ssh certificate from https://depenguin.me and needs the --no-check-certificate to work reliably.\nwget --no-check-certificate https://depenguin.me/run.sh chmod +x run.sh ./run.sh https://your.host/keys.txt Wait for the packages to install and Qemu to load. You\u0026rsquo;ll be prompted with connection details when ready.\nConnect To The mfsBSD Instance On Port 1022 # In a new terminal session connect to the mfsBSD instance on port 1022 and change to root:\nssh -p 1022 mfsbsd@your-host-ip sudo su - Check Disks Again # Make sure you have the correct disks for the settings below with the following command:\ngeom disk list If you want a mirror setup (Raid1) and have two disks, make sure it\u0026rsquo;s ada0 ada1, or whatever matches your setup. Xneelo servers change the disk names between boots.\nIf you want a four disk setup with Raid10, you can configure that too, as long as all drive capacities match.\nFor mixed capacity setup with four disk systems, find the correct two drives for your root array. (In a four-disk system this is ada0 ada2 on some boots.)\nWipe Disks # Wipe disks as follows:\n./mfsbsd_clean.sh zroot ada0 ada1 YOU ARE ABOUT TO DESTROY ZPOOL \u0026#39;zroot\u0026#39; AND PARTITIONS ada0 ada1 THIS OPERATION MEANS DATA LOSS AND CANNOT BE UNDONE Please type \u0026#39;ACCEPTDATALOSS\u0026#39; and press enter to continue. ACCEPTDATALOSS ... DONE Create Settings File # cp depenguin_settings.sh.sample depenguin_settings.sh Then edit the settings file with your information\nnano depenguin_settings.sh conf_hostname=\u0026#34;depenguintest\u0026#34; conf_interface=\u0026#34;igb0\u0026#34; conf_ipv4=\u0026#34;1.2.3.4\u0026#34; conf_ipv6=\u0026#34;fdxx:xxxx:xxxx::1\u0026#34; conf_gateway=\u0026#34;6.7.8.9\u0026#34; conf_nameserveripv4one=\u0026#34;196.22.142.222\u0026#34; conf_nameserveripv4two=\u0026#34;41.203.18.183\u0026#34; conf_nameserveripv6one=\u0026#34;2606:4700:4700::1111\u0026#34; conf_nameserveripv6two=\u0026#34;2606:4700:4700::1001\u0026#34; conf_username=\u0026#34;myusername\u0026#34; conf_pubkeyurl=\u0026#34;http://url.host/keys.txt\u0026#34; conf_disks=\u0026#34;ada0 ada1\u0026#34; conf_disktype=\u0026#34;mirror\u0026#34; run_installer=\u0026#34;1\u0026#34; Run The Installer # ./depenguin_bsdinstall.sh The process will exit automatically when complete.\nDeactive The Rescue System # If the Xneelo system is still active (10mins) you must deactivate it before rebooting. It might still be active because installation is fast.\nIn Other Session: Reboot # Back in the rescue console, reboot with:\nreboot This will exit the rescue console and boot into FreeBSD.\nConnect To Your Server # After a few minutes to boot up, connect to your server via ssh:\nssh YOUR-USER@ip-address sudo su - Check DNS is available and then perform initial system configuration such as:\nfreebsd-update fetch freebsd-update install pkg update -f pkg upgrade pkg install python39 sudo bash rsync jq unzip Your server is ready for provisioning with an ansible script such as A Beginner\u0026rsquo;s Guide to Building a Virtual Datacenter on FreeBSD with Ansible, Pot and More or Run Your Own Mastodon Server on FreeBSD in a Potluck Container.\nOptional: Enable Password On Account For Console Access # Set a password on your user account if you want to login to the console via the Xneelo remote management interface.\nsudo su - passwd \u0026lt;your username\u0026gt; ","date":"February 7, 2024","externalUrl":null,"permalink":"/blog/2024-02-07/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eLets walk through the steps of installing FreeBSD 14.0 on a dedicated server from Xneelo.\u003c/p\u003e\n\u003cp\u003e\u003ca\n  href=\"https://xneelo.co.za\"\n    target=\"_blank\"\n  \u003eXneelo\u003c/a\u003e is a South African hosting provider with dedicated servers and a rescue system, similar to \u003ca\n  href=\"https://www.hetzner.com\"\n    target=\"_blank\"\n  \u003eHetzner\u003c/a\u003e.\u003c/p\u003e","title":"FreeBSD 14.0 Installation on Xneelo Dedicated Server","type":"blog"},{"content":"","date":"February 7, 2024","externalUrl":null,"permalink":"/tags/system-administration/","section":"Tags","summary":"","title":"System Administration","type":"tags"},{"content":"","date":"February 7, 2024","externalUrl":null,"permalink":"/tags/xneelo/","section":"Tags","summary":"","title":"Xneelo","type":"tags"},{"content":" Introduction # Lets say you want to test a fresh batch of pot images on a local cluster of virtualbox hosts, because FreeBSD 13.2 is out and you just upgraded.\nOnly it turns out that virtualbox is no longer working after the upgrade to 13.2.\nRunning dmesg shows the output:\nKLD vboxdrv.ko: depends on kernel - not available or version mismatch linker_load_file: /boot/modules/vboxdrv.ko - unsupported file type Previously this could be solved by removing the packages and re-installing them, but this time it makes no difference.\nSwitching to latest package stream results in the same error as above.\nHowever, there is a solution, and it involves compiling virtualbox-ose from ports yourself, and then setting up new virtualbox machines with optimal settings.\n\u0026ldquo;Why not use Bhyve?\u0026rdquo; - because other projects use vagrant, and vagrant uses virtualbox\nHow To Set Up a Minio Cluster From Potluck, Complete With Nextcloud, Monitoring And Alerting How To Easily Set Up OpenLDAP and Matrix Synapse with Potluck Images. It\u0026rsquo;s not possible to run Bhyve and VirtualBox concurrently.\nBelow is a series of steps to install virtualbox from ports, and then setup FreeBSD-13.2 virtual machines with optimal settings.\nPhase One: Install VirtualBox from Ports # Delete VirtualBox # First, in case you already have virtualbox installed, delete the installed packages:\npkg delete -f virtualbox-ose virtualbox-ose-kmod Download FreeBSD src Tree # Then download the FreeBSD src tree, as we need it to compile virtualbox:\nfetch -o /tmp ftp://ftp.freebsd.org/pub/`uname -s`/releases/`uname -m`/`uname -r | cut -d\u0026#39;-\u0026#39; -f1,2`/src.txz tar -C / -xvf /tmp/src.txz Download Ports src Tree # Next download the Ports src tree:\nportsnap fetch update portsnap extract Compile VirtualBox # Change to the virtualbox-ose directory to setup the config and compile virtualbox.\nThe QT5 and X11 components are not needed for my development host, so they\u0026rsquo;ve been deselected in the config below.\nTo setup config and build from ports tree, do the following:\ncd /usr/ports/emulators/virtualbox-ose make config deselect all QT and X11 options [ ] Native Language support [ ] QT5 [ ] X11 make install This process will take 1-2 hours to complete, and will download and compile various packages in the process.\nDo not proceed until compiling is complete.\nConfigure VirtualBox # Now that virtualbox is compiled and installed, we\u0026rsquo;ll check if aio is used, and if so, add items to sysctl.conf.\nhttps://man.freebsd.org/cgi/man.cgi?query=aio\nThe aio facility provides system calls for asynchronous I/O. Asynchronous I/O operations are not completed synchronously by the calling thread. Instead, the calling thread invokes one system call to request an asynchronous I/O operation. The status of a completed request is retrieved later via a separate system call.\nRun the following to check if AIO is in use:\nkldstat -v | grep aio If there\u0026rsquo;s any result, edit /etc/sysctl.conf and add:\n# VIRTUALBOX aio(4) SETTINGS vfs.aio.max_aio_queue_per_proc=65536 vfs.aio.max_aio_per_proc=8192 vfs.aio.max_aio_queue=65536 vfs.aio.max_buf_aio=8192 Next we need to enable the vboxnet service as follows:\nservice vboxnet enable Add your username, and any other username which needs to run virtualbox, to the vboxusers group:\n(sudo) pw groupmod vboxusers -m \u0026lt;username\u0026gt; Setup a very broad networking file as follows:\nmkdir -p /usr/local/etc/vbox vi /usr/local/etc/vbox/networks.conf And add:\n* 0.0.0.0/0 This step is optional, as it might already be fixed in the package, setup a symlink as follows:\nmkdir -p /etc/vbox ln -s /usr/local/etc/vbox/networks.conf /etc/vbox/networks.conf Make sure the host computer has gateway enabled by editing /etc/rc.conf and adding:\ngateway_enable=\u0026#34;YES\u0026#34; Enable the vboxdrv driver by editing /boot/loader.conf and adding:\nvboxdrv_load=\u0026#34;YES\u0026#34; Finally shutdown to restart your box:\nshutdown -r now Phase Two: Optimised VirtualBox Hosts # For our purposes we want 5 virtual machines running FreeBSD-13.2, which can be combined into a single cluster using a tool such as the Beginner\u0026rsquo;s Guide to Building a Virtual Datacenter on FreeBSD with Ansible, Pot.\nWe\u0026rsquo;ll setup each virtual server individually, because a manual step is required to remove the attached ISO as CDROM.\nIt might also be possible to clone the first machine and simplify the process, however for this guide we are opting for manual repetition.\nPrerequisites # Make directories and download install media. These directories could be ZFS datasets, and also located elsewhere. For our purposes they\u0026rsquo;re in the root.\nmkdir /vms mkdir /iso cd /iso fetch https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/13.2/FreeBSD-13.2-RELEASE-amd64-disc1.iso Make sure you have 5 free IP addresses in your local range, to allocate one to each virtual machine, or configure reserved addresses in a DHCP solution.\nSetup and Install server1 # Protip: the --cpus flag should only be set to 1 CPU for the best results when running multiple virtualbox instances on a single FreeBSD host.\nConfiguring for multiple CPUs doesn\u0026rsquo;t result in the same outcome as with other hypervisors. Instead it slows things down and causes network delays between the individual virtual machines.\nFor our first server we want to configure the basics, attach a disk and ISO image, set a VNC password, and then start the VM:\nVBoxManage createvm --ostype FreeBSD_64 --register --basefolder /vms --name server1 VBoxManage modifyvm server1 --memory 2048 --ioapic on --cpus 1 --chipset ich9 --nic1 bridged --nictype1 virtio --bridgeadapter1 re0 VBoxManage createhd --size 50000 --filename \u0026#34;/vms/server1/server1.vdi\u0026#34; VBoxManage storagectl server1 --name \u0026#34;SATA Controller\u0026#34; --add sata --portcount 4 --bootable on VBoxManage storageattach server1 --storagectl \u0026#34;SATA Controller\u0026#34; --port 1 --type hdd --medium \u0026#34;/vms/server1/server1.vdi\u0026#34; VBoxManage storageattach server1 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --type dvddrive --medium \u0026#34;/iso/FreeBSD-13.2-RELEASE-amd64-disc1.iso\u0026#34; VBoxManage modifyvm server1 --vrdeproperty VNCPassword=password You can now start the virtual machine, listening for VNC on port 5001:\nVBoxHeadless --startvm server1 --vrde on --vrdeproperty TCP/Ports=5001 From another host, using a VNC client, connect to your box on port 5001 with the password \u0026ldquo;password\u0026rdquo;. You should get a regular FreeBSD boot process and install screen.\nYou can now complete the regular FreeBSD install process.\nMake sure to do the following during the installation:\nZFS on root, automatic Set root password When adding a user, make sure to add them to the wheel group when asked to invite to other groups When installation is complete and you get an opportunity to exit to a shell for some last minute changes, you can do the following:\nvi /etc/ssh/sshd_config And set the following parameters:\nPermitRootLogin prohibit-password PasswordAuthentication no Then you can also add your SSH key to the root and user .ssh/authorized_keys provided a copy of the file is in a web-accessible location. For example:\ncd /root mkdir .ssh cd .ssh fetch http://your.host/keys.txt -o authorized_keys chmod 600 authorized_keys cd .. chmod 700 .ssh cd /home/yourusername mkdir .ssh cd .ssh fetch http://your.host/keys.txt -o authorized_keys chmod 600 authorized_keys cd .. chmod 700 .ssh chown -R yourusername:yourusername .ssh This will allow you to login to the system using your SSH keys on first boot.\nYou can now exit the installer and reboot, but before you do, you need to be prepared to unmount the CDROM drive.\nIn a new terminal window to your box, run the following during the step where the VNC window boots into the BIOS\nVBoxManage storageattach server1 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --medium emptydrive If you\u0026rsquo;re too late, just hit 4 during the installer boot screen to reboot and it should boot from disk.\nThe host will reboot and you can test a login within the VNC window, then type\nshutdown -h now This will shutdown the host and the VNC window will prompt to press any key to reboot. Don\u0026rsquo;t do that!\nPress \u0026ldquo;ctrl+c\u0026rdquo; back in your original terminal window where the virtual machine was started. This will end the process and the VNC window should die too.\nStart up the virtual machine again in a headless fashion with\nVBoxManage startvm server1 --type headless You will need to connect via VNC to port 5001 and \u0026ldquo;press any key to reboot\u0026rdquo;\nYour server1 host will now be running in the background at the IP you configured during setup. You can SSH to it and perform some initial package installs.\nssh yourusername@server1 sudo su - pkg install sudo nano rsync jq python39 bash Make sure to run visudo and uncomment the following line:\n%wheel ALL=(ALL:ALL) NOPASSWD: ALL The host is now ready for further setup via automation tools, we can move on to the next server.\nSetup and Install server2 # For the second server we\u0026rsquo;ll do everything the same as the first, except use different file names, IP address, VNC port.\nVBoxManage createvm --ostype FreeBSD_64 --register --basefolder /vms --name server2 VBoxManage modifyvm server2 --memory 2048 --ioapic on --cpus 1 --chipset ich9 --nic1 bridged --nictype1 virtio --bridgeadapter1 re0 VBoxManage createhd --size 50000 --filename \u0026#34;/vms/server2/server2.vdi\u0026#34; VBoxManage storagectl server2 --name \u0026#34;SATA Controller\u0026#34; --add sata --portcount 4 --bootable on VBoxManage storageattach server2 --storagectl \u0026#34;SATA Controller\u0026#34; --port 1 --type hdd --medium \u0026#34;/vms/server2/server2.vdi\u0026#34; VBoxManage storageattach server2 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --type dvddrive --medium \u0026#34;/iso/FreeBSD-13.2-RELEASE-amd64-disc1.iso\u0026#34; VBoxManage modifyvm server2 --vrdeproperty VNCPassword=password You can now start the virtual machine, listening for VNC on port 5002:\nVBoxHeadless --startvm server2 --vrde on --vrdeproperty TCP/Ports=5002 Complete the FreeBSD installation, making sure to invite your user to the wheel group, and completing other steps for server1.\nThen during reboot remove the CDROM in a new terminal window with:\nVBoxManage storageattach server2 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --medium emptydrive Then boot up, login, shutdown and exit the process used to start server2\nStart up server2 in a headless fashion:\nVBoxManage startvm server2 --type headless You will need to connect via VNC to port 5002 and \u0026ldquo;press any key to reboot\u0026rdquo;\nYour server2 host will now be running in the background at the IP you configured during setup. You can SSH to it and perform some initial package installs as above.\nSetup and Install server3 # For the this server we\u0026rsquo;ll do everything the same as before, except use different file names, IP address, VNC port.\nVBoxManage createvm --ostype FreeBSD_64 --register --basefolder /vms --name server3 VBoxManage modifyvm server3 --memory 2048 --ioapic on --cpus 1 --chipset ich9 --nic1 bridged --nictype1 virtio --bridgeadapter1 re0 VBoxManage createhd --size 50000 --filename \u0026#34;/vms/server3/server3.vdi\u0026#34; VBoxManage storagectl server3 --name \u0026#34;SATA Controller\u0026#34; --add sata --portcount 4 --bootable on VBoxManage storageattach server3 --storagectl \u0026#34;SATA Controller\u0026#34; --port 1 --type hdd --medium \u0026#34;/vms/server3/server3.vdi\u0026#34; VBoxManage storageattach server3 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --type dvddrive --medium \u0026#34;/iso/FreeBSD-13.2-RELEASE-amd64-disc1.iso\u0026#34; VBoxManage modifyvm server3 --vrdeproperty VNCPassword=password You can now start the virtual machine, listening for VNC on port 5003:\nVBoxHeadless --startvm server3 --vrde on --vrdeproperty TCP/Ports=5003 Complete the FreeBSD installation, making sure to invite your user to the wheel group, and completing other steps for server1.\nThen during reboot remove the CDROM in a new terminal window with:\nVBoxManage storageattach server3 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --medium emptydrive Then boot up, login, shutdown and exit the process used to start server3\nStart up server3 in a headless fashion:\nVBoxManage startvm server3 --type headless You will need to connect via VNC to port 5003 and \u0026ldquo;press any key to reboot\u0026rdquo;\nYour server3 host will now be running in the background at the IP you configured during setup. You can SSH to it and perform some initial package installs as above.\nSetup and Install server4 # For the this server we\u0026rsquo;ll do everything the same as before, except use different file names, IP address, VNC port.\nVBoxManage createvm --ostype FreeBSD_64 --register --basefolder /vms --name server4 VBoxManage modifyvm server4 --memory 2048 --ioapic on --cpus 1 --chipset ich9 --nic1 bridged --nictype1 virtio --bridgeadapter1 re0 VBoxManage createhd --size 50000 --filename \u0026#34;/vms/server4/server4.vdi\u0026#34; VBoxManage storagectl server4 --name \u0026#34;SATA Controller\u0026#34; --add sata --portcount 4 --bootable on VBoxManage storageattach server4 --storagectl \u0026#34;SATA Controller\u0026#34; --port 1 --type hdd --medium \u0026#34;/vms/server4/server4.vdi\u0026#34; VBoxManage storageattach server4 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --type dvddrive --medium \u0026#34;/iso/FreeBSD-13.2-RELEASE-amd64-disc1.iso\u0026#34; VBoxManage modifyvm server4 --vrdeproperty VNCPassword=password You can now start the virtual machine, listening for VNC on port 5004:\nVBoxHeadless --startvm server4 --vrde on --vrdeproperty TCP/Ports=5004 Complete the FreeBSD installation, making sure to invite your user to the wheel group, and completing other steps for server1.\nThen during reboot remove the CDROM in a new terminal window with:\nVBoxManage storageattach server4 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --medium emptydrive Then boot up, login, shutdown and exit the process used to start server4\nStart up server4 in a headless fashion:\nVBoxManage startvm server4 --type headless You will need to connect via VNC to port 5004 and \u0026ldquo;press any key to reboot\u0026rdquo;\nYour server4 host will now be running in the background at the IP you configured during setup. You can SSH to it and perform some initial package installs as above.\nSetup and Install server5 # For the this server we\u0026rsquo;ll do everything the same as before, except use different file names, IP address, VNC port.\nVBoxManage createvm --ostype FreeBSD_64 --register --basefolder /vms --name server5 VBoxManage modifyvm server5 --memory 2048 --ioapic on --cpus 1 --chipset ich9 --nic1 bridged --nictype1 virtio --bridgeadapter1 re0 VBoxManage createhd --size 50000 --filename \u0026#34;/vms/server5/server5.vdi\u0026#34; VBoxManage storagectl server5 --name \u0026#34;SATA Controller\u0026#34; --add sata --portcount 4 --bootable on VBoxManage storageattach server5 --storagectl \u0026#34;SATA Controller\u0026#34; --port 1 --type hdd --medium \u0026#34;/vms/server5/server5.vdi\u0026#34; VBoxManage storageattach server5 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --type dvddrive --medium \u0026#34;/iso/FreeBSD-13.2-RELEASE-amd64-disc1.iso\u0026#34; VBoxManage modifyvm server5 --vrdeproperty VNCPassword=password You can now start the virtual machine, listening for VNC on port 5005:\nVBoxHeadless --startvm server5 --vrde on --vrdeproperty TCP/Ports=5005 Complete the FreeBSD installation, making sure to invite your user to the wheel group, and completing other steps for server1.\nThen during reboot remove the CDROM in a new terminal window with:\nVBoxManage storageattach server5 --storagectl \u0026#34;SATA Controller\u0026#34; --port 2 --medium emptydrive Then boot up, login, shutdown and exit the process used to start server5\nStart up server5 in a headless fashion:\nVBoxManage startvm server5 --type headless You will need to connect via VNC to port 5005 and \u0026ldquo;press any key to reboot\u0026rdquo;\nYour server5 host will now be running in the background at the IP you configured during setup. You can SSH to it and perform some initial package installs as above.\nComplete # You now have 5 manually installed virtualbox hosts running FreeBSD-13.2. They are optimised for 1 CPU yet run with the full power of the host.\nIf you\u0026rsquo;re going to use ansible to configure them for something, make sure sudo and python39 are installed. And make sure visudo allows users of the wheel group to perform commands.\nWe can recommend trying out the Beginner\u0026rsquo;s Guide to Building a Virtual Datacenter on FreeBSD with Ansible, Pot.\n","date":"August 25, 2023","externalUrl":null,"permalink":"/blog/2023-08-25/","section":"Tech Blog","summary":"\u003ch1 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h1\u003e\n\u003cp\u003eLets say you want to test a fresh batch of \u003ccode\u003epot\u003c/code\u003e images on a local cluster of \u003ccode\u003evirtualbox\u003c/code\u003e hosts, because FreeBSD 13.2 is out and you just upgraded.\u003c/p\u003e","title":"Optimal VirtualBox on FreeBSD 13.2","type":"blog"},{"content":"","date":"August 25, 2023","externalUrl":null,"permalink":"/tags/ports/","section":"Tags","summary":"","title":"Ports","type":"tags"},{"content":"","date":"August 25, 2023","externalUrl":null,"permalink":"/tags/testing/","section":"Tags","summary":"","title":"Testing","type":"tags"},{"content":"","date":"August 25, 2023","externalUrl":null,"permalink":"/tags/virtualbox/","section":"Tags","summary":"","title":"Virtualbox","type":"tags"},{"content":" Introduction # Back in 2020, a three-part blog series was published on building your own Virtual Datacenter (vDC).\nWhile the detailed configuration instructions are outdated meanwhile (the images offer a lot more options today than back then), you can read it as a refresher.\nThis guide introduces an ansible script to automate the provisioning of a vDC for your applications.\nIn addition to building a vDC with consul and nomad, it offers enhancements over the original blog series, such as the inclusion of monitoring and alerting, wireguard mesh networking, along with openldap and mariadb instances, and nginx nomad jobs.\nThe target audience for this post is comfortable installing FreeBSD on servers or virtual machines and comfortable enough to rename the default network interface, or to find out how to do so.\nYou don\u0026rsquo;t need to know much ansible, you can pick it up from the scripts.\nPart 1: Understanding the Basics # 1.1 FreeBSD And Containers # There are several container or jail management solutions for FreeBSD these days. We like pot.\n1.2 Pot For Container Management # pot is a jail manager, a collection of shell scripts connecting standard features in FreeBSD, such as ZFS and jails, into portable images that can run on other hosts.\n1.3 Potluck Repository # The Potluck repository is a DockerHub-like repository of pot application images that can be downloaded and used without having to build your own.\nMost of the potluck images have consul integration. You\u0026rsquo;ll need a consul image running to make use of them.\nPart 2: The Ansible Pot Foundation Framework # 2.1 What Is Installed? # The script will provision 1, 3 or 5 servers. Where 3 or 5 servers are used, a wireguard mesh network will connect them.\nTwo VLANs will be created off the default network interface, which must be named untrusted before you begin.\nOne VLAN will be for pot images, and the other will be for nomad jobs.\nIn cluster environments with the wireguard mesh, all VLANs can see all other server VLANs over wireguard links. This makes it possible to build consul or nomad clusters over wireguard.\nA pf firewall with basic ruleset will be applied. One host will be a frontend host and accept HTTP requests.\nFor service discovery, a consul server or cluster will be setup, and for job orchestration, a nomad server or cluster is included. A nomad client will be installed on the servers directly.\nA traefik-consul image is included, and also haproxy for dealing with the frontend public website, along with acme.sh for certificate registration.\nTwo nomad jobs for nginx pot jails are included, a public website and an intranet site linking to various services in the vDC.\nAn openldap instance is included, which has a useful web GUI automatically setup. This is the easiest, fully-functional openldap service available.\nA mariadb instance is also included, but has no databases added at this stage.\nFor monitoring and alerting, the Beast-of-Argh pot image is used, which is a single pot collection of prometheus, alertmanager, promtail, loki, grafana and custom dashboards and alerts.\n(These applications are also available individually on the potluck website, albeit for more complex security environments)\n2.2 What Are The Applications Used For? # In the simplest vDC, consul is used for service discovery, while nomad is used for job orchestration. The traefik-consul image helps connect the two.\nThe addition of a monitoring and alerting component uses node_exporter for metrics, and publishes via consul, which is pulled into the The Beast-of-Argh image running prometheus. Additionally syslog-ng sends logs to a loki syslog server.\nThe Beast-of-Argh image is in daily usage on live systems and is instumental for keeping an eye on things. Custom dashboards are included.\nwireguard is used to link servers in a cluster, or clients to the network for administration purposes.\npf is used to handle firewalling.\nopenldap is used for accounts management, and has more functionality with other pot images for mail or chat services.\nmariadb is used as a general purpose database for pot wordpress images, or application databases. You might find postgresql is better suited for your needs and that can be catered for with updates.\nWe hope that this example setup with the basic components is sufficient to accelerate development of your own custom services hosted in the environment.\nPart 3: Setting Up The Virtual Datacenter # 3.1 Ensure The System Meets The Requirements # You network interface must be named untrusted. The ansible script is expecting this name. The reasons stem from practises in live pot environments, where it\u0026rsquo;s deemed a good idea.\nYou can configure this in /etc/rc.conf. Make sure to get the interface name correct, then reboot or restart networking.\nAs an example with a non-public IP, edit /etc/rc.conf for default interface to:\nifconfig_igb0_name=\u0026#34;untrusted\u0026#34; ifconfig_untrusted=\u0026#34;inet 192.168.1.1 netmask 255.255.255.0\u0026#34; You must have ssh key access to the server as a user. This user must be a member of the wheel group and have sudo permissions without a password.\npkg install sudo visudo %wheel ALL=(ALL:ALL) NOPASSWD: ALL You must have decided on the quarterly or latest package stream beforehand. This script will install from the already configured stream. pot images make use of quarterly packages.\n3.2 Clone The git Repository # Clone the git repo:\npkg install git git clone https://codeberg.org/Honeyguide/ansible-pot-foundation.git cd ansible-pot-foundation 3.3 Copy host.sample to hosts File # Inside the correct inventory directory, copy the hosts.sample file to hosts and edit to your needs.\nFor example, a 5 server cluster will use the file in inventory.five:\ncd inventory.five cp hosts.sample hosts vi hosts 3.4 Configure Variables in hosts File # Replace the following values in your inventory hosts file with your data:\n* REPLACE-WITH-USERNAME (username for your server) * REPLACE-WITH-IP-HOSTNAME (IP or hostname of your server) * REPLACE-WITH-EMAIL-ADDRESS (your email address) * REPLACE-WITH-DOMAIN-NAME (your domain name) * REPLACE-FRONTEND-IP (the IP address of your frontend host) * REPLACE-GRAFANA-USER (username for grafana, e.g. admin) * REPLACE-GRAFANA-PASSWORD (password for grafana user) * REPLACE-WITH-LDAP-PASSWORD (ldap master password) * REPLACE-DEFAULT-LDAP-USER (default openldap user) * REPLACE-DEFAULT-LDAP-USER-PASSWORD (openldap user password) * REPLACE-WITH-MYSQL-ROOT-PASSWORD (mariadb root password) * REPLACE-WITH-PROM-SCRAPE-PASSWORD (password for prometheus stat scraping) Some of these appear multiple times, for example REPLACE-WITH-EMAIL-ADDRESS, and may differ in value for different parts of the system. You might want a one email address associated with acme.sh registration and another to get system notices from alertmanager.\n3.5 Run The site.yml Play # Install a full system by running:\nansible-playbook -i inventory.five/hosts site.yml Make sure to specify the correct inventory. Single-server setups will use inventory.single, while cluster setups will use inventory.three or inventory.five.\nThe whole setup can take an hour to complete for new systems, and some errors are expected (and ignored) because they trigger tasks, such a missing keys which need to be created.\n3.6 If Problems Occur, Clean Environment And Try Again # You can remove the clone images, persistent data, and wireguard setup with:\nansible-playbook -i inventory.five/hosts clean.yml Part 4: Exploring The Demonstration vDC # 4.1 This Is Great, Now What? # Command Line Activities # You can test some of the functionality in the base system. ssh to a server and try running the following commands as root:\n# consul members root@server1:~/bin # consul members Node Address Status Type Build Protocol DC Partition Segment server1_consul_server 10.1.0.10:8301 alive server 1.12.4 2 dc1 default \u0026lt;all\u0026gt; server2_consul_server 10.2.0.10:8301 alive server 1.12.4 2 dc1 default \u0026lt;all\u0026gt; server3_consul_server 10.3.0.10:8301 alive server 1.12.4 2 dc1 default \u0026lt;all\u0026gt; server4_consul_server 10.4.0.10:8301 alive server 1.12.4 2 dc1 default \u0026lt;all\u0026gt; server5_consul_server 10.5.0.10:8301 alive server 1.12.4 2 dc1 default \u0026lt;all\u0026gt; server1_beast_server 10.1.0.100:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server1_consul_client 10.1.0.1:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server1_mariadb_server 10.1.0.13:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server1_nomad_server 10.1.0.11:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server1_openldap1_server 10.1.0.14:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server1_traefikconsul_server 10.1.0.12:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server2_consul_client 10.2.0.1:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server2_nomad_server 10.2.0.11:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server3_consul_client 10.3.0.1:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server3_nomad_server 10.3.0.11:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server4_consul_client 10.4.0.1:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server4_nomad_server 10.4.0.11:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server5_consul_client 10.5.0.1:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; server5_nomad_server 10.5.0.11:8301 alive client 1.12.4 2 dc1 default \u0026lt;default\u0026gt; Or check the nomad tools in the /root/bin directory:\n# bin/nomad-raft-status.sh root@server1:~ # bin/nomad-raft-status.sh Node ID Address State Voter RaftProtocol nomad-server-clone.server4.global bf7c92a3-966b-e245-5ca2-2f544365b008 10.4.0.11:4647 leader true 3 nomad-server-clone.server2.global 13b42085-af1c-3e82-6210-2361556fe3cb 10.2.0.11:4647 follower true 3 nomad-server-clone.server5.global 4b336424-fe5b-ba84-6bc9-eb67dae5d2c6 10.5.0.11:4647 follower true 3 nomad-server-clone.server3.global ca64ecf5-a7b0-1910-8769-681835ad6dc1 10.3.0.11:4647 follower true 3 nomad-server-clone.server1.global 2f13c1c4-43a0-1464-1b78-3cccdb35e616 10.1.0.11:4647 follower true 3 And check on nomad jobs:\n# bin/nomad-job-status.sh root@server1:~ # bin/nomad-job-status.sh ID Type Priority Status Submit Date publicwebsite service 50 running 2023-05-04T22:53:16+02:00 stdwebsite service 50 running 2023-05-04T22:53:16+02:00 Graphical Interfaces # Wireguard Access # To access the intranet site, you need to connect to the wireguard network as a roadwarrior client.\nTwo wireguard roadwarrior clients are configured on the primary server. Using the keys created you can add a wireguard link to the server.\nroot@server1:/usr/local/etc/wireguard # ls -al total 26 drwx------ 2 root wheel 11 May 4 22:48 . drwxr-xr-x 24 root wheel 42 May 1 21:27 .. -rw------- 1 root wheel 45 May 4 22:48 admin_one_preshared.key -rw------- 1 root wheel 45 May 4 22:48 admin_one_private.key -rw------- 1 root wheel 45 May 4 22:48 admin_one_public.key -rw------- 1 root wheel 45 May 4 22:48 admin_two_preshared.key -rw------- 1 root wheel 45 May 4 22:48 admin_two_private.key -rw------- 1 root wheel 45 May 4 22:48 admin_two_public.key -rw------- 1 root wheel 45 May 4 22:48 server_private.key -rw------- 1 root wheel 45 May 4 22:48 server_public.key -rw------- 1 root wheel 1283 May 4 22:48 wg0.conf Get the contents of admin_one_preshared.key, admin_one_private.key, admin_one_public.key and server_public.key and configure a wireguard connection on your client host:\n[Interface] PrivateKey = { contents admin_one_private.key } Address = 10.254.1.2/32 ListenPort = 51820 DNS = 1.1.1.1 MTU = 1360 [Peer] PublicKey = { contents server_public.key } PresharedKey = { contents admin_one_preshared.key } Endpoint = { server1 public IP }:51820 AllowedIPs = 10.0.0.0/8 PersistentKeepalive = 25 Then connect wireguard to join the environment.\nDashboards And More # You should now be able to access the intranet page at http://10.101.0.1:25080/, with links to the following services on different IPs within the environment.\nAlertmanager Grafana Prometheus Consul UI Nomad UI Traefik-Consul Dashboard 4.2 Build Your Own Applications! # You can now build our own applications, to be run via nomad job, or as additional pot images. Monitoring and alerting is catered for, along with authentication via opeldap, or database in mariadb.\nPerhaps you\u0026rsquo;d like to add a mail server pot image, or a run one or many wordpress sites via nomad jobs?\nThe basic foundation is in place to get experimenting, or adapted to your needs.\n4.3 Potential Applications For The vDC # Multiple live environments make use of a similar setup to this environment, using pot images on FreeBSD.\nThe infrastructure which hosts this site, or the Potluck image repository, all works in the same way, just with more pot image applications and nomad jobs in the mix.\nPart 5: Expanding Knowledge And Resources # 5.1 Useful Links # Pot homepage and github. Potluck site and github Hashicorp Consul Hashicorp Nomad Traefik Wireguard Prometheus and Alertmanager Grafana and Grafana Loki 5.2 Advanced Topics # You can check out the other pot images on the Potluck site and integrate them manually, or with updates to the ansible playbooks.\nYou can also try your hand at building pot images within your environment. Check out the sources on github for ideas.\n5.3 Feedback # We\u0026rsquo;d love to get some feedback, or even better, pull requests on github. Let us know how the setup performs in your environment.\nSummary # The ansible pot foundation vDC is an easy to deploy, functional environment to host your applications. It can help reduce the time taken to setup an environment evaluating FreeBSD with pot containers.\n","date":"June 9, 2023","externalUrl":null,"permalink":"/blog/2023-06-09/","section":"Tech Blog","summary":"\u003ch1 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h1\u003e\n\u003cp\u003eBack in 2020, a three-part blog series was published on building your own Virtual Datacenter (vDC).\u003c/p\u003e","title":"A Beginner's Guide to Building a Virtual Datacenter on FreeBSD with Ansible, Pot and More","type":"blog"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/consul/","section":"Tags","summary":"","title":"Consul","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/grafana/","section":"Tags","summary":"","title":"Grafana","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/haproxy/","section":"Tags","summary":"","title":"Haproxy","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/loki/","section":"Tags","summary":"","title":"Loki","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/mariadb/","section":"Tags","summary":"","title":"Mariadb","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/mesh/","section":"Tags","summary":"","title":"Mesh","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/nginx/","section":"Tags","summary":"","title":"Nginx","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/nomad/","section":"Tags","summary":"","title":"Nomad","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/openldap/","section":"Tags","summary":"","title":"Openldap","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/pot/","section":"Tags","summary":"","title":"Pot","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/prometheus/","section":"Tags","summary":"","title":"Prometheus","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/syslog-ng/","section":"Tags","summary":"","title":"Syslog-Ng","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/traefik/","section":"Tags","summary":"","title":"Traefik","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/vdc/","section":"Tags","summary":"","title":"VDC","type":"tags"},{"content":"","date":"June 9, 2023","externalUrl":null,"permalink":"/tags/wireguard/","section":"Tags","summary":"","title":"Wireguard","type":"tags"},{"content":" Introduction # This is a post about running your own Matrix-Synapse instance using Pot jails.\nYou will see that once the initial (and also not very complicated) configuration is complete, it takes only 6 commands to e.g. run a complete openldap server if you are using the Potluck container images.\nIt\u0026rsquo;s a detailed HOWTO run Consul, OpenLDAP and Matrix-Synapse together using Pot jails.\nHint: If you do not want to go through the complete step by step guide, you can also skip directly to the end of this article and download the Openldap-Matrix-Sampler.\nWhat is Pot? # Pot is \u0026ldquo;another container framework based on jails, to run FreeBSD containers on FreeBSD. Every running instance is called pot , like the one that I use to cook all the different type of pasta. It\u0026rsquo;s heavily based on FreeBSD, in particular on jails, ZFS, pf and rctl.\u0026rdquo;\nPotluck images make use of Pot to create deployable jails. Potluck images themselves are application-specific deployable pot jails (container images). Many require some integration with other pot jails, such as consul.\nWhat is Matrix-Synapse? # \u0026ldquo;Matrix is an open standard for communications on the internet, supporting federation, encryption and VoIP.\u0026rdquo; \u0026ldquo;Synapse is an open-source Matrix homeserver written and maintained by the Matrix.org Foundation.\u0026rdquo;\nYou can find a list of Matrix clients to connect to this or other servers at https://matrix.org/clients/\nWhat is Consul? # \u0026ldquo;HashiCorp Consul is a service networking solution that enables teams to manage secure network connectivity between services and across on-prem and multi-cloud environments and runtimes. Consul offers service discovery, service mesh, traffic management, and automated updates to network infrastructure device.\u0026rdquo;\nBasic FreeBSD Configuration # Prerequisites # We need to setup some basic settings on a FreeBSD 13.1 host dedicated to running pot jails:\nsysrc gateway_enable=\u0026#34;YES\u0026#34; sysctl -w security.jail.allow_raw_sockets=1 sysctl -w net.inet.ip.forwarding=1 echo \u0026#34;security.jail.allow_raw_sockets=1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf echo \u0026#34;net.inet.ip.forwarding=1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf sysrc clear_tmp_enable=\u0026#34;YES\u0026#34; Networking # Next add an IP alias for the internal address range we need.\nAdjust the network interface igb0 and address according to your network below and in the other steps of this howto:\n$ ifconfig igb0 10.20.0.1 netmask 255.255.255.255 alias And add to /etc/rc.conf:\n$ sysrc ifconfig_igb0_alias0=\u0026#34;10.20.0.1 netmask 255.255.255.255\u0026#34; Reboot or restart networking and routing\nCreate ZFS Datasets # The following ZFS datasets are expected to be present:\n$ zfs create -o mountpoint=/mnt/srv zroot/srv $ zfs create -o mountpoint=/mnt/data zroot/data $ zfs create -o mountpoint=/mnt/data/jaildata zroot/data/jaildata $ zfs create -o mountpoint=/mnt/data/jaildata/openldap zroot/data/jaildata/openldap $ zfs create -o mountpoint=/mnt/data/jaildata/matrix zroot/data/jaildata/matrix Package Installation and pot Configuration # Install Packages # pkg$ install -y bash curl sudo python39 go119 gmake rsync tmux jq dmidecode pftop openssl nmap consul pot potnet Setup pot.conf and Enable the Service # $ pot init -v $ service pot enable consul Client Configuration On Host # Configure consul Client, But Don\u0026rsquo;t Start It Yet # $ mkdir -p /usr/local/etc/consul.d $ chmod 0750 /usr/local/etc/consul.d You need to adjust the IP addresses, datacenter name and gossip key (encrypt field) below:\n$ cat \u0026gt;/usr/local/etc/consul.d/agent.json\u0026lt;\u0026lt;EOF { \u0026#34;bind_addr\u0026#34;: \u0026#34;192.168.88.90\u0026#34;, \u0026#34;client_addr\u0026#34;: \u0026#34;127.0.0.1\u0026#34;, \u0026#34;server\u0026#34;: false, \u0026#34;node_name\u0026#34;: bsdtmp, \u0026#34;datacenter\u0026#34;: \u0026#34;mydc\u0026#34;, \u0026#34;log_level\u0026#34;: \u0026#34;WARN\u0026#34;, \u0026#34;data_dir\u0026#34;: \u0026#34;/var/db/consul\u0026#34;, \u0026#34;verify_incoming\u0026#34;: false, \u0026#34;verify_outgoing\u0026#34;: false, \u0026#34;verify_server_hostname\u0026#34;: false, \u0026#34;verify_incoming_rpc\u0026#34;: false, \u0026#34;encrypt\u0026#34;: \u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34;, \u0026#34;enable_syslog\u0026#34;: true, \u0026#34;leave_on_terminate\u0026#34;: true, \u0026#34;start_join\u0026#34;: [ \u0026#34;{{ consul_ip }}\u0026#34; ], \u0026#34;telemetry\u0026#34;: { \u0026#34;prometheus_retention_time\u0026#34;: \u0026#34;24h\u0026#34; } } EOF Set Permissions # $ chmod 0600 /usr/local/etc/consul.d/agent.json $ hown -R consul:wheel /usr/local/etc/consul.d/ Setup Logs and Enable Service # $ mkdir -p /var/log/consul $ touch /var/log/consul/consul.log $ service consul enable Run consul Server As Potluck Image # Download The consul Image # $ pot import -p consul-amd64-13_1 -t 2.2.1 -U https://potluck.honeyguide.net/consul Setup and Start The consul Image # Adjust the gossip key (see consul client installation above), network interface and interface name below:\n$ pot clone \\ -P consul-amd64-13_1_2_2_1 \\ -p consul-clone \\ -N alias -i \u0026#34;igb0|10.20.0.2\u0026#34; $ pot set-env -p consul-clone \\ -E DATACENTER=mydc \\ -E NODENAME=consul \\ -E IP=10.20.0.2 \\ -E BOOTSTRAP=1 \\ -E PEERS=\u0026#34;1.2.3.4\u0026#34; \\ -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; $ pot set-attr -p consul-clone -A start-at-boot -V True $ pot start consul-clone That is all you need to do to run a consul server if you are using Potluck!\nStart The consul client # After the consul pot has started, you can now start the consul client as it now can find the consul server:\n$ service consul start If all went well you should see something like this:\n$ consul members Node Address Status Type Build Protocol DC Partition Segment consul 10.20.0.2:8301 alive server 1.12.4 2 mydc default \u0026lt;all\u0026gt; bsdtmp 192.168.88.90:8301 alive client 1.12.4 2 mydc default \u0026lt;default\u0026gt; Run openldap As Potluck Image # Download The openldap Image # $ pot import -p openldap-amd64-13_1 -t 1.6.17 -U https://potluck.honeyguide.net/openldap Setup and Start The openldap Image # Again adjust the interface name and IP address below:\n$ pot clone \\ -P openldap-amd64-13_1_1_6_17 \\ -p openldap-clone \\ -N alias -i \u0026#34;igb0|10.20.0.3\u0026#34; $ pot mount-in -p openldap-clone -d /mnt/data/jaildata/openldap -m /mnt $ pot set-env -p openldap-clone \\ -E NODENAME=ldap \\ -E DATACENTER=mydc \\ -E IP=10.20.0.3 \\ -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; \\ -E CONSULSERVERS=10.20.0.2 \\ -E DOMAIN=ldap.local \\ -E MYCREDS=\u0026#34;password\u0026#34; \\ -E HOSTNAME=ldap.local \\ -E DEFAULTGROUPS=Y \\ -E USERNAME=matrixuser \\ -E PASSWORD=matrixpass $ pot set-attr -p openldap-clone -A start-at-boot -V True $ pot start openldap-clone Again, these 6 commands are enough to download and run a completely configured openldap server.\nNote: This image is also preconfigured for redundant multi-master setups.\nOptional: Test openldap # You can now connect to the running pot (i.e. jail) and check if openldap is working correctly:\n$ pot term openldap-clone $ ldapsearch -LLL -x -H ldapi:// dn: dc=ldap,dc=local objectClass: organization objectClass: dcObject dc: ldap o: ldap dn: ou=People,dc=ldap,dc=local objectClass: organizationalUnit ou: People dn: ou=group,dc=ldap,dc=local objectClass: organizationalUnit ou: group dn: cn=mail,ou=group,dc=ldap,dc=local objectClass: posixGroup gidNumber: 3000 cn: mail dn: uid=matrixuser,ou=People,dc=ldap,dc=local objectClass: posixAccount objectClass: shadowAccount objectClass: inetOrgPerson cn: matrixuser sn: genericuser uid: matrixuser uidNumber: 5000 gidNumber: 5000 homeDirectory: /home/matrixuser loginShell: /bin/sh userPassword:: *encrypted string* If everything looks good, you can press Ctrl-d to exit the openldap-clone pot.\nRun matrix-synapse As Potluck Image # Download The matrix-synapse Image # $ pot import -p matrix-synapse-amd64-13_1 -t 1.2.3 -U https://potluck.honeyguide.net/matrix-synapse Setup and Start The matrix-synapse Image # For live usage, remove NOSSL parameter and have certificate registered. The ssh key can also be adjusted to the correct path.\nAgain, also adjust IP address and interface name if necessary.\n$ pot clone \\ -P matrix-synapse-amd64-13_1_1_2_3 \\ -p matrix-clone \\ -N alias -i \u0026#34;igb0|10.20.0.4\u0026#34; $ pot mount-in -p matrix-clone -d /mnt/data/jaildata/matrix -m /mnt $ pot copy-in -p matrix-clone -s /root/.ssh/id_rsa.pub -d /root/importauthkey $ pot set-env -p matrix-clone \\ -E DATACENTER=mydc \\ -E CONSULSERVERS=\u0026#34;10.20.0.2\u0026#34; \\ -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; \\ -E NODENAME=matrix \\ -E IP=10.20.0.4 \\ -E DOMAIN=matrix.local \\ -E ALERTEMAIL=\u0026#34;solo@nowhere.net\u0026#34; \\ -E REGISTRATIONENABLE=false \\ -E MYSHAREDSECRET=complicatedpassword \\ -E SMTPHOST=mail.local \\ -E SMTPPORT=25 \\ -E SMTPUSER=\u0026#34;solo@nowhere.net\u0026#34; \\ -E SMTPPASS=\u0026#34;password\u0026#34; \\ -E SMTPFROM=\u0026#34;matrix@ldap.local\u0026#34; \\ -E SSLEMAIL=none \\ -E LDAPSERVER=10.20.0.3 \\ -E LDAPPASSWORD=password \\ -E LDAPDOMAIN=ldap.local \\ -E CONTROLUSER=true \\ [ -E NOSSL=true ] $ pot set-attr -p matrix-clone -A start-at-boot -V True $ pot start matrix-clone Again, just 6 commands are needed to run a matrix-synapse server connected to the openldap server from the previous chapter.\nConnect With a Matrix Client # You can find a suitable Matrix client at https://matrix.org/clients/\nConnect with your server\u0026rsquo;s URL and the matrix user and password you created, e.g. for testing purposes the matrixuser account created by configuring the openldap image above or by using the LDAP Account Manager below.\nUsing LDAP Account Manager or LAM # LAM is a web front-end to openldap which is listening on the IP address you have assigned to the openldap pot above.\nThe openldap image already includes a pre-configured setup, so you can directly log in with user Manager and the password you set by configuring the openldap image.\nYou can use this interface to add new accounts or edit existing ones:\nThe following screens show how you can easily use basic functionality.\nYou can flip between Users and Groups by clicking Accounts and hovering:\nThe defaul page on login is the user list:\nEdit a user to see more details and adjust fields:\nUNIX details can be edited too:\nGroups have mail as a default group:\nServer information is available via the tools menu:\nAlternatively: Try the Openldap-Matrix-Sampler Environment # There is a sampler you can try at openldap-matrix-sampler which includes a multi-master openldap setup on a single host for demonstration purposes.\nIt closely matches the setup above, but with two openldap servers.\nNormally each openldap pot jail would run on a separate host for redundancy reasons.\n","date":"March 8, 2023","externalUrl":null,"permalink":"/blog/2023-03-08/","section":"Tech Blog","summary":"\u003ch1 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h1\u003e\n\u003cp\u003eThis is a post about running your own Matrix-Synapse instance using Pot jails.\u003c/p\u003e\n\u003cp\u003eYou will see that once the initial (and also not very complicated) configuration is complete, it takes only 6 commands to e.g. run a complete \u003ccode\u003eopenldap\u003c/code\u003e server if you are using the Potluck container images.\u003c/p\u003e","title":"How To Easily Set Up OpenLDAP and Matrix Synapse with Potluck Images","type":"blog"},{"content":"","date":"March 8, 2023","externalUrl":null,"permalink":"/tags/matrix/","section":"Tags","summary":"","title":"Matrix","type":"tags"},{"content":"","date":"March 8, 2023","externalUrl":null,"permalink":"/tags/matrix-synapse/","section":"Tags","summary":"","title":"Matrix-Synapse","type":"tags"},{"content":" We Are Not Digital Natives, But We Offer The Experience of Digital Midwives # On a strategic level, this involves providing advisory services and enterprise architecture concepts to ensure our clients have access to the best solutions available to fulfil their missions.\nThis involves being well-versed in the latest, state-of-the-art technologies, as well as understanding that the latest tech hype cycle is usually more about the tech companies reaching their business goals than ensuring they add value to their clients\u0026rsquo; businesses.\nOur aim is to cut through the hype and help you choose and implement the right digital tools to empower you to reach your mission targets.\nAdditionally, we never forget that technology and architecture are just one building block. Equally important are cultural and organisational change management and implementation. While we are nerds, we are socially compatible nerds, and we will work with you every step of the way.\nBest Fit and Digital Sovereignty, Not Path of Least Resistance # If you are looking for an average consultancy that designs and implements average, expensive, run-of-the-mill solutions offered by big tech companies, we are not who you are looking for.\nMiddle management sometimes thinks it can do no wrong by implementing software and solutions on a well-trodden path. This is not necessarily because they will be successful, but because they will not be blamed if their project fails — after all, they are only doing what everyone else is doing.\nYou know though that the success and even survival of your company depends more than ever on implementing and using technology that really supports your value streams and takes data security and digital sovereignty into account.\nWe are able to work out what it takes to meet these requirements.\nWe offer results-driven, efficient, state-of-the-art, bullshit-free advice and solutions.\nWe don\u0026rsquo;t just talk about open source; we eat, sleep and breathe it.\nWhile our main focus is on strategic consultancy and architectural concepts, we can also develop software solutions when it makes sense: We always make sure we understand the ins and outs of what we are talking about.\nWalk the Talk: Dogfooding # We ensure that our approach and technological solutions are tried and tested.\nWe participate in the open-source community by providing free services, for example:\nHosting for the mastodon.africa Project Official GhostBSD Mirror for South Africa Potluck FreeBSD Container (Jail) Repository Honeyguide Open Source Codeberg Git Repository We also use the software and architecture building blocks we propose to our clients to manage our in-house business.\n","date":"December 23, 2022","externalUrl":null,"permalink":"/digital/","section":"","summary":"\u003ch2 class=\"relative group\"\u003eWe Are Not Digital Natives, But We Offer The Experience of Digital Midwives \n    \u003cdiv id=\"we-are-not-digital-natives-but-we-offer-the-experience-of-digital-midwives\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#we-are-not-digital-natives-but-we-offer-the-experience-of-digital-midwives\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eOn a strategic level, this involves providing \u003cstrong\u003eadvisory services and enterprise architecture concepts\u003c/strong\u003e to ensure our clients have access to the best solutions available to fulfil their missions.\u003c/p\u003e","title":"Digital Innovation","type":"page"},{"content":"","date":"December 11, 2022","externalUrl":null,"permalink":"/tags/beast-of-argh/","section":"Tags","summary":"","title":"Beast-of-Argh","type":"tags"},{"content":" Introduction # Let\u0026rsquo;s build a two-server minio storage system, complete with alerting and monitoring.\nThen to illustrate application, we\u0026rsquo;ll include a nextcloud pot image using minio S3 as file storage.\nHint: If you do not want to go through the complete step by step guide, you can also skip directly to the end of this article and download the Minio-Sampler.\nRequirements # You need two servers connected via high speed network, minimum 1GB/s.\nEach server needs the following basic configuration:\nTwo x 2Tb SATA/SSD drives, configured as ZFS mirror for ZROOT Four x 4Tb SATA drives, each becomes a minio disk 64Gb to 128Gb memory Drive Arrays # Configure ZROOT as a ZFS mirror array of two disks, which includes the default zroot \u0026amp; related datasets.\nAn additional two datasets, srv and data, are created on this mirror array.\nsrv is where pot images will be run from, and data is for storing persistent data.\nHint: the pot default is to use /opt for running pots\nWith the 4 extra disks, minio will operate with erasure coding across 8 drives and 2 servers.\nFor the purposes of illustration, the minio hosts below are on a firewalled internal network in the 10.100.1.0 range.\nWhere external access is required, such as to the nextcloud web frontend, a suitable forwarding/proxy rule is required on a firewall host.\nPart 1: Minio Installation # We\u0026rsquo;ll assume FreeBSD-13.1 is already installed, with a ZFS mirror of 2 drives for zroot, with dataset zroot/srv mounted on /mnt/srv, and dataset zroot/data mounted on /mnt/data.\n1. Preparing HOSTS File and DNS # On both servers # Add the IP and hostname for the minio hosts to /etc/hosts:\n10.100.1.3 minio1 10.100.1.4 minio2 Make sure your internal DNS servers are added to /etc/resolv.conf, or make use of 8.8.8.8, or 1.1.1.1, or 9.9.9.9:\nnameserver 1.1.1.1 nameserver 9.9.9.9 2. Configure ssh Access # On Both Servers # As root run ssh-keygen and accept defaults.\nedit /root/.ssh/config and add:\nHost minio1 HostName 10.100.1.3 StrictHostKeyChecking no User %%username%% IdentityFile ~/.ssh/id_rsa Port 22 ServerAliveInterval 20 Host minio1 HostName 10.100.1.4 StrictHostKeyChecking no User %%username%% IdentityFile ~/.ssh/id_rsa Port 22 ServerAliveInterval 20 Add the pubkey from minio1 to /root/.ssh/authorized_keys on minio2\nAdd the pubkey from minio2 to /root/.ssh/authorized_keys on minio1\nConnect from each server to the other, and accept keys.\nRemote root access can be removed shortly after setup, it\u0026rsquo;s needed to scp files from one to the other.\n3. Setup Self-Signed Certificates For Use By minio # On minio1 # Run the following commands:\nmkdir -p /usr/local/etc/ssl/CAs cd /usr/local/etc/ssl/CAs openssl genrsa -out rootca.key 8192 openssl req -sha256 -new -x509 -days 3650 -key rootca.key -out rootca.crt -subj \u0026#34;/C=US/ST=None/L=City/O=Organisation/CN=minio1\u0026#34; rsync -avz rootca.key root@minio2:/usr/local/etc/ssl/CAs/ rsync -avz rootca.crt root@minio2:/usr/local/etc/ssl/CAs/ Edit /usr/local/etc/ssl/openssl.conf:\nbasicConstraints = CA:FALSE nsCertType = server nsComment = \u0026#34;OpenSSL Generated Server Certificate\u0026#34; subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth subjectAltName = @alt_names [alt_names] IP.1 = 10.100.1.3 DNS.1 = minio1 Run the following commands:\ncd /usr/local/etc/ssl openssl genrsa -out private.key 4096 chown minio:minio /usr/local/etc/ssl/private.key chmod 0644 /usr/local/etc/ssl/private.key openssl req -new -key private.key -out public.crt -subj \u0026#34;/C=US/ST=None/L=City/O=Organisation/CN=minio1\u0026#34; openssl x509 -req -in public.crt -CA CAs/rootca.crt -CAkey CAs/rootca.key -CAcreateserial -out public.crt -days 3650 -sha256 -extfile openssl.conf chown minio:minio /usr/local/etc/ssl/public.crt chmod 0644 /usr/local/etc/ssl/public.crt Create certificate bundle for nginx:\ncat public.crt CAs/rootca.crt \u0026gt;\u0026gt; bundle.pem On minio2 # Create /usr/local/etc/ssl/openssl.conf:\nbasicConstraints = CA:FALSE nsCertType = server nsComment = \u0026#34;OpenSSL Generated Server Certificate\u0026#34; subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth subjectAltName = @alt_names [alt_names] IP.1 = 10.100.1.4 DNS.1 = minio2 Run the following commands:\nopenssl genrsa -out private.key 4096 chown minio:minio /usr/local/etc/ssl/private.key chmod 0644 /usr/local/etc/ssl/private.key openssl req -new -key private.key -out public.crt -subj /C=US/ST=None/L=City/O=Organisation/CN=minio2 openssl x509 -req -in public.crt -CA CAs/rootca.crt -CAkey CAs/rootca.key -CAcreateserial -out public.crt -days 3650 -sha256 -extfile openssl.conf chown minio:minio /usr/local/etc/ssl/public.crt chmod 0644 /usr/local/etc/ssl/public.crt Create certificate bundle for nginx:\ncat public.crt CAs/rootca.crt \u0026gt;\u0026gt; bundle.pem 4. Preparing The SATA Drives # On Both Servers # mkdir -p /mnt/minio zpool create -m /mnt/minio/disk1 minio-disk1 ada1p1 gpart create -s GPT ada2 gpart add -t freebsd-zfs -l minio-disk2 ada2 zpool create -m /mnt/minio/disk2 minio-disk2 ada2p1 gpart create -s GPT ada3 gpart add -t freebsd-zfs -l minio-disk3 ada3 zpool create -m /mnt/minio/disk3 minio-disk3 ada3p1 gpart create -s GPT ada4 gpart add -t freebsd-zfs -l minio-disk4 ada4 zpool create -m /mnt/minio/disk4 minio-disk4 ada4p1 5. Installation and Configuration of minio # On Both Servers # pkg install -y minio service minio enable sysrc minio_certs=\u0026#34;/usr/local/etc/ssl\u0026#34; sysrc minio_env=\u0026#34;MINIO_ACCESS_KEY=access12345 MINIO_SECRET_KEY=password1234567890\u0026#34; sysrc minio_disks=\u0026#34;https://minio{1...2}:9000/mnt/minio/disk{1...4}\u0026#34; service minio start Access minio at https://10.100.1.3:9000 or https://10.100.1.4:9000 and login with the same credentials used during setup.\nYou will need to accept the self-signed certificate!\nPart 2 - Monitoring # In order to run the beast-of-argh monitoring solution, or the nextcloud nomad image, we need to setup several pot images on minio1 host, specifically:\nConsul server Nomad server Mariadb server Then we can use the beast-of-argh image which utilises consul, node_exporter, syslog-ng.\nWe can also use nomad to schedule a nextcloud instance after setting up a bucket in minio.\n6. Installation and Configuration of ZFS Datasets For This Project # Reminder: zroot/srv mounted on /mnt/srv; and zroot/data mounted on /mnt/data\nOn minio1 # zfs create -o mountpoint=/mnt/srv zroot/srv zfs create -o mountpoint=/mnt/srv/pot zroot/srv/pot zfs create -o mountpoint=/mnt/data zroot/data zfs create -o mountpoint=/mnt/data/jaildata zroot/data/jaildata zfs create -o mountpoint=/mnt/data/jaildata/traefik zroot/data/jaildata/traefik zfs create -o mountpoint=/mnt/data/jaildata/beast zroot/data/jaildata/beast zfs create -o mountpoint=/mnt/data/jaildata/mariadb zroot/data/jaildata/mariadb mkdir -p /mnt/data/jaildata/mariadb/var_db_mysql 7. Install Client-Side consul and nomad With Configuration # On Both Servers # Install consul, nomad, node_exporter packages:\npkg install -y consul nomad node_exporter Create necessary directories:\nmkdir -p /usr/local/etc/consul.d chown consul:wheel /usr/local/etc/consul.d chmod 750 /usr/local/etc/consul.d Generate a gossip key with consul-keygen and save for use:\nconsul-keygen BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA= Create a user for node_exporter and configure node_exporter for later use:\npw useradd -n nodeexport -c \u0026#39;nodeexporter user\u0026#39; -m -s /usr/bin/nologin -h - service node_exporter enable sysrc node_exporter_args=\u0026#34;--log.level=warn\u0026#34; sysrc node_exporter_user=nodeexport sysrc node_exporter_group=nodeexport service node_exporter restart On minio1: # Configure /usr/local/etc/consul.d/agent.json for consul, using the gossip key created above:\n{ \u0026#34;bind_addr\u0026#34;: \u0026#34;10.100.1.3\u0026#34;, \u0026#34;server\u0026#34;: false, \u0026#34;node_name\u0026#34;: \u0026#34;minio1\u0026#34;, \u0026#34;datacenter\u0026#34;: \u0026#34;myminiotest\u0026#34;, \u0026#34;log_level\u0026#34;: \u0026#34;WARN\u0026#34;, \u0026#34;data_dir\u0026#34;: \u0026#34;/var/db/consul\u0026#34;, \u0026#34;verify_incoming\u0026#34;: false, \u0026#34;verify_outgoing\u0026#34;: false, \u0026#34;verify_server_hostname\u0026#34;: false, \u0026#34;verify_incoming_rpc\u0026#34;: false, \u0026#34;encrypt\u0026#34;: \u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34;, \u0026#34;enable_syslog\u0026#34;: true, \u0026#34;leave_on_terminate\u0026#34;: true, \u0026#34;start_join\u0026#34;: [ \u0026#34;{{ consul_ip }}\u0026#34; ], \u0026#34;telemetry\u0026#34;: { \u0026#34;prometheus_retention_time\u0026#34;: \u0026#34;24h\u0026#34; }, \u0026#34;service\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;node-exporter\u0026#34;, \u0026#34;tags\u0026#34;: [\u0026#34;_app=host-server\u0026#34;, \u0026#34;_service=node-exporter\u0026#34;, \u0026#34;_hostname=minio1\u0026#34;, \u0026#34;_datacenter=myminiotest\u0026#34;], \u0026#34;port\u0026#34;: 9100 } } Check permissions:\nchown -R consul:wheel /usr/local/etc/consul.d/ Don\u0026rsquo;t start consul service yet.\nConfigure /usr/local/etc/nomad/client.hcl for nomad:\nbind_addr = \u0026#34;10.100.1.3\u0026#34; datacenter = \u0026#34;myminiotest\u0026#34; advertise { # This should be the IP of THIS MACHINE and must be routable by every node # in your cluster http = \u0026#34;10.100.1.3\u0026#34; rpc = \u0026#34;10.100.1.3\u0026#34; } client { enabled = true options { \u0026#34;driver.raw_exec.enable\u0026#34; = \u0026#34;1\u0026#34; } servers = [\u0026#34;10.100.1.11\u0026#34;] } plugin_dir = \u0026#34;/usr/local/libexec/nomad/plugins\u0026#34; consul { address = \u0026#34;127.0.0.1:8500\u0026#34; client_service_name = \u0026#34;minio1\u0026#34; auto_advertise = true client_auto_join = true } tls { http = false rpc = false verify_server_hostname = false verify_https_client = false } telemetry { collection_interval = \u0026#34;15s\u0026#34; publish_allocation_metrics = true publish_node_metrics = true prometheus_metrics = true disable_hostname = true } enable_syslog=true log_level=\u0026#34;WARN\u0026#34; syslog_facility=\u0026#34;LOCAL1\u0026#34; Remove unnecessary files and set permissions:\nrm -r /usr/local/etc/nomad/server.hcl chown nomad:wheel /usr/local/etc/nomad/client.hcl chmod 644 /usr/local/etc/nomad/client.hcl chown root:wheel /var/tmp/nomad chmod 700 /var/tmp/nomad mkdir -p /var/log/nomad touch /var/log/nomad/nomad.log Finally configure nomad sysrc entries:\nsysrc nomad_user=\u0026#34;root\u0026#34; sysrc nomad_group=\u0026#34;wheel\u0026#34; sysrc nomad_env=\u0026#34;PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin\u0026#34; sysrc nomad_args=\u0026#34;-config=/usr/local/etc/nomad/client.hcl\u0026#34; sysrc nomad_debug=\u0026#34;YES\u0026#34; Don\u0026rsquo;t start nomad service yet.\nOn minio2 # Configure /usr/local/etc/consul.d/agent.json for consul, using the gossip key created above:\n{ \u0026#34;bind_addr\u0026#34;: \u0026#34;10.100.1.4\u0026#34;, \u0026#34;server\u0026#34;: false, \u0026#34;node_name\u0026#34;: \u0026#34;minio2\u0026#34;, \u0026#34;datacenter\u0026#34;: \u0026#34;myminiotest\u0026#34;, \u0026#34;log_level\u0026#34;: \u0026#34;WARN\u0026#34;, \u0026#34;data_dir\u0026#34;: \u0026#34;/var/db/consul\u0026#34;, \u0026#34;verify_incoming\u0026#34;: false, \u0026#34;verify_outgoing\u0026#34;: false, \u0026#34;verify_server_hostname\u0026#34;: false, \u0026#34;verify_incoming_rpc\u0026#34;: false, \u0026#34;encrypt\u0026#34;: \u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34;, \u0026#34;enable_syslog\u0026#34;: true, \u0026#34;leave_on_terminate\u0026#34;: true, \u0026#34;start_join\u0026#34;: [ \u0026#34;{{ consul_ip }}\u0026#34; ], \u0026#34;telemetry\u0026#34;: { \u0026#34;prometheus_retention_time\u0026#34;: \u0026#34;24h\u0026#34; }, \u0026#34;service\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;node-exporter\u0026#34;, \u0026#34;tags\u0026#34;: [\u0026#34;_app=host-server\u0026#34;, \u0026#34;_service=node-exporter\u0026#34;, \u0026#34;_hostname=minio2\u0026#34;, \u0026#34;_datacenter=myminiotest\u0026#34;], \u0026#34;port\u0026#34;: 9100 } } Check permissions:\nchown -R consul:wheel /usr/local/etc/consul.d/ Don\u0026rsquo;t start consul service yet.\n8. Installation and Configuration of pot # On minio1 # Install pot from packages:\npkg install -y pot potnet Create a pot.conf or copy the pot.conf.sample to pot.conf and edit accordingly:\nvi /usr/local/etc/pot.conf POT_ZFS_ROOT=zroot/srv/pot POT_FS_ROOT=/mnt/srv/pot POT_CACHE=/var/cache/pot POT_TMP=/tmp POT_NETWORK=10.192.0.0/10 POT_NETMASK=255.192.0.0 POT_GATEWAY=10.192.0.1 POT_EXTIF=vtnet1 # em0, igb0, ixl0, vtnet0 etc Initiate pot and enable the service:\npot init -v service pot enable 9. Installation and Configuration of consul Pot Image # Next you will download and run the consul pot image.\nOn minio1: # pot import -p consul-amd64-13_1 -t 2.0.27 -U https://potluck.honeyguide.net/consul pot clone -P consul-amd64-13_1_2_0_27 -p consul-clone -N alias -i \u0026#34;vtnet1|10.100.1.10\u0026#34; pot set-env -p consul-clone -E DATACENTER=myminiotest -E NODENAME=consul -E IP=10.100.1.10 -E BOOTSTRAP=1 -E PEERS=\u0026#34;1.2.3.4\u0026#34; -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; -E REMOTELOG=\u0026#34;10.100.1.99\u0026#34; pot set-attr -p consul-clone -A start-at-boot -V True pot start consul-clone On Both Servers # service consul restart service node_exporter restart 10. Installation and Configuration of nomad Pot Image # Install the nomad pot image.\nOn minio1: # pot import -p nomad-server-amd64-13_1 -t 2.0.21 -U https://potluck.honeyguide.net/nomad-server pot clone -P nomad-server-amd64-13_1_2_0_21 -p nomad-server-clone -N alias -i \u0026#34;vtnet1|10.100.1.11\u0026#34; pot set-env -p nomad-server-clone -E NODENAME=nomad -E DATACENTER=myminiotest -E IP=\u0026#34;10.100.1.11\u0026#34; -E CONSULSERVERS=\u0026#34;10.100.1.10\u0026#34; -E BOOTSTRAP=1 -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; -E REMOTELOG=\u0026#34;10.100.1.99\u0026#34; -E IMPORTJOBS=0 pot set-attr -p nomad-server-clone -A start-at-boot -V True pot start nomad-server-clone On Both Servers # service nomad restart 11. Installation and Configuration of traefik Pot Image # Install the traefik-consul pot image.\nOn minio1 # pot import -p traefik-consul-amd64-13_1 -t 1.3.5 -U https://potluck.honeyguide.net/traefik-consul pot clone -P traefik-consul-amd64-13_1_1_3_5 -p traefik-consul-clone -N alias -i \u0026#34;vtnet1|10.100.1.12\u0026#34; pot set-env -p traefik-consul-clone -E NODENAME=traefik -E DATACENTER=myminiotest -E IP=\u0026#34;10.100.1.12\u0026#34; -E CONSULSERVERS=\u0026#34;10.100.1.10\u0026#34; -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; -E REMOTELOG=\u0026#34;10.100.1.99\u0026#34; pot set-attr -p traefik-consul-clone -A start-at-boot -V True pot mount-in -p traefik-consul-clone -d /mnt/data/jaildata/traefik -m /var/log/traefik pot start traefik-consul-clone 12. Installation and Configuration of mariadb Pot Image # Install the mariadb pot image.\nOn minio1 # pot import -p mariadb-amd64-13_1 -t 2.0.12 -U https://potluck.honeyguide.net/mariadb pot clone -P mariadb-amd64-13_1_2_0_12 -p mariadb-clone -N alias -i \u0026#34;vtnet1|10.100.1.14\u0026#34; pot mount-in -p mariadb-clone -d /mnt/data/jaildata/mariadb/var_db_mysql -m /var/db/mysql pot set-env -p mariadb-clone -E DATACENTER=myminiotest -E NODENAME=mariadb -E IP=\u0026#34;10.100.1.14\u0026#34; -E CONSULSERVERS=\u0026#34;10.100.1.10\u0026#34; -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; -E DBROOTPASS=myrootpass -E DBSCRAPEPASS=myscrapepass -E DUMPSCHEDULE=\u0026#34;5 21 * * *\u0026#34; -E DUMPUSER=\u0026#34;root\u0026#34; -E DUMPFILE=\u0026#34;/var/db/mysql/full_mariadb_backup.sql\u0026#34; -E REMOTELOG=\u0026#34;10.100.1.99\u0026#34; pot set-attr -p mariadb-clone -A start-at-boot -V True pot start mariadb-clone 13. Installation and Configuration of Beast-Of-Argh Pot Image # On minio1 # You can also install the beast-of-argh monitoring solution, which includes grafana, prometheus, alertmanager and the loki log monitoring system. It also acts as a syslog server with syslog-ng.\npot import -p beast-of-argh-amd64-13_1 -t 0.0.29 -U https://potluck.honeyguide.net/beast-of-argh/ pot clone -P beast-of-argh-amd64-13_1_0_0_29 -p beast-clone -N alias -i \u0026#34;vtnet1|10.100.1.99\u0026#34; pot mount-in -p beast-clone -d /mnt/data/jaildata/beast -m /mnt pot set-env -p beast-clone -E DATACENTER=myminiotest -E NODENAME=beast -E IP=\u0026#34;10.100.1.99\u0026#34; -E CONSULSERVERS=\u0026#34;10.100.1.10\u0026#34; -E GOSSIPKEY=\u0026#34;BBtPyNSRI+/iP8RHB514CZ5By3x1jJLu4SqTVzM4gPA=\u0026#34; -E GRAFANAUSER=admin -E GRAFANAPASSWORD=password -E SCRAPECONSUL=\u0026#34;10.100.1.10\u0026#34; -E SCRAPENOMAD=\u0026#34;10.100.1.11\u0026#34; -E TRAEFIKSERVER=\u0026#34;10.100.1.12\u0026#34; -E SMTPHOSTPORT=\u0026#34;127.0.0.1:25\u0026#34; -E SMTPFROM=\u0026#34;your@example.com\u0026#34; -E ALERTADDRESS=\u0026#34;you@example.com\u0026#34; -E SMTPUSER=\u0026#34;username\u0026#34; -E SMTPPASS=\u0026#34;password\u0026#34; -E REMOTELOG=\u0026#34;10.100.1.99\u0026#34; pot set-attribute -p beast-clone -A start-at-boot -V YES pot start beast-clone Once started, add your minio IPs to /mnt/prometheus/targets.d/minio.yml as follows:\npot term beast-clone cd /mnt/prometheus/targets.d/ minio:\nvi minio.yml - targets: - 10.100.1.3:9000 - 10.100.1.4:9000 labels: job: minio mariadb:\nvi mysql.yml - targets: - 10.100.1.15:9104 labels: job: mysql The servers should be polled via consul but if you need to add them manually, you can edit mytargets.yml:\n#- targets: # - 10.0.0.2:9100 # - 10.0.0.3:9100 # labels: # job: jobtype1 Finally reload prometheus with:\nservice reload prometheus 14. Install syslog-ng # You need to replace syslogd with syslog-ng on both servers.\nOn Both Servers # pkg install -y syslog-ng Set up /usr/local/etc/syslog-ng.conf as follows:\n@version: \u0026#34;3.38\u0026#34; @include \u0026#34;scl.conf\u0026#34; # options options { chain_hostnames(off); use_dns (no); dns-cache(no); use_fqdn (no); keep_hostname(no); flush_lines(0); threaded(yes); log-fifo-size(2000); stats_freq(0); time_reopen(120); ts_format(iso); }; # local sources source src { system(); internal(); }; # Add additional log files here #source s_otherlogs { # file(\u0026#34;/var/log/service/service.log\u0026#34;); # file(\u0026#34;/var/log/service2/access.log\u0026#34;); #}; # destinations destination messages { file(\u0026#34;/var/log/messages\u0026#34;); }; destination security { file(\u0026#34;/var/log/security\u0026#34;); }; destination authlog { file(\u0026#34;/var/log/auth.log\u0026#34;); }; destination maillog { file(\u0026#34;/var/log/maillog\u0026#34;); }; destination lpd-errs { file(\u0026#34;/var/log/lpd-errs\u0026#34;); }; destination xferlog { file(\u0026#34;/var/log/xferlog\u0026#34;); }; destination cron { file(\u0026#34;/var/log/cron\u0026#34;); }; destination debuglog { file(\u0026#34;/var/log/debug.log\u0026#34;); }; destination consolelog { file(\u0026#34;/var/log/console.log\u0026#34;); }; destination all { file(\u0026#34;/var/log/all.log\u0026#34;); }; destination newscrit { file(\u0026#34;/var/log/news/news.crit\u0026#34;); }; destination newserr { file(\u0026#34;/var/log/news/news.err\u0026#34;); }; destination newsnotice { file(\u0026#34;/var/log/news/news.notice\u0026#34;); }; destination slip { file(\u0026#34;/var/log/slip.log\u0026#34;); }; destination ppp { file(\u0026#34;/var/log/ppp.log\u0026#34;); }; #destination console { file(\u0026#34;/dev/console\u0026#34;); }; destination allusers { usertty(\u0026#34;*\u0026#34;); }; # pot settings destination loghost { tcp( \u0026#34;10.100.1.99\u0026#34; port(514) disk-buffer( mem-buf-size(134217728) # 128MiB disk-buf-size(2147483648) # 2GiB reliable(yes) dir(\u0026#34;/var/log/syslog-ng-disk-buffer\u0026#34;) ) ); }; # log facility filters filter f_auth { facility(auth); }; filter f_authpriv { facility(authpriv); }; filter f_not_authpriv { not facility(authpriv); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_ftp { facility(ftp); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_security { facility(security); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; # log level filters filter f_emerg { level(emerg); }; filter f_alert { level(alert..emerg); }; filter f_crit { level(crit..emerg); }; filter f_err { level(err..emerg); }; filter f_warning { level(warning..emerg); }; filter f_notice { level(notice..emerg); }; filter f_info { level(info..emerg); }; filter f_debug { level(debug..emerg); }; filter f_is_debug { level(debug); }; # program filters filter f_ppp { program(\u0026#34;ppp\u0026#34;); }; filter f_all { level(debug..emerg) and not (program(\u0026#34;devd\u0026#34;) and level(debug..info) ); }; log { source(src); filter(f_notice); filter(f_not_authpriv); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; log { source(src); filter(f_security); destination(security); }; log { source(src); filter(f_auth); filter(f_info); destination(authlog); }; log { source(src); filter(f_authpriv); filter(f_info); destination(authlog); }; log { source(src); filter(f_mail); filter(f_info); destination(maillog); }; log { source(src); filter(f_lpr); filter(f_info); destination(lpd-errs); }; log { source(src); filter(f_ftp); filter(f_info); destination(xferlog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_is_debug); destination(debuglog); }; log { source(src); filter(f_emerg); destination(allusers); }; log { source(src); filter(f_ppp); destination(ppp); }; log { source(src); filter(f_all); destination(loghost); }; # turn on sending otherlogs too #log { source(s_otherlogs); destination(loghost); }; Then run:\nservice syslogd onestop service syslogd disable service syslog-ng enable sysrc syslog_ng_flags=\u0026#34;-R /tmp/syslog-ng.persist\u0026#34; service syslog-ng restart 15. Login to grafana, prometheus, alertmanager # You can now login to grafana or prometheus or alertmanager via the respective front ends:\ngrafana: http://ip:3000 prometheus: http://ip:9090 alertmanager: http://ip:9093 Part 3: nextcloud with minio storage # 16. Create Minio Bucket For Nextcloud # Setup Bucket Via Web Frontend # Login to the minio interface and create a new bucket called mydata.\nAccept the default options.\nAlternatively Set Up Bucket Via Command Line # On minio1 # You can also setup a bucket via the command line by first setting an alias\nminio-client alias set sampler https://10.100.1.3:9000 sampler samplerpasswordislong --api S3v4 --insecure --config-dir /root/.minio-client/ and then create a bucket:\nminio-client mb --insecure --config-dir /root/.minio-client/ --with-lock sampler/mydata The --insecure flag is required with self-signed certificates\n17. Create mariadb Database For nextcloud # nextcloud requires a database and user to be setup prior to installation.\nOn minio1 # Login to minio1 and setup a database for nextcloud the quick way:\npot term mariadb-clone mysql DROP DATABASE IF EXISTS nextcloud CREATE DATABASE nextcloud CREATE USER \u0026#39;nextcloud\u0026#39;@\u0026#39;10.100.1.%\u0026#39; IDENTIFIED BY \u0026#39;mynextcloud1345swdwfr3t34rw\u0026#39; GRANT ALL PRIVILEGES on nextcloud.* to \u0026#39;nextcloud\u0026#39;@\u0026#39;10.100.1.%\u0026#39; FLUSH PRIVILEGES QUIT 18. Setting a Custom nextcloud Configuration # You will create 3 custom nextcloud config files to copy in with the nextcloud image, specifically:\nobjectstore.config.php.in mysql.config.php.in custom.config.php.in On minio1 # In /root/nomadjobs on minio1, create a custom config.php file called objectstore.config.php.in as follows:\n\u0026lt;?php $CONFIG = array ( \u0026#39;objectstore\u0026#39; =\u0026gt; array ( \u0026#39;class\u0026#39; =\u0026gt; \u0026#39;OC\\\\Files\\\\ObjectStore\\\\S3\u0026#39;, \u0026#39;arguments\u0026#39; =\u0026gt; array ( \u0026#39;bucket\u0026#39; =\u0026gt; \u0026#39;mydata\u0026#39;, // your bucket name \u0026#39;autocreate\u0026#39; =\u0026gt; true, \u0026#39;key\u0026#39; =\u0026gt; \u0026#39;sampler\u0026#39;, // your key \u0026#39;secret\u0026#39; =\u0026gt; \u0026#39;samplerpasswordislong\u0026#39;, // your secret \u0026#39;use_ssl\u0026#39; =\u0026gt; true, \u0026#39;region\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;hostname\u0026#39; =\u0026gt; \u0026#39;10.100.1.3\u0026#39;, \u0026#39;port\u0026#39; =\u0026gt; \u0026#39;9000\u0026#39;, \u0026#39;use_path_style\u0026#39; =\u0026gt; true, ), ), ); Then create mysql.config.php.in:\n\u0026lt;?php $CONFIG = array ( \u0026#39;dbtype\u0026#39; =\u0026gt; \u0026#39;mysql\u0026#39;, \u0026#39;version\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;dbname\u0026#39; =\u0026gt; \u0026#39;nextcloud\u0026#39;, \u0026#39;dbhost\u0026#39; =\u0026gt; \u0026#39;10.100.1.15:3360\u0026#39;, \u0026#39;dbtableprefix\u0026#39; =\u0026gt; \u0026#39;oc_\u0026#39;, \u0026#39;dbuser\u0026#39; =\u0026gt; \u0026#39;nextcloud\u0026#39;, \u0026#39;dbpassword\u0026#39; =\u0026gt; \u0026#39;mynextcloud1345swdwfr3t34rw\u0026#39;, \u0026#39;mysql.utf8mb4\u0026#39; =\u0026gt; true, ); As well as creating custom.config.php.in:\n\u0026lt;?php $CONFIG = array ( \u0026#39;trusted_domains\u0026#39; =\u0026gt; array ( 0 =\u0026gt; \u0026#39;nextcloud.minio1\u0026#39;, 1 =\u0026gt; \u0026#39;10.100.1.3:9000\u0026#39;, ), \u0026#39;datadirectory\u0026#39; =\u0026gt; \u0026#39;/mnt/nextcloud\u0026#39;, \u0026#39;config_is_read_only\u0026#39; =\u0026gt; false, \u0026#39;loglevel\u0026#39; =\u0026gt; 1, \u0026#39;logfile\u0026#39; =\u0026gt; \u0026#39;/mnt/nextcloud/nextcloud.log\u0026#39;, \u0026#39;memcache.local\u0026#39; =\u0026gt; \u0026#39;\\OC\\Memcache\\APCu\u0026#39;, \u0026#39;filelocking.enabled\u0026#39; =\u0026gt; false, \u0026#39;overwrite.cli.url\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;overwritehost\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;overwriteprotocol\u0026#39; =\u0026gt; \u0026#39;https\u0026#39;, \u0026#39;installed\u0026#39; =\u0026gt; false, \u0026#39;mail_from_address\u0026#39; =\u0026gt; \u0026#39;nextcloud\u0026#39;, \u0026#39;mail_smtpmode\u0026#39; =\u0026gt; \u0026#39;smtp\u0026#39;, \u0026#39;mail_smtpauthtype\u0026#39; =\u0026gt; \u0026#39;PLAIN\u0026#39;, \u0026#39;mail_domain\u0026#39; =\u0026gt; \u0026#39;minio1\u0026#39;, \u0026#39;mail_smtphost\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;mail_smtpport\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;mail_smtpauth\u0026#39; =\u0026gt; 1, \u0026#39;maintenance\u0026#39; =\u0026gt; false, \u0026#39;theme\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;twofactor_enforced\u0026#39; =\u0026gt; \u0026#39;false\u0026#39;, \u0026#39;twofactor_enforced_groups\u0026#39; =\u0026gt; array ( ), \u0026#39;twofactor_enforced_excluded_groups\u0026#39; =\u0026gt; array ( 0 =\u0026gt; \u0026#39;no_2fa\u0026#39;, ), \u0026#39;updater.release.channel\u0026#39; =\u0026gt; \u0026#39;stable\u0026#39;, \u0026#39;ldapIgnoreNamingRules\u0026#39; =\u0026gt; false, \u0026#39;ldapProviderFactory\u0026#39; =\u0026gt; \u0026#39;OCA\\\\User_LDAP\\\\LDAPProviderFactory\u0026#39;, \u0026#39;encryption_skip_signature_check\u0026#39; =\u0026gt; true, \u0026#39;encryption.key_storage_migrated\u0026#39; =\u0026gt; false, \u0026#39;allow_local_remote_servers\u0026#39; =\u0026gt; true, \u0026#39;mail_sendmailmode\u0026#39; =\u0026gt; \u0026#39;smtp\u0026#39;, \u0026#39;mail_smtpname\u0026#39; =\u0026gt; \u0026#39;nextcloud@minio1\u0026#39;, \u0026#39;mail_smtppassword\u0026#39; =\u0026gt; \u0026#39;\u0026#39;, \u0026#39;mail_smtpsecure\u0026#39; =\u0026gt; \u0026#39;ssl\u0026#39;, \u0026#39;app.mail.verify-tls-peer\u0026#39; =\u0026gt; false, \u0026#39;app_install_overwrite\u0026#39; =\u0026gt; array ( 0 =\u0026gt; \u0026#39;camerarawpreviews\u0026#39;, 1 =\u0026gt; \u0026#39;keeweb\u0026#39;, 2 =\u0026gt; \u0026#39;calendar\u0026#39;, ), \u0026#39;apps_paths\u0026#39; =\u0026gt; array ( 0 =\u0026gt; array ( \u0026#39;path\u0026#39; =\u0026gt; \u0026#39;/usr/local/www/nextcloud/apps\u0026#39;, \u0026#39;url\u0026#39; =\u0026gt; \u0026#39;/apps\u0026#39;, \u0026#39;writable\u0026#39; =\u0026gt; true, ), 1 =\u0026gt; array ( \u0026#39;path\u0026#39; =\u0026gt; \u0026#39;/usr/local/www/nextcloud/apps-pkg\u0026#39;, \u0026#39;url\u0026#39; =\u0026gt; \u0026#39;/apps-pkg\u0026#39;, \u0026#39;writable\u0026#39; =\u0026gt; false, ), ), ); 19. Add nomad Job for nextcloud in nomad Dashboard # Open the nomad dashboard in your browser. Click \u0026ldquo;run job\u0026rdquo; and paste the following:\njob \u0026#34;nextcloud\u0026#34; { datacenters = [\u0026#34;samplerdc\u0026#34;] type = \u0026#34;service\u0026#34; group \u0026#34;group1\u0026#34; { count = 1 network { port \u0026#34;http\u0026#34; { static = 10443 } } task \u0026#34;nextcloud1\u0026#34; { driver = \u0026#34;pot\u0026#34; service { tags = [\u0026#34;nginx\u0026#34;, \u0026#34;www\u0026#34;, \u0026#34;nextcloud\u0026#34;] name = \u0026#34;nextcloud-server\u0026#34; port = \u0026#34;http\u0026#34; check { type = \u0026#34;tcp\u0026#34; name = \u0026#34;tcp\u0026#34; interval = \u0026#34;60s\u0026#34; timeout = \u0026#34;30s\u0026#34; } } config { image = \u0026#34;https://potluck.honeyguide.net/nextcloud-nginx-nomad\u0026#34; pot = \u0026#34;nextcloud-nginx-nomad-amd64-13_1\u0026#34; tag = \u0026#34;0.64\u0026#34; command = \u0026#34;/usr/local/bin/cook\u0026#34; args = [\u0026#34;-d\u0026#34;,\u0026#34;/mnt/nextcloud\u0026#34;,\u0026#34;-s\u0026#34;,\u0026#34;10.100.1.3:9000\u0026#34;] copy = [ \u0026#34;/root/nomadjobs/objectstore.config.php.in:/root/objectstore.config.php\u0026#34;, \u0026#34;/root/nomadjobs/mysql.config.php.in:/root/mysql.config.php\u0026#34;, \u0026#34;/root/nomadjobs/custom.config.php.in:/root/custom.config.php\u0026#34;, \u0026#34;/path/to/minio/rootca.crt:/root/rootca.crt\u0026#34; ] mount = [ \u0026#34;/mnt/data/jaildata/nextcloud/nextcloud_www:/usr/local/www/nextcloud\u0026#34;, \u0026#34;/mnt/data/jaildata/nextcloud/storage:/mnt/nextcloud\u0026#34; ] port_map = { http = \u0026#34;80\u0026#34; } } resources { cpu = 1000 memory = 2000 } } } } Accept the options and run the job.\nThere will be a short wait before the nextcloud image is live, it can take 10mins.\n20. Configure nextcloud # Once the nextcloud image is live, you can now login at the public fronted (via haproxy/other) and configure, or configure via the command line.\nOn minio1 # pot list *find nextcloud name* pot term $nextcloud-full-name-randomised... su -m www -c \u0026#39;cd/usr/local/www/nextcloud/; php occ maintenance:install \\ --database \u0026#34;mysql\u0026#34; \\ --database-host \u0026#34;10.200.1.15\u0026#34; \\ --database-port \u0026#34;3306\u0026#34; \\ --database-name \u0026#34;nextcloud\u0026#34; \\ --database-user \u0026#34;nextcloud\u0026#34; \\ --database-pass \u0026#34;mynextcloud1345swdwfr3t34rw\u0026#34; \\ --database-table-space \u0026#34;oc_\u0026#34; \\ --admin-user \u0026#34;sampler\u0026#34; \\ --admin-pass \u0026#34;sampler123\u0026#34; \\ --data-dir \u0026#34;/mnt/nextcloud\u0026#34;\u0026#39; Alternatively: Try the Minio-Sampler Environment # If you\u0026rsquo;d like to skip configuring all the services above manually, and simply run something with an example cluster and apps, please give Minio-sampler a try!\nMake sure to review the detailed installation instructions, and install vagrant, virtualbox, and packer for your platform.\nTo get started:\ngit clone https://codeberg.org/Honeyguide/minio-sampler.git cd minio-sampler export PATH=$(pwd)/bin:$PATH (optional: sudo chmod 775 /tmp) edit config.ini (set a free IP on your LAN) and then run:\nminsampler init mysample cd mysample minsampler packbox minsampler startvms This process will take up to 2 hours to finish, at which point you can access a shell:\nvagrant ssh minio1 sudo su - ./preparenextcloud.sh # not working! Or open http://AC.CE.SS.IP in your browser to get the introduction page with links to minio, grafana, prometheus, nextcloud.\nPot jails use 10.200.1.0/24 in the sampler\nIf for some reason the nextcloud nomad job doesn\u0026rsquo;t start, or wasn\u0026rsquo;t added:\nvagrant ssh minio1 sudo su - cat nomadjobs/nextcloud.nomad Copy the text. In the nomad dashboard, by select \u0026ldquo;run job\u0026rdquo; and paste.\n","date":"December 11, 2022","externalUrl":null,"permalink":"/blog/2022-12-11/","section":"Tech Blog","summary":"\u003ch1 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h1\u003e\n\u003cp\u003eLet\u0026rsquo;s build a two-server \u003ccode\u003eminio\u003c/code\u003e storage system, complete with alerting and monitoring.\u003c/p\u003e\n\u003cp\u003eThen to illustrate application, we\u0026rsquo;ll include a \u003ccode\u003enextcloud\u003c/code\u003e pot image using \u003ccode\u003eminio\u003c/code\u003e S3 as file storage.\u003c/p\u003e","title":"How To Set Up a Minio Cluster From Potluck, Complete With Nextcloud, Monitoring And Alerting","type":"blog"},{"content":"","date":"December 11, 2022","externalUrl":null,"permalink":"/tags/monitoring/","section":"Tags","summary":"","title":"Monitoring","type":"tags"},{"content":"","date":"December 11, 2022","externalUrl":null,"permalink":"/tags/nextcloud/","section":"Tags","summary":"","title":"Nextcloud","type":"tags"},{"content":"","date":"May 19, 2021","externalUrl":null,"permalink":"/tags/containerd/","section":"Tags","summary":"","title":"Containerd","type":"tags"},{"content":"","date":"May 19, 2021","externalUrl":null,"permalink":"/tags/oci/","section":"Tags","summary":"","title":"Oci","type":"tags"},{"content":"","date":"May 19, 2021","externalUrl":null,"permalink":"/tags/oss/","section":"Tags","summary":"","title":"Oss","type":"tags"},{"content":"","date":"May 19, 2021","externalUrl":null,"permalink":"/tags/runj/","section":"Tags","summary":"","title":"Runj","type":"tags"},{"content":"This post is a continuation of the small PoC that describes how to manually run a Potluck image directly with runj and describes how Potluck images can be run via containerd.\nThe Potluck container image library wants to provide complex services out of the box, like e.g. a complete postfix secondary mailserver which also is used in this example.\nInstalling containerd # We reuse the VM from the runj PoC, so let\u0026rsquo;s start it and directly connect to the console:\n$ vm start -f runjvm All the following commands are executed inside the VM:\n$ cd /usr/ports/sysutils/containerd \u0026amp;\u0026amp; make install clean It is suggested that you also install a tool like screen or tmux.\nStart containerd # Start tmux or screen and start the daemon:\n$ containerd Then open a new terminal so you can execute the following commands.\nTest Basic Setup # Optionally, you can now easily test the containerd environment according to Samuel Karp\u0026rsquo;s post to check if everything is working:\n$ ctr image pull --snapshotter zfs public.ecr.aws/samuelkarp/freebsd:12.1-RELEASE ... $ ctr run \\ --snapshotter zfs \\ --runtime wtf.sbk.runj.v1 \\ --rm \\ public.ecr.aws/samuelkarp/freebsd:12.1-RELEASE \\ my-container-id \\ sh -c \u0026#39;echo \u0026#34;Hello from the container!\u0026#34;\u0026#39; Convert and Start Postfix From Potluck # Prepare OCI Image # Download the latest Backup MX Postfix Potluck image (which contains a zfs send blob) and create the ZFS filesystem from it:\n$ curl https://potluck.honeyguide.net/postfix-backupmx-nomad/postfix-backupmx-nomad-amd64-13_0_1.0.3.xz \u0026gt; pfix.xz $ xzcat pfix.xz | zfs recv -F zroot/pfix Create a tarball from the filesystem:\n$ cd /zroot/pfix/m \u0026amp;\u0026amp; tar cf /root/postfix.tar . \u0026amp;\u0026amp; cd $ xz postfix.tar Create an OCI image:\n$ runj demo oci-image --input postfix.tar.xz Import the image and check if it is registered:\n$ ctr image import --index-name postfix image.tar ... $ ctr image ls ... Run Image in containerd # Start the image with two parameters that are passed into the image (see image documentation):\n$ ctr run \\ --snapshotter zfs \\ --runtime wtf.sbk.runj.v1 \\ --rm \\ postfix \\ my-postfix \\ /usr/local/bin/cook -d mydomain.tld -h myhostname Now only the network configuration needs to happen to expose the postfix SMTP port to the outside world.\n","date":"May 19, 2021","externalUrl":null,"permalink":"/blog/2021-05-19/","section":"Tech Blog","summary":"\u003cp\u003eThis post is a continuation \u003ca\n  href=\"/blog/2021-05-13/\"\u003eof the small PoC that describes how to manually run a Potluck image directly with runj\u003c/a\u003e and describes how Potluck images can be run via \u003ccode\u003econtainerd\u003c/code\u003e.\u003c/p\u003e","title":"Running Potluck Images with containerd","type":"blog"},{"content":"","date":"May 19, 2021","externalUrl":null,"permalink":"/authors/stephanlichtenauer/","section":"Authors","summary":"","title":"Stephanlichtenauer","type":"authors"},{"content":"A quite new project called runj has been created which offers an interface between FreeBSD Jails and containerd. runj is already in the Ports tree or it can be found at https://github.com/samuelkarp/runj.\nThe Potluck container image library wants to provide complex services out of the box, like e.g. a complete postfix secondary mailserver. Many of the images are prepared to be orchestrated via nomad and nomad-pot-driver (which also is in the Ports tree).\nThis small how-to shows that these blocking nomad jails can also very easily be started via runj though.\nSetting Up a Test VM # We are using bhyve with vm-bhyveto create a FreeBSD 13 VM to provide a test environment. Of course you can also run the examples below directly on your FreeBSD host or in a VM in any other hypervisor.\n$ vm iso https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/FreeBSD-13.0-RELEASE-amd64-disc1.iso $ vm create runjvm Increase the memory size of the VM in runjvm.conf in your VM directory to 2048M before you install FreeBSD in it.\n$ vm install -f runjvm FreeBSD-13.0-RELEASE-amd64-disc1.iso You need to ensure that your VM is set up using the ZFS file system but otherwise you can use the default configuration options.\nInstalling runj # Start the VM and directly connect to the console:\n$ vm start -f runjvm All the following commands are executed inside the VM:\n$ pkg install git ... $ git clone https://git.FreeBSD.org/ports.git /usr/ports $ cd /usr/ports/sysutils/runj \u0026amp;\u0026amp; make install clean Converting a Potluck Image to a runj Jail # We use the the git-nomad v1.0.3 image in this example, which is a jail that is prepared to be run within a nomad environment and therefore contains a /usr/local/bin/cook script (for more details, see the Potluck flavour how-to.)\nAll the following commands are still executed inside the VM.\nDownload Image and Convert Into ZFS Dataset # $ curl https://potluck.honeyguide.net/git-nomad/git-nomad-amd64-13_0_1.0.3.xz \u0026gt; gitfs.xz $ xz -d gitfs.xz $ cat gitfs | zfs recv -F zroot/git Prepare for runj # $ mkdir git \u0026amp;\u0026amp; cd git Create config.json with the content below which essentially points to the git filesystem we just created and tells runj to start the cook script:\n{ \u0026#34;ociVersion\u0026#34;: \u0026#34;1.0.2-runj-dev\u0026#34;, \u0026#34;process\u0026#34;: { \u0026#34;args\u0026#34;: [ \u0026#34;/usr/local/bin/cook\u0026#34; ] }, \u0026#34;root\u0026#34;: { \u0026#34;path\u0026#34;: \u0026#34;/zroot/git/m\u0026#34; } } Note that the pot jails are always located in a .../m subdirectory.\nFinally, create the jail in runj from this configuration:\n$ runj create git /root/git Run in runj # Start the jail with\n$ runj start git If everything works, you should see the cook process with a simple ps ax.\nThis simple example does not yet set any of the parameters exposed by the image and it also does not deal with setting up a network configuration so that services running within the jail can be reached from the outside. Also, re-packaging for deployment via containerd would be an obvious next step.\nNonetheless, it shows that it should not be complicated to use the Potluck image library and the image creation tools and processes it uses in a containerd environment, too.\n","date":"May 13, 2021","externalUrl":null,"permalink":"/blog/2021-05-13/","section":"Tech Blog","summary":"\u003cp\u003eA quite new project called \u003ccode\u003erunj\u003c/code\u003e has been created which offers an interface between FreeBSD Jails and \u003ccode\u003econtainerd\u003c/code\u003e. \u003ccode\u003erunj\u003c/code\u003e is already in the Ports tree or it can be found at \u003ca\n  href=\"https://github.com/samuelkarp/runj\"\n    target=\"_blank\"\n  \u003ehttps://github.com/samuelkarp/runj\u003c/a\u003e.\u003c/p\u003e","title":"Running Potluck Images with runj","type":"blog"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/devops/","section":"Tags","summary":"","title":"Devops","type":"tags"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/series/freebsd-virtual-data-centre-with-potluck/","section":"Series","summary":"","title":"FreeBSD Virtual Data Centre With Potluck","type":"series"},{"content":" Nginx Example Service (via Traefik) # Overview # As written earlier, you can place the nginx job via the nomad dashboard.\nWhen you start the job the first time, the image will according to the job description be automatically downloaded from Potluck to your compute host which - depending on your Internet connection - might take some time.\nThis image will then be cloned by the driver and started on the host. You can define various parameters, e.g. which hosts may be used by nomad for each job.\nSince many services can be run on one host, nomad will assign a port to nginx which will very likely not be the standard port 80.\nThe nginx service will automatically be registered by nomad with consul as soon and as long as it is running.\ntraefik as reverse-proxy in turn will query consul to find out on which host and port nginx is running and provide the HTTP service to the user on a well-known address and port:\nVersion 1: Complete Job With Mounting S3 Bucket # The complete job description which copies your own nginx config file into the jail and mounts an outside web file directory located on the minio storage bucket at /mnt looks like this:\njob \u0026#34;web\u0026#34; { datacenters = [\u0026#34;my-vdc\u0026#34;] type = \u0026#34;service\u0026#34; group \u0026#34;group1\u0026#34; { count = 1 task \u0026#34;www1\u0026#34; { driver = \u0026#34;pot\u0026#34; service { tags = [\u0026#34;nginx\u0026#34;, \u0026#34;www\u0026#34;] name = \u0026#34;my-web\u0026#34; port = \u0026#34;http\u0026#34; check { type = \u0026#34;tcp\u0026#34; name = \u0026#34;tcp\u0026#34; interval = \u0026#34;60s\u0026#34; timeout = \u0026#34;30s\u0026#34; } } config { image = \u0026#34;https://potluck.honeyguide.net/nginx-nomad\u0026#34; pot = \u0026#34;nginx-nomad-amd64-12_1\u0026#34; tag = \u0026#34;1.1.2\u0026#34; command = \u0026#34;nginx\u0026#34; args = [\u0026#34;-g\u0026#34;,\u0026#34;\u0026#39;daemon off;\u0026#39;\u0026#34;] copy = [ \u0026#34;/mnt/s3bucket/web/nginx.conf:/usr/local/etc/nginx/nginx.conf\u0026#34;, ] mount = [ \u0026#34;/mnt/s3bucket/web/www:/mnt\u0026#34; ] port_map = { http = \u0026#34;80\u0026#34; } } resources { cpu = 200 memory = 64 network { mbits = 10 port \u0026#34;http\u0026#34; {} } } } } } The nginx.conf file which you should save in /mnt/s3bucket/web/nginx.conf (the directory referenced in the job description above):\nworker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; access_log /dev/stdout combined; sendfile on; keepalive_timeout 65; server { listen 80; server_name _; location / { root /mnt; index index.html index.htm; } error_page 404 /404.html; error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/local/www/nginx; } } } error_log /dev/stderr; Note that the log is written to stdout and stderr.\nI\u0026rsquo;ll leave the challenge of creating a test index.html file in /mnt/s3bucket/web/www that will then be delivered by nginxup to you\u0026hellip;\nVersion 2: Simple Example Without Any Outside Storage # The job description of the simple example looks nearly like above, but it uses the default nginx config file and site coming with the Potluck image. That means the following parts of the job description above are removed:\n... copy = [ \u0026#34;/mnt/s3bucket/web/nginx.conf:/usr/local/etc/nginx/nginx.conf\u0026#34;, ] mount = [ \u0026#34;/mnt/s3bucket/web/www:/mnt\u0026#34; ] ... So the complete job description that you can copy and paste into the nomad dashboard looks like this:\njob \u0026#34;web\u0026#34; { datacenters = [\u0026#34;my-vdc\u0026#34;] type = \u0026#34;service\u0026#34; group \u0026#34;group1\u0026#34; { count = 1 task \u0026#34;www1\u0026#34; { driver = \u0026#34;pot\u0026#34; service { tags = [\u0026#34;nginx\u0026#34;, \u0026#34;www\u0026#34;] name = \u0026#34;my-web\u0026#34; port = \u0026#34;http\u0026#34; check { type = \u0026#34;tcp\u0026#34; name = \u0026#34;tcp\u0026#34; interval = \u0026#34;60s\u0026#34; timeout = \u0026#34;30s\u0026#34; } } config { image = \u0026#34;https://potluck.honeyguide.net/nginx-nomad\u0026#34; pot = \u0026#34;nginx-nomad-amd64-12_1\u0026#34; tag = \u0026#34;1.1.2\u0026#34; command = \u0026#34;nginx\u0026#34; args = [\u0026#34;-g\u0026#34;,\u0026#34;\u0026#39;daemon off;\u0026#39;\u0026#34;] port_map = { http = \u0026#34;80\u0026#34; } } resources { cpu = 200 memory = 64 network { mbits = 10 port \u0026#34;http\u0026#34; {} } } } } } Accessing The Website # No matter which example you use, as soon as this service is started, you can access it with a host: my-web header (i.e. the name field from the job description) via traefik at http://10.10.10.14:8080/.\nThis is the same way e.g. apache identifies vhosts, so you could add \u0026ldquo;my-web\u0026rdquo; as hostname together with the IP to your /etc/hosts file on your client and then open http://my-web:8080/ in the browser.\nExample /private/etc/hosts on macOS:\n## # Host Database # # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ## 127.0.0.1 localhost 255.255.255.255 broadcasthost ::1 localhost 10.10.10.14 my-web Assuming that you use the simple example with the default web site, the results will look like this:\nAlternatively, instead of editing your hosts file, you can use curl with -H to set the host header:\n$ curl -H \u0026#34;host: my-web\u0026#34; http://10.10.10.14:8080 \u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;Welcome to nginx!\u0026lt;/title\u0026gt; \u0026lt;style\u0026gt; body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } \u0026lt;/style\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;h1\u0026gt;Welcome to nginx!\u0026lt;/h1\u0026gt; \u0026lt;p\u0026gt;If you see this page, the nginx web server is successfully installed and working. Further configuration is required.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;For online documentation and support please refer to \u0026lt;a href=\u0026#34;http://nginx.org/\u0026#34;\u0026gt;nginx.org\u0026lt;/a\u0026gt;.\u0026lt;br/\u0026gt; Commercial support is available at \u0026lt;a href=\u0026#34;http://nginx.com/\u0026#34;\u0026gt;nginx.com\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Thank you for using nginx.\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; Git Example Service (DNS Lookup) # The services being run do not need to be web services accessed via traefik. You can also run any other server which can then be looked up by using the Consul DNS interface.\nTo see how this can work and assuming you have set up the S3 bucket, you can run the Potluck git image by copying and pasting this git job description into nomad:\njob \u0026#34;git\u0026#34; { datacenters = [\u0026#34;my-vdc\u0026#34;] type = \u0026#34;service\u0026#34; group \u0026#34;group1\u0026#34; { count = 1 task \u0026#34;git1\u0026#34; { driver = \u0026#34;pot\u0026#34; service { tags = [\u0026#34;git\u0026#34;] name = \u0026#34;git\u0026#34; port = \u0026#34;ssh\u0026#34; check { type = \u0026#34;tcp\u0026#34; name = \u0026#34;tcp\u0026#34; interval = \u0026#34;60s\u0026#34; timeout = \u0026#34;30s\u0026#34; } } config { image = \u0026#34;https://potluck.honeyguide.net/git-nomad\u0026#34; pot = \u0026#34;git-nomad-amd64-12_1\u0026#34; tag = \u0026#34;1.0\u0026#34; command = \u0026#34;/usr/local/bin/cook\u0026#34; args = [\u0026#34;\u0026#34;] mount = [ \u0026#34;/mnt/s3bucket/git:/var/db/git\u0026#34; ] port_map = { ssh = \u0026#34;22\u0026#34; } } resources { cpu = 200 memory = 64 network { mbits = 10 port \u0026#34;ssh\u0026#34; {} } } } } } If you don\u0026rsquo;t want to mount any outside storage for now, simply remove this section from the job above:\n... mount = [ \u0026#34;/mnt/s3bucket/git:/var/db/git\u0026#34; ] ... This image exposes the user named git that can be accessed via ssh and private/public keys.\nWhen you don\u0026rsquo;t mount outside storage, providing a public key for full access is not straightforward, but you can nonetheless test the ssh connection attempt described below.\nAgain, nomad will choose a host to run the job on, assign an available port for the ssh service and register the service with the tags from the description with consul:\nTo find the ssh service, you can simply use dig and query _\u0026lt;tag\u0026gt;._\u0026lt;service\u0026gt;.service.consul:\n$ dig @10.10.10.12 -p 8600 _git._git.service.consul SRV ; \u0026lt;\u0026lt;\u0026gt;\u0026gt; DiG 9.10.6 \u0026lt;\u0026lt;\u0026gt;\u0026gt; @10.10.10.12 -p 8600 _git._git.service.consul SRV ; (1 server found) ;; global options: +cmd ;; Got answer: ;; -\u0026gt;\u0026gt;HEADER\u0026lt;\u0026lt;- opcode: QUERY, status: NOERROR, id: 41096 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;_git._git.service.consul.\tIN\tSRV ;; ANSWER SECTION: _git._git.service.consul. 0\tIN\tSRV\t1 1 30733 c0a8b226.addr.my-vdc.consul. ;; ADDITIONAL SECTION: c0a8b226.addr.my-vdc.consul. 0\tIN\tA\t10.10.10.11 test.node.my-vdc.consul. 0\tIN\tTXT\t\u0026#34;consul-network-segment=\u0026#34; ;; Query time: 162 msec ;; SERVER: 10.10.10.12#8600(10.10.10.12) ;; WHEN: Fri Jul 31 17:24:09 CEST 2020 ;; MSG SIZE rcvd: 162 You can see in the ANSWER SECTION that the port having been assigned is 30733 and in the ADDITIONAL SECTION that the IP address is 10.10.10.11.\nWith this information, you can test the connection:\n$ ssh git@10.10.10.11 -p 30733 The authenticity of host \u0026#39;[10.10.10.11]:30733 ([10.10.10.11]:30733)\u0026#39; can\u0026#39;t be established. ECDSA key fingerprint is SHA256:g6BNIaiq1JrjIF6yPCYItMEzQsbGBk7raTztmppYuqE. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added \u0026#39;[10.10.10.11]:30733\u0026#39; (ECDSA) to the list of known hosts. Password for git@gitgit1_d306ad66-30f2-d563-f628-f29a77c50377.test: ","date":"August 3, 2020","externalUrl":null,"permalink":"/blog/2020-08-03/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eNginx Example Service (via Traefik) \n    \u003cdiv id=\"nginx-example-service-via-traefik\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#nginx-example-service-via-traefik\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\u003ch3 class=\"relative group\"\u003eOverview \n    \u003cdiv id=\"overview\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#overview\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003eAs written earlier, you can place the \u003ccode\u003enginx\u003c/code\u003e job via the \u003ccode\u003enomad\u003c/code\u003e dashboard.\u003c/p\u003e","title":"FreeBSD Virtual Data Centre with Potluck: DevOps \u0026 Infrastructure as Code - Part III","type":"blog"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/infrastructure-as-code/","section":"Tags","summary":"","title":"Infrastructure as Code","type":"tags"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/microservice/","section":"Tags","summary":"","title":"Microservice","type":"tags"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/object-storage/","section":"Tags","summary":"","title":"Object Storage","type":"tags"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/orchestration/","section":"Tags","summary":"","title":"Orchestration","type":"tags"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"August 3, 2020","externalUrl":null,"permalink":"/tags/virtual-data-centre/","section":"Tags","summary":"","title":"Virtual Data Centre","type":"tags"},{"content":" Step 3 - Nomad, Consul \u0026amp; Traefik Servers # Note 1: If you do not want to use the prebuilt images as shown below for whatever reason, you can easily recreate the jails from the flavour configuration files yourself by following the instructions on the page of each of the images at Potluck.\nInstalling the consul, nomad and traefik servers on host 10.10.10.10 is as simple as cloning the three Potluck images with 3*5 commands.\nThis creates three jails with consul listening on 10.10.10.12, nomad listening on 10.10.10.13 and traefik listening on 10.10.10.14. Please replace em0 below with the network interface of your host.\nNote 2: Potluck uses Let\u0026rsquo;s Encrypt certificates, so if you get certificate errors when running pot import, install the root certificate package first with pkg install ca_root_nss.\n# consul $ pot import -p consul-amd64-12_1 -t 1.0 -U https://potluck.honeyguide.net/consul ... $ pot clone -P consul-amd64-12_1_1_0 -p consul-clone -N alias -i \u0026#34;em0|10.10.10.12\u0026#34; $ pot set-env -p consul-clone -E DATACENTER=my-vdc -E NODENAME=consulserver -E IP=10.10.10.12 $ pot set-attr -p consul-clone -A start-at-boot -V True $ pot start consul-clone # nomad $ pot import -p nomad-server-amd64-12_1 -t 1.0 -U https://potluck.honeyguide.net/nomad-server ... $ pot clone -P nomad-server-amd64-12_1_1_0 -p nomad-server-clone -N alias -i \u0026#34;em0|10.10.10.13\u0026#34; $ pot set-env -p nomad-server-clone -E DATACENTER=my-vdc -E IP=10.10.10.13 -E CONSULSERVER=10.10.10.12 $ pot set-attr -p nomad-server-clone -A start-at-boot -V True $ pot start nomad-server-clone # traefik $ pot import -p traefik-consul-amd64-12_1 -t 1.1 -U https://potluck.honeyguide.net/traefik-consul ... $ pot clone -P traefik-consul-amd64-12_1_1_1 -p traefik-consul-clone -N alias -i \u0026#34;em0|10.10.10.14\u0026#34; $ pot set-env -p traefik-consul-clone -E CONSULSERVER=10.10.10.12 $ pot set-attr -p traefik-consul-clone -A start-at-boot -V True $ pot start traefik-consul-clone The nomad and traefik instances are linked to the consul instance through the CONSULSERVER=10.10.10.12 configuration parameter.\nIn the example above, we have cloned the images coming from Potluck to assign static host addresses to them.\nAside from nomad which needs a routable address not being NATted, you could run consul and traefik also with the public bridge default configuration of the image and port forwarding. For details, see the traefik and consul Potluck pages and note that the consul GIT example below will not be as simple since port forwarding for UDP ports (necessary for DNS) is a little bit more complicated with the current pot version.\nStep 4 - Nomad Compute Nodes (Clients) # Unfortunately, for the compute nodes it is not possible to provide an image or flavour as the setup needs to be done on the (physical or virtual) host and not inside a jail - the compute node needs to be able to use pot to start containers after all.\nThe setup is not complicated though and can easily be automated via e.g. ansible or salt.\nSince only consul clients may speak with the consul server we set up above, each compute node needs to have consul installed as well:\n$ pkg install consul nomad nomad-pot-driver ... $ sysrc nomad_enable=\u0026#34;YES\u0026#34; $ sysrc nomad_user=\u0026#34;root\u0026#34; $ sysrc nomad_env=\u0026#34;PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin\u0026#34; $ sysrc nomad_args=\u0026#34;-config=/usr/local/etc/nomad/client.hcl\u0026#34; $ sysrc consul_enable=\u0026#34;YES\u0026#34; Also, we need a work around for a bug in the /usr/local/etc/rc.d/consul script:\n$ sysrc consul_group=\u0026#34;wheel\u0026#34; Create the directory /usr/local/etc/consul.d:\n$ mkdir -p /usr/local/etc/consul.d \u0026amp;\u0026amp; chmod 750 /usr/local/etc/consul.d \u0026hellip;and there this agent.json file:\n{ \u0026#34;bind_addr\u0026#34;: \u0026#34;10.10.10.11\u0026#34;, \u0026#34;server\u0026#34;: false, \u0026#34;datacenter\u0026#34;: \u0026#34;my-vdc\u0026#34;, \u0026#34;log_level\u0026#34;: \u0026#34;INFO\u0026#34;, \u0026#34;enable_syslog\u0026#34;: true, \u0026#34;leave_on_terminate\u0026#34;: true, \u0026#34;start_join\u0026#34;: [ \u0026#34;10.10.10.12\u0026#34; ] } Also create a log directory and file:\n$ mkdir -p /var/log/consul \u0026amp;\u0026amp; touch /var/log/consul/consul.log Our /usr/local/etc/nomad/client.hcl looks like this:\nbind_addr = \u0026#34;10.10.10.11\u0026#34; plugin_dir = \u0026#34;/usr/local/libexec/nomad/plugins\u0026#34; datacenter = \u0026#34;my-vdc\u0026#34; client { enabled = true options { \u0026#34;driver.raw_exec.enable\u0026#34; = \u0026#34;1\u0026#34; } servers = [\u0026#34;10.10.10.13\u0026#34;] } consul { # The address to the Consul agent. address = \u0026#34;127.0.0.1:8500\u0026#34; # The service name to register the server and client with Consul. client_service_name = \u0026#34;test-compute-node\u0026#34; # Enables automatically registering the services. auto_advertise = true # Enabling the server and client to bootstrap using Consul. client_auto_join = true } enable_syslog=true log_level=\u0026#34;INFO\u0026#34; syslog_facility=\u0026#34;LOCAL1\u0026#34; Again create a log file:\n$ mkdir -p /var/log/nomad \u0026amp;\u0026amp; touch /var/log/nomad/nomad.log \u0026hellip;and start both services:\n$ /usr/local/etc/rc.d/consul start ... $ /usr/local/etc/rc.d/nomad start Access Consul, Nomad \u0026amp; Traefik Dashboards # Consul # Each of the servers has web dashboards.\nThe consul dashboard lists all the services that you have registered, here it already shows the nginx service you will start in step 5:\nNomad # The nomad dashboard is running at http://10.10.10.13:4646:\nYou can place the nomad jobs described later simply via the dashboard at http://10.10.10.13:4646/ui/jobs/run.\nTraefik # The traefik dashboard is running at http://10.10.10.14:9002:\n","date":"August 2, 2020","externalUrl":null,"permalink":"/blog/2020-08-02/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eStep 3 - Nomad, Consul \u0026amp; Traefik Servers \n    \u003cdiv id=\"step-3---nomad-consul--traefik-servers\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#step-3---nomad-consul--traefik-servers\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003cem\u003eNote 1: If you do not want to use the prebuilt images as shown below for whatever reason, you can easily recreate the jails from the flavour configuration files yourself by following the instructions on the page of each of the images at \u003ca\n  href=\"https://potluck.honeyguide.net/\"\n    target=\"_blank\"\n  \u003ePotluck\u003c/a\u003e\u003c/em\u003e.\u003c/p\u003e","title":"FreeBSD Virtual Data Centre with Potluck: DevOps \u0026 Infrastructure as Code - Part II","type":"blog"},{"content":" Introduction # Yes, FreeBSD Lacks Kubernetes - But It Does Not Really Matter\u0026hellip; # One of the main complaints about FreeBSD is the lack of Docker and Kubernetes, which in turn is seen as inability to use FreeBSD as a platform for bleeding edge concepts like micro services and scale-out container orchestration.\nWhile for sure the Linux ecosystem is more diverse with a broader selection of toolstacks to choose from, this article will show that FreeBSD also contains (more than) a complete selection of tools required to run a Virtual Datacenter (vDC).\nFor most of the applications used here, alternatives exist in the ports and package collection as well. E.g. in addition to minio, there are more distributed storage solutions like ceph or moosefs.\nUsage of Potluck # One major building block we use for quickly setting up the environment below and also for provisioning containerised services are Potluck images/flavours.\nSo beside showcasing the power of FreeBSD as a platform, this article series should give an idea of the concept of Potluck. While admittedly it is still in very early stages, it has the potential to be to complex container images and sets of different interdependent container images (like nomad, consul and traefik) what the FreeBSD package collection is to individual applications.\nThe 3 (or with storage: 4) steps in this article are enough to get a basic configuration of your vDC up and running.\nDepending on your network speed, setting up the core configuration of consul, nomad and traefik in step 3 with the Potluck images will only take a few minutes.\nAt this point, more advanced features like high-availability configuration are not exposed by the images/flavours yet.\nThe plan is clearly to improve this in the coming weeks and months. Last not least, feel free to open a ticket or pull request at the github page if you come across a bug or requirement!\nArchitecture # Components # Our vDC is built on the following tools and applications:\nFreeBSD as base system FreeBSD Jails as container platform and pot as jail management tool minio and s3fs as distributed storage layer (optional) Nomad with pot driver as service orchestration platform Potluck as artifactory (like Dockerhub) Consul as service discovery platform Traefik as reverse proxy Please note that for testing pot, nomad, consul and traefik on one system, a FreeBSD package called minipot is existing that can simply be installed with pkg install minipot.\nWhile minipot is an excellent yet simple demonstration of the basic setup, this article aims to lay the foundation for a setup somebody can transform into an actual production environment.\nSystems Used # We assume you have at least two (physical or virtual) hosts in your configuration:\nConsul/Nomad/Traefik Server (here 10.10.10.10) You could run these services on different hosts as well of course At least one Nomad Compute Node (Client) (here 10.10.10.11) You probably will have more than one such node in real life Step 1 - Installing Pot # Installing pot is simple. Run the following commands on each of your hosts (10.10.10.10 and 10.10.10.11):\n$ pkg install pot ... $ echo kern.racct.enable=1 \u0026gt;\u0026gt; /boot/loader.conf $ sysrc pot_enable=\u0026#34;YES\u0026#34; $ pot init -v kern.racct.enable=1 is not mandatory but needed if you want to allow pot (and thus also nomad) to limit resource usage of jails.\nTo ensure this setting is enabled, reboot both hosts.\nStep 2 - Optional: Storage Configuration # Why Network Storage? # Note: For setting up the environment described here, this chapter is optional. You can also run the two example nomad jobs below without having any network storage, since the example cluster has only one compute node anyway - or you can leave out the permanent mount-in storage for test purposes completely.\nThe basic idea of service orchestration is distributing services across different hosts, based on capacity demands and requirements. The containers (jails) are created on the compute node when they are started and destroyed when they are stopped.\nIf these services require read or potentially write access to storage that lives beyond the lifetime of the service on the host (e.g. database files, data file directories), it is necessary to mount outside storage into these containers on their initialisation.\nIn principle, any network file system like NFS can be used for this, just make sure the same shares are mounted in the same place on each host that potentially could be selected to run the service.\nOur Setup Using minio # Since scale out storage is a good fit to a scale out compute platform, you could follow the minio setup guide to create a distributed storage cluster and simply mount the buckets on each compute node with s3fs via /etc/fstab.\nIf you do that, please note that due to a bug in fusefs-s3fs, you will not be able to chmod directories so they are actually readable by non-root users within the jails (e.g. by the nginx user in the example job below).\nTherefore, you should mount the bucket with appropriately lax umask permissions like in this /etc/fstab entry that you need to add to the 10.10.10.11 node (assuming you otherwise followed the two articles above to set up minio):\ntest-bucket\t/mnt/s3bucket\tfuse\trw,_netdev,uid=1001,gid=1001,allow_other,umask=002,mountprog=/usr/local/bin/s3fs,late,passwd_file=/usr/local/etc/s3fs.accesskey,url=https://10.10.10.10:9000,use_path_request_style,use_cache=/tmp/s3fs,enable_noobj_cache,no_check_certificate,use_xattr,complement_stat\t0 0 ","date":"August 1, 2020","externalUrl":null,"permalink":"/blog/2020-08-01/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\u003ch3 class=\"relative group\"\u003eYes, FreeBSD Lacks Kubernetes - But It Does Not Really Matter\u0026hellip; \n    \u003cdiv id=\"yes-freebsd-lacks-kubernetes---but-it-does-not-really-matter\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#yes-freebsd-lacks-kubernetes---but-it-does-not-really-matter\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003eOne of the main complaints about FreeBSD is the lack of Docker and Kubernetes, which in turn is seen as inability to use FreeBSD as a platform for bleeding edge concepts like micro services and scale-out container orchestration.\u003c/p\u003e","title":"FreeBSD Virtual Data Centre with Potluck: DevOps \u0026 Infrastructure as Code - Part I","type":"blog"},{"content":"","date":"July 28, 2020","externalUrl":null,"permalink":"/tags/amd64/","section":"Tags","summary":"","title":"Amd64","type":"tags"},{"content":"For FreeBSD, there are not many 64bit assembler (amd64) examples available on the web and virtually none that are a little bit more complex.\nTo change this, I ported a DOS Tetris game to 64bit FreeBSD, using SVGALIB to max VGA out with 320x200 and 256 colours - it is 2020 after all!\nJokes aside, this example shows how to set up an assembler program that uses C runtime functions and also links in additional libraries available in ports/packages.\nNote: The keyboard handling could be smoother, but that would require changes that go beyond the scope of this small showcase.\nBackground # The amd64 ABI of FreeBSD follows in most aspects the Linux ABI, so you can frequently work with Linux examples and tutorials.\nThat means parameters to library functions are passed in this order (exluding floats):\n1. rdi 2. rsi 3. rdx 4. rcx 5. r8 6. r9 Note that syscall has different calling conventions though.\nThere are some Linux tutorials that you probably want to look at as well.\nPreparations # The assembler style is nasm, so you need to install the following packages:\n$ pkg install nasm ... $ pkg install svgalib Building and Debugging # You can build the program with this little shell script that uses the system compiler (LLVM) to link with the needed libraries:\n#!/bin/sh # You need to have nasm and svgalib packages installed nasm -w-error=label-redef-late -f elf64 -g -F dwarf tetris.asm export CPATH=/usr/local/include export LIBRARY_PATH=/usr/local/lib cc -g -pipe tetris.o -o tetris -lvga -lvgagl -lm Debugging can simply be done with lldb which comes with the FreeBSD base system. Above shell script adds debug code so the symbols are available in the debugger.\nNote: SVGALIB-programs need to be run as root.\nThe Code # The assembler code can also be found on github.\n;Frame parameters frameHeight equ 8 * 20 frameWidth equ 8 * 10 frameBorderWidth equ 5 frameBeginningX equ 50 frameBeginningY equ 10 nextPieceFrameHeight equ 8 * 5 nextPieceFrameWidth equ 8 * 5 nextPieceFrameBorderWidth equ 5 nextPieceFrameBeginningX equ 155 nextPieceFrameBeginningY equ 10 boardColor equ 0 pieceSize equ 8 scorePosX equ 160 scorePosY equ 100 gameOverPosX equ 120 gameOverPosY equ 80 ; waitFactor*usleepMS should be approx 1sec ; If usleepMS is too long, we will miss keyboard input ; If it is too short, keypresses will be counted several times waitFactor equ 10 usleepMS equ 100000 section .data gameOver: db \u0026#34;GAME OVER\u0026#34;,0 scoreString: db \u0026#34;SCORE\u0026#34;,0 section .bss boardState: resb 20 * 10 nextPieceType: resb 1 pieceType: resb 1 pieceCol: resb 1 piecePos: resb 4 piecePivotPos: resb 1 temporaryPiecePos: resb 4 waitTime: resb 1 score: resw 1 scoreAsString: resb 6 ; 5 digits+0 extern gl_write,gl_setfont,gl_font8x8,gl_setfontcolors,gl_setwritemode,vga_init,vga_setmode,vga_setcolor,gl_fillbox,gl_setcontextvga,vga_getkey,usleep,rand extern exit,keyboard_init,keyboard_translatekeys,keyboard_update,keyboard_keypressed,keyboard_close section .text global main main: ;Initialization sub rsp,8 call vga_init mov rdi, 5 ; SVGALIB 5 = 320x200x256 (see vga.h) call vga_setmode mov rdi, 5 ; we need to init vgagl with the same mode call gl_setcontextvga mov rdi,8 mov rsi,8 mov rdx, [gl_font8x8] call gl_setfont ; see man gl_write(3) mov rdi,2 call gl_setwritemode mov rdi, 0 mov rsi, 15 call gl_setfontcolors call keyboard_init mov rdi, 7 call drawFrame call drawNextPieceFrame jmp gameInProgres endOfGame: call delayForLongWhile call displayGameOver call delayForLongWhile ;Return to previous video mode call keyboard_close mov rdi, 0 call vga_setmode ;Finish program ; we can not simply call return because we might be called from within ; gameInProgress mov rdi, 0 call exit ;/////Drawing FUNCTIONS//// ;//////////////////////////////////////////////////////////// ;Draw Rectangle, rax - begin point, rcx - Height, rbx - Width, rdi - color drawRect: xor rdx, rdx ; clear dx for ax division push rdi ; Save colour Push rcx ; Save height for later mov rcx, 320 ; divisor in cx div cx ; divide begin point/320 so we get y in ax and x in dx (remainder) mov rdi, rdx ; x1 (param 1) mov rsi, rax ; y1 (param 2) mov rdx, rbx ; width (param 3) pop rcx ; height (param 4, fetch again from stack) pop r8 ; Pass colour call gl_fillbox mov rdi, r8 ; ...and restore colour ret ;//////////////////////////////////////////////////////////// drawFrame: mov rdi, 7 ; colour ;Gora mov rax, frameBeginningY * 320 + frameBeginningX mov rcx, frameBorderWidth mov rbx, frameWidth + 2 * frameBorderWidth call drawRect ;Lewa mov rax, (frameBeginningY + frameBorderWidth) * 320 + frameBeginningX mov rcx, frameHeight mov rbx, frameBorderWidth call drawRect ;Prawa mov rax, (frameBeginningY + frameBorderWidth) * 320 + frameBeginningX + frameWidth + frameBorderWidth mov rcx, frameHeight mov rbx, frameBorderWidth call drawRect ;Dol mov rax, (frameBeginningY + frameHeight + frameBorderWidth) * 320 + frameBeginningX mov rcx, frameBorderWidth mov rbx, frameWidth + 2 * frameBorderWidth call drawRect ret ;//////////////////////////////////////////////////////////// drawNextPieceFrame: mov rdi, 7 ;Gora mov rax, nextPieceFrameBeginningY * 320 + nextPieceFrameBeginningX mov rcx, nextPieceFrameBorderWidth mov rbx, nextPieceFrameWidth + 2 * nextPieceFrameBorderWidth call drawRect ;Lewa mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth) * 320 + nextPieceFrameBeginningX mov rcx, nextPieceFrameHeight mov rbx, nextPieceFrameBorderWidth call drawRect ;Prawa mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth) * 320 + nextPieceFrameBeginningX + nextPieceFrameWidth + nextPieceFrameBorderWidth mov rcx, nextPieceFrameHeight mov rbx, nextPieceFrameBorderWidth call drawRect ;Dol mov rax, (nextPieceFrameBeginningY + nextPieceFrameHeight + nextPieceFrameBorderWidth) * 320 + nextPieceFrameBeginningX mov rcx, nextPieceFrameBorderWidth mov rbx, nextPieceFrameWidth + 2 * nextPieceFrameBorderWidth call drawRect ret ;//////////////////////////////////////////////////////////// drawBoardState: mov rbx, 200 drawBoardStateLoop1: dec rbx push rbx movzx rdi, byte[boardState + rbx] mov rax, rbx call drawOneSquare pop rbx cmp rbx, 0 jne drawBoardStateLoop1 ret ;//////////////////////////////////////////////////////////// clearBoard: mov rax, (frameBeginningY + frameBorderWidth) * 320 + frameBeginningX + frameBorderWidth mov rcx, frameHeight mov rbx, frameWidth mov rdi, boardColor call drawRect ret ;//////////////////////////////////////////////////////////// drawTetromino: movzx rax, byte[piecePos] movzx rdi, byte[pieceCol] call drawOneSquare movzx rax, byte[piecePos + 1] movzx rdi, byte[pieceCol] call drawOneSquare movzx rax, byte[piecePos + 2] movzx rdi, byte[pieceCol] call drawOneSquare movzx rax, byte[piecePos + 3] movzx rdi, byte[pieceCol] call drawOneSquare ret ;//////////////////////////////////////////////////////////// ;rax - PieceNumber, rdi - color drawOneSquare: mov BL, 10 div BL ;AH - X, AL - Y mov CX, AX ;Calculate Y offset mov AL, CL xor AH, AH mov BX, 320 * pieceSize mul BX push rax ;Calculate X offset mov AL, CH xor AH, AH mov BX, pieceSize mul BX pop rdx ;Move to fit frame add AX, DX add AX, (frameBeginningY + frameBorderWidth) * 320 + frameBeginningX + frameBorderWidth mov BX, pieceSize mov CX, pieceSize jmp drawRect ;/////GAME FUNCTIONS//// ;//////////////////////////////////////////////////////////// gameInProgres: ;-TO DO mov byte[waitTime], waitFactor call generateNextPieceNumber placeNext: call updateBoard call generateNextPiece call generateNextPieceNumber call setNewDelay call writeScore jmp checkIfNotEnd pieceInProgress: call clearBoard call drawBoardState call drawTetromino call scoreToString call getPlayerInput cmp AX, 0x0F0F ;AX = FFFF - Place Next Piece cmp AX, 0xFFFF je placeNext call moveOneDown cmp AX, 0xFFFF je placeNext jmp pieceInProgress ;--------- checkIfNotEnd: movzx rbx, byte[piecePos] mov AL, [boardState + rbx] cmp AL, boardColor jne endOfGame movzx rbx, byte[piecePos + 1] movzx rax, byte[boardState + rbx] cmp AL, boardColor jne endOfGame movzx rbx, byte[piecePos + 2] movzx rax, byte[boardState + rbx] cmp AL, boardColor jne endOfGame movzx rbx, byte[piecePos + 3] movzx rax, byte[boardState + rbx] cmp AL, boardColor jne endOfGame jmp pieceInProgress ;//////////////////////////////////////////////////////////// getPlayerInput: movzx rcx, byte[waitTime] waitForKey: dec CX cmp CX, 0 je noInput ; We need to run this in a loop with frequent polling. ; If we just check every second or so if a keyboard has been pressed, ; we miss out on key events with keyboard_update from svgalib push rcx ; we need to save the loop counter across the svgalib calls ; the various push/pops are not super-elegant, yes, I know... call delayForWhile ; we need to ensure that we don\u0026#39;t call this too frequently ; otherwise the same keypress is returned again call keyboard_update pop rcx mov rdi, 75 ;SCANCODE_CURSORLEFT push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne moveOneLeft mov rdi, 97 ;SCANCODE_CURSORBLOCKLEFT push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne moveOneLeft mov rdi, 80 ;SCANCODE_CURSORSDOWN push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne downKey mov rdi, 100 ;SCANCODE_CURSORBLOCKDOWN push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne downKey mov rdi, 77 ;SCANCODE_CURSORRIGHT push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne moveOneRight mov rdi, 98 ;SCANCODE_CURSORBLOCKRIGHT push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne moveOneRight mov rdi, 72 ;SCANCODE_CURSORUP push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne rotateCounterClockwise mov rdi, 95 ;SCANCODE_CURSORBLOCKUP push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne rotateCounterClockwise mov rdi, 76 ;SCANCODE_KEYPAD5 push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne rotateClockwise mov rdi, 1 ;SCANCODE_ESCAPE push rcx call keyboard_keypressed pop rcx cmp rax, 0 jne endOfGame jmp waitForKey noInput: xor rax, rax ret downKey: inc word[score] call moveOneDown xor rax, rax ret ;//////////////////////////////////////////////////////////// generateNextPieceNumber: mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth mov rcx, nextPieceFrameHeight mov rbx, nextPieceFrameWidth mov rdi, boardColor call drawRect ; Random for next piece call rand xor DX, DX mov CX, 7 div CX mov byte[nextPieceType], DL mov AH, DL ; DOS Code expects next piece type in AH... genFirstPiece: ;I cmp AH, 0 jne genSecondPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + 4 mov rcx, pieceSize mov rbx, 4 * pieceSize mov rdi, 52 call drawRect ret genSecondPiece: ;J cmp AH, 1 jne genThirdPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, 3 * pieceSize mov rdi, 32 call drawRect mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + 3 * pieceSize mov rcx, pieceSize mov rbx, pieceSize mov rdi, 32 call drawRect ret genThirdPiece: ;L cmp AH, 2 jne genForthPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, 3 * pieceSize mov rdi, 43 call drawRect mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, pieceSize mov rdi, 43 call drawRect ret genForthPiece: ;O cmp AH, 3 jne genFifthPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize + 4 mov rcx, 2 * pieceSize mov rbx, 2 * pieceSize mov rdi, 45 call drawRect ret genFifthPiece: ;S cmp AH, 4 jne genSixthPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + 2 * pieceSize mov rcx, pieceSize mov rbx, 2 * pieceSize mov rdi, 48 call drawRect mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, 2 * pieceSize mov rdi, 48 call drawRect ret genSixthPiece: ;T cmp AH, 5 jne genSeventhPiece mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, 3 * pieceSize mov rdi, 34 call drawRect mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + 2 * pieceSize mov rcx, pieceSize mov rbx, pieceSize mov rdi, 34 call drawRect ret genSeventhPiece: ;Z mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + pieceSize mov rcx, pieceSize mov rbx, 2 * pieceSize mov rdi, 40 call drawRect mov rax, (nextPieceFrameBeginningY + nextPieceFrameBorderWidth + 2 * pieceSize + 4) * 320 + nextPieceFrameBeginningX + nextPieceFrameBorderWidth + 2 * pieceSize mov rcx, pieceSize mov rbx, 2 * pieceSize mov rdi, 40 call drawRect ret ;--------------------- generateNextPiece: mov AH, byte[nextPieceType] mov byte[pieceType], AH mov byte[piecePivotPos], 3 firstPiece: ;I cmp AH, 0 jne secondPiece mov byte[piecePos], 13 mov byte[piecePos + 1], 14 mov byte[piecePos + 2], 15 mov byte[piecePos + 3], 16 mov byte[pieceCol], 52 ret secondPiece: ;J cmp AH, 1 jne thirdPiece mov byte[piecePos], 13 mov byte[piecePos + 1], 14 mov byte[piecePos + 2], 15 mov byte[piecePos + 3], 25 mov byte[pieceCol], 32 ret thirdPiece: ;L cmp AH, 2 jne forthPiece mov byte[piecePos], 13 mov byte[piecePos + 1], 14 mov byte[piecePos + 2], 15 mov byte[piecePos + 3], 23 mov byte[pieceCol], 43 ret forthPiece: ;O cmp AH, 3 jne fifthPiece mov byte[piecePos], 14 mov byte[piecePos + 1], 15 mov byte[piecePos + 2], 24 mov byte[piecePos + 3], 25 mov byte[pieceCol], 45 ret fifthPiece: ;S cmp AH, 4 jne sixthPiece mov byte[piecePos], 14 mov byte[piecePos + 1], 15 mov byte[piecePos + 2], 23 mov byte[piecePos + 3], 24 mov byte[pieceCol], 48 ret sixthPiece: ;T cmp AH, 5 jne seventhPiece mov byte[piecePos], 13 mov byte[piecePos + 1], 14 mov byte[piecePos + 2], 15 mov byte[piecePos + 3], 24 mov byte[pieceCol], 34 ret seventhPiece: ;Z mov byte[piecePos], 13 mov byte[piecePos + 1], 14 mov byte[piecePos + 2], 24 mov byte[piecePos + 3], 25 mov byte[pieceCol], 40 ret ;/////////////////////////////////////////////////////////// solidifyPiece: movzx rbx, byte[piecePos] mov AL, byte[pieceCol] mov byte[boardState + rbx], AL movzx rbx, byte[piecePos + 1] mov byte[boardState + rbx], AL movzx rbx, byte[piecePos + 2] mov byte[boardState + rbx], AL movzx rbx, byte[piecePos + 3] mov byte[boardState + rbx], AL ret ;///////////////// - TO DO updateBoard: mov DL, 20 updateBoardLoop: dec DL call clearOneRow cmp DL, 0 jne updateBoardLoop ret clearOneRow: ;DL - row To clear mov BL, 10 mov AL, DL mul BL xor rbx, rbx mov BX, AX mov CX, 10 clearOneRowLoop1: cmp byte[boardState + rbx], boardColor je notclearOneRow inc BX loop clearOneRowLoop1 add word[score], 100 cmp DL, 0 je notclearOneRow push rdx clearOneRowLoop2: dec DL call moveRowDown cmp DL, 1 jne clearOneRowLoop2 pop rdx jmp clearOneRow notclearOneRow: ret moveRowDown: ;DL - beginRow mov BL, 10 mov AL, DL mul BL xor rbx, rbx mov BX, AX mov CX, 10 moveRowDownLoop: mov AL, byte[boardState + rbx] mov byte[boardState + rbx + 10], AL mov byte[boardState + rbx], boardColor inc BX loop moveRowDownLoop ret ;///////////////// - TO DO moveOneDown: xor rax, rax mov rbx, 10 xor DL, DL ;Check Frame Collision cmp byte[piecePos], 19 * 10 jae cantMoveOneDown cmp byte[piecePos + 1], 19 * 10 jae cantMoveOneDown cmp byte[piecePos + 2], 19 * 10 jae cantMoveOneDown cmp byte[piecePos + 3], 19 * 10 jae cantMoveOneDown ;Check Space Collsion xor rbx, rbx mov BL, byte[piecePos] add BL, 10 cmp byte[boardState + rbx], boardColor jne cantMoveOneDown mov BL, byte[piecePos + 1] add BL, 10 cmp byte[boardState + rbx], boardColor jne cantMoveOneDown mov BL, byte[piecePos + 2] add BL, 10 cmp byte[boardState + rbx], boardColor jne cantMoveOneDown mov BL, byte[piecePos + 3] add BL, 10 cmp byte[boardState + rbx], boardColor jne cantMoveOneDown add byte[piecePos], 10 add byte[piecePos + 1], 10 add byte[piecePos + 2], 10 add byte[piecePos + 3], 10 add byte[piecePivotPos], 10 xor rax, rax ret cantMoveOneDown: call solidifyPiece mov rax, 0xFFFF ret ;----------------------------------- moveOneLeft: mov BL, 10 ;Check Frame Collision xor AX, AX mov AL, byte[piecePos] div BL cmp AH, 0 je cantMoveOneLeft xor AX, AX mov AL, byte[piecePos + 1] div BL cmp AH, 0 je cantMoveOneLeft xor AX, AX mov AL, byte[piecePos + 2] div BL cmp AH, 0 je cantMoveOneLeft xor AX, AX mov AL, byte[piecePos + 3] div BL cmp AH, 0 je cantMoveOneLeft ;Check Space Collsion xor rbx, rbx mov BL, byte[piecePos] dec BL cmp byte[boardState + rbx], boardColor jne cantMoveOneLeft mov BL, byte[piecePos + 1] dec BL cmp byte[boardState + rbx], boardColor jne cantMoveOneLeft mov BL, byte[piecePos + 2] dec BL cmp byte[boardState + rbx], boardColor jne cantMoveOneLeft mov BL, byte[piecePos + 3] dec BL cmp byte[boardState + rbx], boardColor jne cantMoveOneLeft dec byte[piecePos] dec byte[piecePos + 1] dec byte[piecePos + 2] dec byte[piecePos + 3] dec byte[piecePivotPos] cantMoveOneLeft: xor AX, AX ret ;----------------------------------- moveOneRight: mov BL, 10 ;Check Frame Collision xor AX, AX mov AL, byte[piecePos] div BL cmp AH, 9 je cantMoveOneRight xor AX, AX mov AL, byte[piecePos + 1] div BL cmp AH, 9 je cantMoveOneRight xor AX, AX mov AL, byte[piecePos + 2] div BL cmp AH, 9 je cantMoveOneRight xor AX, AX mov AL, byte[piecePos + 3] div BL cmp AH, 9 je cantMoveOneRight ;Check Space Collsion xor rbx, rbx mov BL, byte[piecePos] inc BL cmp byte[boardState + rbx], boardColor jne cantMoveOneRight mov BL, byte[piecePos + 1] inc BL cmp byte[boardState + rbx], boardColor jne cantMoveOneRight mov BL, byte[piecePos + 2] inc BL cmp byte[boardState + rbx], boardColor jne cantMoveOneRight mov BL, byte[piecePos + 3] inc BL cmp byte[boardState + rbx], boardColor jne cantMoveOneRight inc byte[piecePos] inc byte[piecePos + 1] inc byte[piecePos + 2] inc byte[piecePos + 3] inc byte[piecePivotPos] cantMoveOneRight: ret rotateClockwise: ;CheckBoard mov rbx, 4 rotateClockwiseLoop1: dec BX push rbx xor AX, AX mov BL, 10 mov AL, byte[piecePivotPos] div BL pop rbx cmp AH, 0 jb cantRotateClockwise cmp AH, 6 ja cantRotateClockwise cmp AL, 16 ja cantRotateClockwise mov CX, AX xor AX, AX mov AL, byte[piecePos + rbx] push rbx mov BL, 10 sub AL, byte[piecePivotPos] div BL mov DX, AX ;AH - X, AL - Y mov AH, DL cmp byte[pieceType], 0 je rotateClockwiseSpc mov AL, 3 rotateClockwiseSpcBack: sub AL, DH mov DX, AX mov AL, DL mov BL, 10 mul BL add AL, DH pop rbx add AL, byte[piecePivotPos] mov byte[temporaryPiecePos + rbx], AL cmp BX, 0 jne rotateClockwiseLoop1 xor rbx, rbx mov BL, byte[temporaryPiecePos] cmp byte[boardState + rbx], boardColor jne cantRotateClockwise mov BL, byte[temporaryPiecePos + 1] cmp byte[boardState + rbx], boardColor jne cantRotateClockwise mov BL, byte[temporaryPiecePos + 2] cmp byte[boardState + rbx], boardColor jne cantRotateClockwise mov BL, byte[temporaryPiecePos + 3] cmp byte[boardState + rbx], boardColor jne cantRotateClockwise mov AL, byte[temporaryPiecePos] mov byte[piecePos], AL mov AL, byte[temporaryPiecePos + 1] mov byte[piecePos + 1], AL mov AL, byte[temporaryPiecePos + 2] mov byte[piecePos + 2], AL mov AL, byte[temporaryPiecePos + 3] mov byte[piecePos + 3], AL cantRotateClockwise: xor AX, AX ret rotateClockwiseSpc: mov AL, 4 jmp rotateClockwiseSpcBack ;///////////////// rotateCounterClockwise: ;CheckBoard mov rbx, 4 rotateCounterClockwiseLoop1: dec BX push rbx xor AX, AX mov BL, 10 mov AL, byte[piecePivotPos] div BL pop rbx cmp AH, 0 jb cantRotateCounterClockwise cmp AH, 6 ja cantRotateCounterClockwise cmp AL, 16 ja cantRotateCounterClockwise mov CX, AX xor AX, AX mov AL, byte[piecePos + rbx] push rbx mov BL, 10 sub AL, byte[piecePivotPos] div BL mov DX, AX ;AH - X, AL - Y mov AL, DH cmp byte[pieceType], 0 je rotateCounterClockwiseSpc mov AH, 3 rotateCounterClockwiseSpcBack: sub AH, DL mov DX, AX mov AL, DL mov BL, 10 mul BL add AL, DH pop rbx add AL, byte[piecePivotPos] mov byte[temporaryPiecePos + rbx], AL cmp BX, 0 jne rotateCounterClockwiseLoop1 ;xor BX, BX movzx r8, byte[temporaryPiecePos] cmp byte[boardState + r8], boardColor jne cantRotateCounterClockwise movzx r8, byte[temporaryPiecePos + 1] cmp byte[boardState + r8], boardColor jne cantRotateCounterClockwise movzx r8, byte[temporaryPiecePos + 2] cmp byte[boardState + r8], boardColor jne cantRotateCounterClockwise movzx r8, byte[temporaryPiecePos + 3] cmp byte[boardState + r8], boardColor jne cantRotateCounterClockwise mov AL, byte[temporaryPiecePos] mov byte[piecePos], AL mov AL, byte[temporaryPiecePos + 1] mov byte[piecePos + 1], AL mov AL, byte[temporaryPiecePos + 2] mov byte[piecePos + 2], AL mov AL, byte[temporaryPiecePos + 3] mov byte[piecePos + 3], AL cantRotateCounterClockwise: xor AX, AX ret rotateCounterClockwiseSpc: mov AH, 4 jmp rotateCounterClockwiseSpcBack ;///////////////// ;Delay for one/10 second (loop in input is 10 times) delayForWhile: mov rdi, usleepMS call usleep ret ;-------------------- delayForLongWhile: mov rdi, 0x000F8480 call usleep ret ;///////////////// setNewDelay: cmp byte[waitTime], 1 jb noSetNewDelat mov AX, word[score] xor DX, DX mov BX, 1000 div BX mov DX, AX mov AX, 10 cmp AX, DX jl set1Delay sub AX, DX mov byte[waitTime], AL noSetNewDelat: ret set1Delay: mov byte[waitTime], 1 ret ;///////////////// scoreToString: xor DX, DX mov AX, word[score] mov BX, 10000 div BX add AX, 48 mov byte[scoreAsString], AL mov AX, DX xor DX, DX mov BX, 1000 div BX add AX, 48 mov byte[scoreAsString + 1], AL mov AX, DX xor DX, DX mov BX, 100 div BX add AX, 48 mov byte[scoreAsString + 2], AL mov AX, DX xor DX, DX mov BX, 10 div BX add AX, 48 mov byte[scoreAsString + 3], AL add DX, 48 mov byte[scoreAsString + 4], DL mov byte[scoreAsString + 5], 0 ; 0 for C string end mov BX, 5 mov DL, 24 mov rdi, scorePosX mov rsi, scorePosY-15 mov rdx, scoreString call gl_write mov rdi, scorePosX ; x of position mov rsi, scorePosY ; y of position mov rdx, scoreAsString ; position of string (=1 character) call gl_write ret ;//////////////////////// writeScore: mov rdi, scorePosX mov rsi, scorePosY-15 mov rdx, scoreString call gl_write mov rdi, scorePosX ; x of position mov rsi, scorePosY ; y of position mov rdx, scoreAsString ; position of string (=1 character) call gl_write ret ;---------------- displayGameOver: xor rax, rax mov rcx, 200 mov rbx, 320 mov rdi, boardColor call drawRect mov rdi, gameOverPosX ; x of position mov rsi, gameOverPosY ; y of position mov rdx, gameOver ; position of string (=1 character) call gl_write ret ","date":"July 28, 2020","externalUrl":null,"permalink":"/blog/2020-07-28/","section":"Tech Blog","summary":"\u003cp\u003eFor FreeBSD, there are not many 64bit assembler (amd64) examples available on the web and virtually none that are a little bit more complex.\u003c/p\u003e\n\u003cp\u003eTo change this, I ported \u003ca\n  href=\"https://github.com/Arkowski24/nasm-tetris\"\n    target=\"_blank\"\n  \u003ea DOS Tetris game\u003c/a\u003e to 64bit FreeBSD, using SVGALIB to max VGA out with  320x200 and 256 colours - it is 2020 after all!\u003c/p\u003e","title":"FreeBSD 64bit Assembler (amd64)","type":"blog"},{"content":"","date":"July 28, 2020","externalUrl":null,"permalink":"/tags/nasm/","section":"Tags","summary":"","title":"Nasm","type":"tags"},{"content":"","date":"July 28, 2020","externalUrl":null,"permalink":"/tags/svgalib/","section":"Tags","summary":"","title":"Svgalib","type":"tags"},{"content":"","date":"July 28, 2020","externalUrl":null,"permalink":"/tags/x64/","section":"Tags","summary":"","title":"X64","type":"tags"},{"content":"","date":"July 23, 2020","externalUrl":null,"permalink":"/tags/distributed-file-system/","section":"Tags","summary":"","title":"Distributed File System","type":"tags"},{"content":"","date":"July 23, 2020","externalUrl":null,"permalink":"/tags/ffs/","section":"Tags","summary":"","title":"FFS","type":"tags"},{"content":"","date":"July 23, 2020","externalUrl":null,"permalink":"/tags/zfs/","section":"Tags","summary":"","title":"ZFS","type":"tags"},{"content":" Introduction # minio is a well-known S3 compatible object storage platform that supports high availability and scalability features and is very easy to configure.\nThere is a separate post already describing how to set up minio on FreeBSD.\nThis post explains how you can use minio (or any other S3-compatible storage platform) to provide HA filesystems on FreeBSD.\nWe describe two ways to mount S3 buckets into the file system on FreeBSD:\ns3fs (pkg install fusefs-s3fs) s3backer (pkg install fusefs-s3backer) with any filesystem like ZFS or FFS on top s3fs provides direct access to the bucket content but is relatively slow. It can be mounted on many nodes at the same time though.\ns3backerprovides the S3 bucket as block device and you can use any file system like ZFS on top of it. With appropriately configured caching it can be very fast since transactions are asynchronous.\nSince a zpool or other file systems can only be mounted on one host at the same time, it is suitable e.g. for HA storage or if a container (jail) can be run on different hosts but not in parallel (e.g. in a failover scenario). Then, the filesystem in the bucket can be unmounted and re-mounted on another host via network quickly.\nOf course, network drives can also be mounted e.g. via NFS, but minio as storage layer provides scale out and high availability features that are not easily replicated by NFS file servers or NAS systems.\nThe following steps use the same values as the previous post explaining how to set up minio.\nfusefs \u0026amp; Credential Preparations # To ensure that fusefs-based filesystems can be run on FreeBSD, you need to add fuse to /boot/loader.conf:\n$ echo fuse_load=\u0026#34;YES\u0026#34; \u0026gt;\u0026gt; /boot/loader.conf Either reboot or load the module the first time by hand:\n$ kldload fuse Also, both S3-tools need the access key. Create a file containing the login credentials from your minio setup:\n$ echo \u0026#34;myaccesskey:mysecretkey\u0026#34; \u0026gt; /usr/local/etc/s3fs.accesskey Mount Bucket as Filesystem # Install the file system driver:\n$ pkg install fusefs-s3fs Create a mount point, e.g. /mnt/s3bucket and the bucket you want to mount, in our case test-bucket (simply via the minio web interface).\nThen you can mount the bucket already:\n$ s3fs test-bucket /mnt/s3bucket -o passwd_file=/usr/local/etc/s3fs.accesskey,url=https://10.10.10.10:9000 -o use_path_request_style -o use_cache=/tmp/s3fs -o enable_noobj_cache -o no_check_certificate -o use_xattr -o complement_stat You can get a detailed description of the options via man s3fs.\nIf you want to mount the bucket automatically on boot, add the following line to your /etc/fstab:\ntest-bucket\t/mnt/s3bucket\tfuse\trw,_netdev,allow_other,mountprog=/usr/local/bin/s3fs,late,passwd_file=/usr/local/etc/s3fs.accesskey,url=https://10.10.10.10:9000,use_path_request_style,use_cache=/tmp/s3fs,enable_noobj_cache,no_check_certificate,use_xattr,complement_stat\t0\t0 Bucket As Raw Device With ZFS # To create a filesystem like ZFS inside the bucket, install s3backer:\n$ pkg install fusefs-s3backer We again use an empty bucket called test-bucket which we mount from the command line first (assuming that you have unmounted the s3fs-bucket above first):\n$ s3backer --blockSize=128k --size=500g --listBlocks test-bucket --baseURL=https://10.10.10.10:9000/ --insecure --accessFile=/usr/local/etc/s3fs.accesskey --blockCacheFile=/tmp/s3backer.cache --blockCacheSize=1000000 /mnt/s3bucket This command creates a 500GB raw device and also sets the cache size to 1,000,000 128k blocks.\nThen let us create the zpool:\n$ zpool create -O compression=lz4 -O atime=off s3pool /mnt/s3bucket/file Note that there is file being appended to the path that you have used for mounting the bucket.\nÀfter it has been created, it can be imported simply by running e.g.\n$ zpool import -d /mnt/s3bucket s3pool That\u0026rsquo;s it.\nYou can also mount the block device again via /etc/fstab:\ns3pool /mnt/s3bucket fuse rw,allow_other,mountprog=/usr/local/bin/s3backer,late,blockSize=128k,size=500g,listBlocks,baseURL=https://10.10.10.10:9000/,insecure,accessFile=/usr/local/etc/s3fs.accesskey,blockCacheFile=/tmp/s3backer.cache,blockCacheSize=1000000 0 0 After mounting the block device in /etc/fstab, you will want to mount the the zpool.\n","date":"July 23, 2020","externalUrl":null,"permalink":"/blog/2020-07-23/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003ca\n  href=\"https://min.io\"\n    target=\"_blank\"\n  \u003eminio\u003c/a\u003e is a well-known S3 compatible object storage platform that supports high availability and scalability features and is very easy to configure.\u003c/p\u003e","title":"ZFS High Availability Filesystem With minio on FreeBSD","type":"blog"},{"content":" Introduction # minio is a well-known S3 compatible object storage platform that supports high availability features.\nFor FreeBSD a port is available that has already been described in 2018 on the vermaden blog.\nNonetheless, for a distributed setup along the lines of the minio documentation with TLS encryption, even the official minio documentation unfortunately lacks some detail.\nThat means the certificate setup below might be interesting even if you plan to run minio on another platform and not FreeBSD.\nIn the offical TLS documentation linked to above, self signed certificates are described but not how a Root CA certificate is created which is needed when minio is to be run not only on a single host.\nThe setup in this document assumes two hosts with two drives each, the minimum setup for enabling Erasure Code. This setup can easily be scaled to more nodes and also managed by tools like ànsible.\nBoth hosts in this example use the IP addresses 10.10.10.10 and 10.10.10.11 and each mounts its two drives at /mnt/minio-drive1 and /mnt/minio-drive2.\nInstalling The Packages # Install the following packages on both hosts to run minio:\n$ pkg install minio ... $ pkg install openssl ... Certificate Set Up # On both hosts, create /usr/local/etc/ssl and /usr/local/etc/ssl/CAs directories.\nOn one of the hosts, run the following commands in the /usr/local/etc/ssl/CAs directory:\nopenssl genrsa -out rootca.key 8192 openssl req -sha256 -new -x509 -days 3650 -key rootca.key -out rootca.crt Copy these root certificates into the /usr/local/etc/ssl/CAs directory of the other system.\nOn both systems, create /usr/local/etc/ssl/openssl.conf. Replace the example IP address 10.10.10.10 below with the IP address of the host on which you create this file:\nbasicConstraints = CA:FALSE nsCertType = server nsComment = \u0026#34;OpenSSL Generated Server Certificate\u0026#34; subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth subjectAltName = @alt_names [alt_names] IP.1 = 10.10.10.10 This alt name is important as minio will check the certificate. Please note that the document Use OpenSSL (with IP address) to Generate a Certificate also lists a similar configuration file which does not correctly add the alt names to the certificate though.\nAfterwards, in the same directory, run\n$ openssl genrsa -out private.key 4096 ... $ openssl req -new -key private.key -out public.csr ... $ openssl x509 -req -in public.csr -CA CAs/rootca.crt -CAkey CAs/rootca.key -CAcreateserial -out public.crt -days 3650 -sha256 -extfile openssl.conf Feel free to use appropriate values for each of the certificate attributes when asked by openssl.\nYou can check if the correct alt name IP address entry is contained in the certificate with openssl x509 -noout -text -in public.crt - if you don\u0026rsquo;t see it in the output, check your configuration file above.\nConfigure minio Service # Add the following to the /etc/rc.conf file on both systems:\nminio_enable=“YES“ minio_disks=“https://10.10.10.10:9000/mnt/minio-data1 https://10.10.10.10:9000/mnt/minio-data2 https://10.10.10.11:9000/mnt/minio-data1 https://10.10.10.11:9000/mnt/minio-data2“ minio_certs=\u0026#34;/usr/local/etc/ssl\u0026#34; minio_env=\u0026#34;MINIO_ACCESS_KEY=myaccesskey MINIO_SECRET_KEY=mysecretkey\u0026#34; Prepare Drives and Start minio # On both systems, run\n$ chown minio /mnt/minio-data1 \u0026amp;\u0026amp; chmod u+rxw /mnt/minio-data1 $ chown minio /mnt/minio-data2 \u0026amp;\u0026amp; chmod u+rxw /mnt/minio-data2 To test your configuration, you can run minio from the shell so you see the output:\nsu -m minio -c \u0026#39;env \\\\ MINIO_ACCESS_KEY=myaccesskey \\\\ MINIO_SECRET_KEY=mysecretkey \\\\ minio server \\\\ -S /usr/local/etc/ssl \\\\ https://10.10.10.10:9000/mnt/minio-data1 \\\\ https://10.10.10.10:9000/mnt/minio-data2 \\\\ https://10.10.10.11:9000/mnt/minio-data1 \\\\ https://10.10.10.11:9000/mnt/minio-data2\u0026#39; Please note that the \u0026ndash;config-dir parameter shown in old documentation is obsolete as minio stores the configuration within the data directories meanwhile.\nWhen everything works, you should be able to connect by using the access key and the secret key from above as credentials to both systems with your browser on https://10.10.10.10:9000 and https://10.10.10.11:9000 and both hosts should replicate all the data that you load into each of them.\nIf everything works, stop both minio server processes with kill -9 and start them as service with /usr/local/etc/rc.d/minio start.\n","date":"July 8, 2020","externalUrl":null,"permalink":"/blog/2020-07-08/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003ca\n  href=\"https://min.io\"\n    target=\"_blank\"\n  \u003eminio\u003c/a\u003e is a well-known S3 compatible object storage platform that supports high availability features.\u003c/p\u003e\n\u003cp\u003eFor FreeBSD a port is available that has already been described in 2018 on the \u003ca\n  href=\"https://vermaden.wordpress.com/2018/04/16/distributed-object-storage-with-minio-on-freebsd/\"\n    target=\"_blank\"\n  \u003evermaden blog\u003c/a\u003e.\u003c/p\u003e","title":"minio Distributed Mode on FreeBSD","type":"blog"},{"content":"","date":"June 21, 2020","externalUrl":null,"permalink":"/tags/browser/","section":"Tags","summary":"","title":"Browser","type":"tags"},{"content":"","date":"June 21, 2020","externalUrl":null,"permalink":"/tags/firefox/","section":"Tags","summary":"","title":"Firefox","type":"tags"},{"content":" Overview # pot is a great and relatively new jail management tool. It offers DevOps style provisioning and can even be used to provide Docker-like, scalable cloud services together with nomad and consul (more about this in Orchestrating jails with nomad and pot).\nWhen using FreeBSD on your desktop, you can also use it simply to easily create \u0026ldquo;throw away\u0026rdquo; browser jails. That way, the browser environment is reliably and completely erased and reset each time you re-create it with one single, simple command.\nPlease note that all the pot commands below should be run as root.\nSetting up pot # Installing and setting up potis described in detail on pot.pizzamig.dev but it is quite straight-forward if you stick to the sensible defaults:\n$ pkg install pot ... $ pot init -v Creating the Jail Flavour (Template) # After that, we create a pot flavour that contains all the details about our browser jail in /usr/local/etc/pot/flavours.\nThe flavour contains of two simple files in this directory.\n/usr/local/etc/pot/flavours/browser:\nset-attribute -A no-rc-script -V YES set-attribute -A persistent -V NO /usr/local/etc/pot/flavours/browser.sh:\n#!/bin/sh [ -w /etc/pkg/FreeBSD.conf ] \u0026amp;\u0026amp; sed -i \u0026#39;\u0026#39; \u0026#39;s/quarterly/latest/\u0026#39; /etc/pkg/FreeBSD.conf ASSUME_ALWAYS_YES=yes pkg bootstrap touch /etc/rc.conf sysrc sendmail_enable=\u0026#34;NONE\u0026#34; sysrc sshd_enable=\u0026#34;YES\u0026#34; echo myjailpassword | pw add user ffoxuser -h 0 mkdir /home/ffoxuser chown ffoxuser /home/ffoxuser pkg install -y xauth firefox pkg clean -y Make the shell script executable (otherwise it is ignored by pot):\n$ chmod ugo+x browser.sh Of course you could instead of firefox also install chromium or any other browser or application.\nCreating a One-Time Jail and Running Your Browser # Now you can simply create the jail with\n$ sudo pot create -p onetime-browser -b 12.1 -N public-bridge -t single -f browser and watch the output:\n===\u0026gt; Creating a new pot ===\u0026gt; pot name : onetime-browser ===\u0026gt; type : single ===\u0026gt; base : 12.1 ===\u0026gt; pot_base : ===\u0026gt; level : 0 ===\u0026gt; network-type: public-bridge ===\u0026gt; ip : 10.192.0.3 ===\u0026gt; bridge : ===\u0026gt; dns : inherit ===\u0026gt; flavours : browser ===\u0026gt; Fetching FreeBSD 12.1 ... You will note that pot assigns a private IP address to the jail, in the case above 10.192.0.3.\nStart the browser jail with\n$ pot start onetime-browser and then start firefox in the jail with\n$ ssh -X ffoxuser@10.192.0.3 firefox If necessary, replace the IP address with the IP address from your output.\nIf you copied the configuration from above, the password you will be asked for is myjailpassword.\nClean Up Afterwards\u0026hellip; # As soon as you are done browsing and you have closed the browser session, stop the jail with\n$ pot stop onetime-browser and delete the jail with\n$ pot destroy -p onetime-browser All the data associated with your browsing session will be completely removed.\n\u0026hellip;Or Use Snapshots # A lot more efficient than recreating the jail from scratch each time would be taking a snapshot immediately after creating the jail with\n$ pot snapshot -p onetime-browser and instead of deleting the jail simply roll back to it after having finished browsing:\n$ pot revert -p onetime-browser ","date":"June 21, 2020","externalUrl":null,"permalink":"/blog/2020-06-21/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eOverview \n    \u003cdiv id=\"overview\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#overview\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003e\u003ccode\u003epot\u003c/code\u003e is a great and relatively new jail management tool. It offers DevOps style provisioning and can even be used to provide Docker-like, scalable cloud services together with \u003ccode\u003enomad\u003c/code\u003e and \u003ccode\u003econsul\u003c/code\u003e (more about this in \u003ca\n  href=\"https://papers.freebsd.org/2020/fosdem/pizzamig-orchestrating_jails_with_nomad_and_pot/\"\n    target=\"_blank\"\n  \u003eOrchestrating jails with nomad and pot\u003c/a\u003e).\u003c/p\u003e","title":"Throw-Away Browser on FreeBSD With \"pot\" Within 5 Minutes","type":"blog"},{"content":"Impressum \u0026 Datenschutz (GDPR) - Germany/DeutschlandAngaben gemäß § 5 TMG\nHoneyguide GmbH\nMaulberger Weg 14\n84137 Vilsbiburg Vertreten durch: Stephan Lichtenauer\nKontakt: E-Mail: i@honeyguide.eu\nRegistereintrag: Eintragung im Registergericht: Landshut\nRegisternummer: HRB 10 642\nUmsatzsteuer-ID: Umsatzsteuer-Identifikationsnummer gemäß §27a Umsatzsteuergesetz: DE238391141\nVerantwortlich für den Inhalt nach § 55 Abs. 2 RStV:\nStephan Lichtenauer Haftungsausschluss: Haftung für Inhalte\nDie Inhalte unserer Seiten wurden mit größter Sorgfalt erstellt. Für die Richtigkeit, Vollständigkeit und Aktualität der Inhalte können wir jedoch keine Gewähr übernehmen. Als Diensteanbieter sind wir gemäß § 7 Abs.1 TMG für eigene Inhalte auf diesen Seiten nach den allgemeinen Gesetzen verantwortlich. Nach §§ 8 bis 10 TMG sind wir als Diensteanbieter jedoch nicht verpflichtet, übermittelte oder gespeicherte fremde Informationen zu überwachen oder nach Umständen zu forschen, die auf eine rechtswidrige Tätigkeit hinweisen. Verpflichtungen zur Entfernung oder Sperrung der Nutzung von Informationen nach den allgemeinen Gesetzen bleiben hiervon unberührt. Eine diesbezügliche Haftung ist jedoch erst ab dem Zeitpunkt der Kenntnis einer konkreten Rechtsverletzung möglich. Bei Bekanntwerden von entsprechenden Rechtsverletzungen werden wir diese Inhalte umgehend entfernen.\nHaftung für Links\nUnser Angebot enthält Links zu externen Webseiten Dritter, auf deren Inhalte wir keinen Einfluss haben. Deshalb können wir für diese fremden Inhalte auch keine Gewähr übernehmen. Für die Inhalte der verlinkten Seiten ist stets der jeweilige Anbieter oder Betreiber der Seiten verantwortlich. Die verlinkten Seiten wurden zum Zeitpunkt der Verlinkung auf mögliche Rechtsverstöße überprüft. Rechtswidrige Inhalte waren zum Zeitpunkt der Verlinkung nicht erkennbar. Eine permanente inhaltliche Kontrolle der verlinkten Seiten ist jedoch ohne konkrete Anhaltspunkte einer Rechtsverletzung nicht zumutbar. Bei Bekanntwerden von Rechtsverletzungen werden wir derartige Links umgehend entfernen.\nUrheberrecht\nDie durch die Seitenbetreiber erstellten Inhalte und Werke auf diesen Seiten unterliegen dem deutschen Urheberrecht. Die Vervielfältigung, Bearbeitung, Verbreitung und jede Art der Verwertung außerhalb der Grenzen des Urheberrechtes bedürfen der schriftlichen Zustimmung des jeweiligen Autors bzw. Erstellers. Downloads und Kopien dieser Seite sind nur für den privaten, nicht kommerziellen Gebrauch gestattet. Soweit die Inhalte auf dieser Seite nicht vom Betreiber erstellt wurden, werden die Urheberrechte Dritter beachtet. Insbesondere werden Inhalte Dritter als solche gekennzeichnet. Sollten Sie trotzdem auf eine Urheberrechtsverletzung aufmerksam werden, bitten wir um einen entsprechenden Hinweis. Bei Bekanntwerden von Rechtsverletzungen werden wir derartige Inhalte umgehend entfernen.\nDatenschutz\nDie Nutzung unserer Webseite ist in der Regel ohne Angabe personenbezogener Daten möglich. Soweit auf unseren Seiten personenbezogene Daten (beispielsweise Name, Anschrift oder eMail-Adressen) erhoben werden, erfolgt dies, soweit möglich, stets auf freiwilliger Basis. Diese Daten werden ohne Ihre ausdrückliche Zustimmung nicht an Dritte weitergegeben. Wir weisen darauf hin, dass die Datenübertragung im Internet (z.B. bei der Kommunikation per E-Mail) Sicherheitslücken aufweisen kann. Ein lückenloser Schutz der Daten vor dem Zugriff durch Dritte ist nicht möglich. Der Nutzung von im Rahmen der Impressumspflicht veröffentlichten Kontaktdaten durch Dritte zur Übersendung von nicht ausdrücklich angeforderter Werbung und Informationsmaterialien wird hiermit ausdrücklich widersprochen. Die Betreiber der Seiten behalten sich ausdrücklich rechtliche Schritte im Falle der unverlangten Zusendung von Werbeinformationen, etwa durch Spam-Mails, vor.\nPAIA Manual - Republic of South Africa Download PAIA Manual (PDF) ","date":"June 7, 2020","externalUrl":null,"permalink":"/disclaimer/","section":"","summary":"\u003ch2\u003eImpressum \u0026 Datenschutz (GDPR) - Germany/Deutschland\u003c/h2\u003e\u003cp\u003eAngaben gemäß § 5 TMG\u003c/p\u003e\u003cp\u003eHoneyguide GmbH\u003cbr\u003e\nMaulberger Weg 14\u003cbr\u003e\n84137 Vilsbiburg \u003cbr\u003e\n\u003c/p\u003e\u003cp\u003e \u003cstrong\u003eVertreten durch: \u003c/strong\u003e\u003cbr\u003e\nStephan Lichtenauer\u003cbr\u003e\n\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eKontakt:\u003c/strong\u003e \u003cbr\u003e\nE-Mail: \u003ca href='mailto:i@honeyguide.eu'\u003ei@honeyguide.eu\u003c/a\u003e\u003c/br\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eRegistereintrag: \u003c/strong\u003e\u003cbr\u003e\nEintragung im Registergericht: Landshut\u003cbr\u003e\nRegisternummer: HRB 10 642\u003cbr\u003e\u003c/p\u003e","title":"Disclaimer (Impressum, PAIA, GDPR/Datenschutz)","type":"page"},{"content":"","date":"June 5, 2020","externalUrl":null,"permalink":"/tags/compat/","section":"Tags","summary":"","title":"Compat","type":"tags"},{"content":"The FreeBSD Linux compatibility layer linux_base-c7 comes from CentOS 7.\nWhen trying to run a Linux application, you might run into the following error:\n$ ./linuxexecutable loolwsd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20\u0026#39; not found (required by loolwsd) loolwsd: /lib64/libstdc++.so.6: version `CXXABI_1.3.9\u0026#39; not found (required by loolwsd) loolwsd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21\u0026#39; not found (required by loolwsd) There is an easy way to find out the API versions available on your system:\n$ cd /compat/linux/usr/lib64 $ strings libstdc++.so.6 | grep LIBC GLIBCXX_3.4 GLIBCXX_3.4.1 GLIBCXX_3.4.2 GLIBCXX_3.4.3 GLIBCXX_3.4.4 GLIBCXX_3.4.5 GLIBCXX_3.4.6 GLIBCXX_3.4.7 GLIBCXX_3.4.8 GLIBCXX_3.4.9 GLIBCXX_3.4.10 GLIBCXX_3.4.11 GLIBCXX_3.4.12 GLIBCXX_3.4.13 GLIBCXX_3.4.14 GLIBCXX_3.4.15 GLIBCXX_3.4.16 GLIBCXX_3.4.17 GLIBCXX_3.4.18 GLIBCXX_3.4.19 GLIBC_2.3 GLIBC_2.2.5 GLIBC_2.14 GLIBC_2.4 GLIBC_2.3.2 GLIBCXX_DEBUG_MESSAGE_LENGTH You see GLIBCXX_3.4.19 is the latest API version available on this system.\n","date":"June 5, 2020","externalUrl":null,"permalink":"/blog/2020-06-05/","section":"Tech Blog","summary":"\u003cp\u003eThe FreeBSD Linux compatibility layer \u003ccode\u003elinux_base-c7\u003c/code\u003e comes from CentOS 7.\u003c/p\u003e\n\u003cp\u003eWhen trying to run a Linux application, you might run into the following error:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e$ ./linuxexecutable\nloolwsd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20\u0026#39; not found (required by loolwsd)\nloolwsd: /lib64/libstdc++.so.6: version `CXXABI_1.3.9\u0026#39; not found (required by loolwsd)\nloolwsd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21\u0026#39; not found (required by loolwsd)\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThere is an easy way to find out the API versions available on your system:\u003c/p\u003e","title":"FreeBSD Linux Compat GLIBCXX \u0026 GLIBC API Versions","type":"blog"},{"content":"","date":"June 5, 2020","externalUrl":null,"permalink":"/tags/linux_base/","section":"Tags","summary":"","title":"Linux_base","type":"tags"},{"content":" Overview # We are an impact enterprise and digital advisory that has been operating in Europe and Southern Africa since 2004.\nWe are socially and ethically driven social scientists, enterprise architects, engineers and researchers working as one team to solve the difficult challenges that we and our clients believe need to be overcome.\nAdvisory Services # Our service portfolio primarily consists of advisory services covering all aspects of Digital Strategy, Architecture \u0026amp; Business Processes. We mainly work with large corporations and public sector organisations, but we also work with smaller companies where we believe our open-source focus can add value.\nThe Honeyguide® team has extensive experience in driving digitalisation forward in medium and large companies, dating back to before the company\u0026rsquo;s foundation in 2004.\nOur expertise covers business and enterprise architecture, programme scoping and management and the design and development of digital processes and applications. We also specialise in streamlining ITIL- and DevOps-based end-to-end IT operations and in the agilisation of organisations — not as an airy-fairy ideology, but as added value where appropriate.\nWhile we have gained insights into the technologies offered by the global IT companies over the years and are familiar with many best practices for implementing them, we are independent and work in the best interests of our clients. Whenever it makes sense, we prefer open-source technology over expensive, run-of-the-mill solutions that lock you in.\nThis is not least because we firmly believe that we must take all aspects of digital and data sovereignty into account in our work.\nFor more details, references or enquiries, please feel free to contact us.\nProprietary Multi Asset Management # There is one more significant advantage for our clients: We are not just a small consultancy selling cloud-based solutions to clients from 9 to 5. We are walking the talk, using the solutions we recommend to manage our business-critical, in-house, multi-asset approach.\n","date":"June 4, 2020","externalUrl":null,"permalink":"/about-hg/","section":"","summary":"\u003ch2 class=\"relative group\"\u003eOverview \n    \u003cdiv id=\"overview\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#overview\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eWe are an impact enterprise and digital advisory that has been operating in Europe and Southern Africa since 2004.\u003c/p\u003e","title":"Who We Are","type":"page"},{"content":"Email: i@honeyguide.eu\nDeltaChat: Honeyguide Profile\nMastodon: @honeyguide@mastodon.africa\nHoneyguide GmbH # Maulberger Weg 14\n84137 Vilsbiburg\nGermany\nHoneyguide Group (Pty) Ltd # 1 8th Avenue\nIntercare Building, First Floor\nSummerstrand\nGqeberha (Port Elizabeth)\n6001\nSouth Africa\n","date":"June 4, 2020","externalUrl":null,"permalink":"/contact/","section":"","summary":"\u003cp\u003eEmail: \u003ca\n  href=\"mailto:i@honeyguide.eu\"\u003ei@honeyguide.eu\u003c/a\u003e\u003cbr\u003e\nDeltaChat: \u003ca\n  href=\"https://i.delta.chat/#E9798E50B30B12AD09EAAB7E5BE7CC5B4B87BE2C\u0026amp;a=9z0y4o1wn%40nine.testrun.org\u0026amp;n=Honeyguide\u0026amp;i=h8Sy7u1tvrr4c6VdLOXDkFTu\u0026amp;s=kgV-vvKJvdYvGgrJO67aF5wE\"\n    target=\"_blank\"\n  \u003eHoneyguide Profile\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eMastodon: \u003ca\n  href=\"https://mastodon.africa/@honeyguide\"\n    target=\"_blank\"\n  \u003e@honeyguide@mastodon.africa\u003c/a\u003e\u003c/p\u003e\n\n\u003ch3 class=\"relative group\"\u003e\u003cstrong\u003eHoneyguide GmbH\u003c/strong\u003e \n    \u003cdiv id=\"honeyguide-gmbh\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#honeyguide-gmbh\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003eMaulberger Weg 14\u003cbr\u003e\n84137 Vilsbiburg\u003cbr\u003e\nGermany\u003c/p\u003e\n\n\u003ch3 class=\"relative group\"\u003e\u003cstrong\u003eHoneyguide Group (Pty) Ltd\u003c/strong\u003e \n    \u003cdiv id=\"honeyguide-group-pty-ltd\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#honeyguide-group-pty-ltd\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003e1 8th Avenue\u003cbr\u003e\nIntercare Building, First Floor\u003cbr\u003e\nSummerstrand\u003cbr\u003e\nGqeberha (Port Elizabeth)\u003cbr\u003e\n6001\u003cbr\u003e\nSouth Africa\u003c/p\u003e","title":"Contact Us","type":"page"},{"content":" Introduction # To complement our Jitsi installation, we add a grafana dashboard to it so we can control usage, system load, traffic spikes etc.\nFor the sake of simplicity, we run our grafana, telegraf and influxdb instances in the same jail as jitsi-meet, so this post builds on top of the post explaining how to set up jitsi-meet.\nInitial Set Up # We connect to the jail and install the packages:\n$ iocage console jitsi ... $ pkg install grafana6 ... $ pkg install influxdb ... $ pkg install telegraf Starting Services # To make sure everything is started automatically, add the services to /etc/rc.conf:\n... grafana_enable=\u0026#34;YES\u0026#34; influxd_enable=\u0026#34;YES\u0026#34; telegraf_enable=\u0026#34;YES\u0026#34; ... Then start all services manually for the first time:\n$ /usr/local/etc/rc.d grafana start ... $ /usr/local/etc/rc.d/influxd start ... $ /usr/local/etc/rc.d/telegraf start Setting up influxdb # Start the influxdb console:\n$ influx There create the user and database:\nCREATE USER admin WITH PASSWORD \u0026#39;MYPASSWORD\u0026#39; WITH ALL PRIVILEGES CREATE DATABASE telegraf Setting up jitsi-videobridge # If you have not yet done this, configure jitsi-videobridge to collect data. Also make sure that it is started correctly with jisti_videobridge_flags=\u0026quot;--apis=rest,xmpp\u0026quot; in /etc/rc.conf so the necessary APIs are provided.\n# The videobridge uses 443 by default with 4443 as a fallback, but since we\u0026#39;re already # running nginx on 443 in this example doc, we specify 4443 manually to avoid a race condition org.jitsi.videobridge.TCP_HARVESTER_PORT=4443 org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=10.10.0.1 org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=197.155.21.68 # 197.155.21.68 is the address assigned to jitsi.honeyguide.net # # For Grafana dashboard # # the callstats credentials io.callstats.sdk.CallStats.appId=\u0026#34;hgstats\u0026#34; io.callstats.sdk.CallStats.keyId=\u0026#34;GRAFANAKEY\u0026#34; ##io.callstats.sdk.CallStats.keyPath= ##io.callstats.sdk.CallStats.appSecret= # the id of the videobridge #io.callstats.sdk.CallStats.bridgeId=jitsi-videobridge.jitsi.honeyguide.net #io.callstats.sdk.CallStats.conferenceIDPrefix=conference.jitsi.honeyguide.net # enable statistics and callstats statistics and the report interval org.jitsi.videobridge.ENABLE_STATISTICS=true org.jitsi.videobridge.STATISTICS_INTERVAL.callstats.io=30000 org.jitsi.videobridge.STATISTICS_TRANSPORT=callstats.io Setting up telegraf # Add the data source created above to /usr/local/etc/telegraf.conf:\n... [[inputs.http]] name_override = \u0026#34;jitsi_stats\u0026#34; urls = [ \u0026#34;http://localhost:8080/colibri/stats\u0026#34; ] data_format = \u0026#34;json\u0026#34; ... Setting up grafana # Initial Connect and Data Source Creation # Connect to your grafana instance the first time (it is listening at http://10.10.0.1:3000/ if your jail has 10.10.0.1 as IP address) with user admin and empty password and set the initial password.\nAfter that, click on the configuration \u0026ldquo;gear\u0026rdquo; icon and choose \u0026ldquo;Data Sources\u0026rdquo;. Click \u0026ldquo;Add Data Source\u0026rdquo; and select \u0026ldquo;Influx DB\u0026rdquo;. Use the following settings (and of course enter the password you have used when creating the database as described above): Import of Jitsi Dashboard # Then click on \u0026ldquo;+\u0026rdquo; in the left menu and choose \u0026ldquo;Import\u0026rdquo; from the menu.\nYou will find the dashboard to import at https://grafana.com/grafana/dashboards/11969/reviews. There you will see that the preconfigured dashboard has the number 11969, so you can simply enter this number in the ID field and click on \u0026ldquo;Import\u0026rdquo;, after that select the newly created InfluxDB as data source from the drop down and create the dashboard.\nThat\u0026rsquo;s it!\n","date":"May 9, 2020","externalUrl":null,"permalink":"/blog/2020-05-09/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eTo complement our \u003ca\n  href=\"/blog/2020-05-08/\"\u003eJitsi installation\u003c/a\u003e, we add a \u003ccode\u003egrafana\u003c/code\u003e dashboard to it so we can control usage, system load, traffic spikes etc.\u003c/p\u003e","title":"Grafana Dashboard for Jitsi-Meet","type":"blog"},{"content":"","date":"May 9, 2020","externalUrl":null,"permalink":"/tags/influxdb/","section":"Tags","summary":"","title":"Influxdb","type":"tags"},{"content":"","date":"May 9, 2020","externalUrl":null,"permalink":"/tags/jitsi/","section":"Tags","summary":"","title":"Jitsi","type":"tags"},{"content":"","date":"May 9, 2020","externalUrl":null,"permalink":"/tags/jitsi-meet/","section":"Tags","summary":"","title":"Jitsi-Meet","type":"tags"},{"content":"","date":"May 9, 2020","externalUrl":null,"permalink":"/tags/telegraf/","section":"Tags","summary":"","title":"Telegraf","type":"tags"},{"content":" Introduction # Due to the situation with COVID-19 that also lead to people being confined to their homes in South Africa as well, we decided to provide a (freely usable of course) Jitsi Meet instance to the community being hosted in South Africa on our FreeBSD environment.\nThat way, communities in South Africa and beyond have a free alternative to the commercial conferencing solutions with sometimes dubious security and privacy histories and at the same time improved user experience due to the lower latency of local hosting.\nOur instance is available at jitsi.honeyguide.net for those wanting to try it out.\nThis tutorial will show you how to set up your own Jitsi Meet from scratch on FreeBSD.\nInitial Set Up # We first of all initialise the jail which we will use (we use iocage for jail management):\n$ iocage create -n jitsi ip4_addr=\u0026#34;igb0|10.10.0.1/24\u0026#34; -r 11.3-RELEASE boot=\u0026#34;on\u0026#34; Then we connect to the jail and prepare pkg and ports (some ports are so new that we need to build them ourselves):\n$ iocage console jitsi ... $ pkg ... $ portsnap fetch ... $ portsnap extract ... Installing All Packages and Ports # jitsi-meet and jitsi-videobridge are built from ports:\n$ cd /usr/ports/www/jitsi-meet $ make install clean ... $ cd /usr/ports/net-im/jitsi-videobridge $ make install clean ... nginx, acme.sh for SSL certificate management and prosody can be installed from packages:\n$ pkg install nginx ... $ pkg install prosody ... $ pkg install acme.sh If you run into problems with your setup, we recommend you compare your configuration with https://github.com/jitsi/jitsi-meet/blob/master/doc/manual-install.md.\nSetting Up prosody # The prosody configuration is located in /usr/local/etc/prosody/prosody.cfg.lua.\nThe log files are located in /var/db/prosody.\nFirst of all, we need to create and register the certificates:\n$ prosodyctl cert generate jitsi.honeyguide.net ... $ prosodyctl cert generate auth.jitsi.honeyguide.net ... $ prosodyctl register focus auth.jitsi.honeyguide.net KEYUSEDINCONFIG Replace KEYUSEDINCONFIG with a key that you will need to use in the config files.\nIn /usr/local/etc/prosody/prosody.cfg.lua, we added the following lines at the end (otherwise, the default configuration works fine):\nVirtualHost \u0026#34;jitsi.honeyguide.net\u0026#34; authentication = \u0026#34;anonymous\u0026#34; ssl = { key = \u0026#34;/var/db/prosody/jitsi.honeyguide.net.key\u0026#34;; certificate = \u0026#34;/var/db/prosody/jitsi.honeyguide.net.crt\u0026#34;; } modules_enabled = { \u0026#34;bosh\u0026#34;; \u0026#34;pubsub\u0026#34;; } c2s_require_encryption = false VirtualHost \u0026#34;auth.jitsi.honeyguide.net\u0026#34; ssl = { key = \u0026#34;/var/db/prosody/auth.jitsi.honeyguide.net.key\u0026#34;; certificate = \u0026#34;/var/db/prosody/auth.jitsi.honeyguide.net.crt\u0026#34;; } authentication = \u0026#34;internal_plain\u0026#34; admins = { \u0026#34;focus@auth.jitsi.honeyguide.net\u0026#34; } Component \u0026#34;conference.jitsi.honeyguide.net\u0026#34; \u0026#34;muc\u0026#34; Component \u0026#34;jitsi-videobridge.jitsi.honeyguide.net\u0026#34; component_secret = \u0026#34;KEYUSEDINCONFIG\u0026#34; Component \u0026#34;focus.jitsi.honeyguide.net\u0026#34; component_secret = \u0026#34;KEYUSEDINCONFIG\u0026#34; Setting up jitsi-videobridge # The jitsi-videobridge configuration is located in /usr/local/etc/jitsi/videobridge/jitsi-videobridge.conf and /usr/local/etc/jitsi/videobridge/sip-communicator.properties.\nThere is one minor problem though: /usr/local/etc/jitsi/videobridge/jitsi-videobridge.conf is currently ignored, the /usr/local/etc/rc.d/jitsi-videobridge startup script does not read the environment file correctly.\nBeing pragmatic, we adjusted the startup script (and saved the original file in jitsi-videobridge.orig). Since also the additional flags from /etc/rc.conf are ignored, we also added \u0026ndash;apis=rest,xmpp for the telegraf set up (for the grafana dashboard, see our separate blog post) there:\n... jitsi_videobridge_start() { daemon -p ${pidfile} -o /var/log/jitsi-videobridge.log ${command} -Xmx3072m \\ -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError \\ -XX:HeapDumpPath=/tmp \\ -Djava.util.logging.config.file=${jitsi_videobridge_logging_config} \\ -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/usr/local/etc/jitsi \\ -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=videobridge \\ -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/ \\ -cp ${jitsi_videobridge_jar} \\ org.jitsi.videobridge.Main \\ --host=localhost \\ --domain=jitsi.honeyguide.net \\ --port=5347 \\ --secret=KEYUSEDINCONFIG --apis=rest,xmpp ${jitsi_videobridge_flags} echo \u0026#34;Started\u0026#34; } ... Please note that if you want to use the restart command for the service, you also need to adjust the jitsi_videobridge_restart() function similarly.\nAs soon as reading from the config file is fixed, our config file will look like this:\nJVB_XMPP_HOST=localhost JVB_XMPP_DOMAIN=jitsi.honeyguide.net JVB_XMPP_PORT=5347 JVB_XMPP_SECRET=KEYUSEDINCONFIG VIDEOBRIDGE_MAX_MEMORY=3072m # VIDEOBRIDGE_DEBUG_OPTIONS=\u0026#34;-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000\u0026#34; We also need to adjust /usr/local/etc/jitsi/videobridge/sip-communicator.properties (this file also already is prepared for the grafana dashboard, but you need to adjust the IP address in any case if your jail does not use the public IP address because e.g. you have 1:1 NAT):\n# The videobridge uses 443 by default with 4443 as a fallback, but since we\u0026#39;re already # running nginx on 443 in this example doc, we specify 4443 manually to avoid a race condition org.jitsi.videobridge.TCP_HARVESTER_PORT=4443 org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=10.10.0.1 org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=197.155.21.68 # 197.155.21.68 is the address assigned to jitsi.honeyguide.net # # For Grafana dashboard # # the callstats credentials io.callstats.sdk.CallStats.appId=\u0026#34;hgstats\u0026#34; io.callstats.sdk.CallStats.keyId=\u0026#34;GRAFANAKEY\u0026#34; ##io.callstats.sdk.CallStats.keyPath= ##io.callstats.sdk.CallStats.appSecret= # the id of the videobridge #io.callstats.sdk.CallStats.bridgeId=jitsi-videobridge.jitsi.honeyguide.net #io.callstats.sdk.CallStats.conferenceIDPrefix=conference.jitsi.honeyguide.net # enable statistics and callstats statistics and the report interval org.jitsi.videobridge.ENABLE_STATISTICS=true org.jitsi.videobridge.STATISTICS_INTERVAL.callstats.io=30000 org.jitsi.videobridge.STATISTICS_TRANSPORT=callstats.io Setting up jicofo # The jicofo startup script /usr/local/etc/rc.d/jicofo expects a /usr/local/etc/ssl/java.pem, so we create it:\n$ keytool -noprompt -keystore /usr/local/etc/ssl/java.pem -importcert -alias prosody -file /var/db/prosody/auth.jitsi.honeyguide.net.crt Remember your keystore password for later on.\nThe jicofo config file is in /usr/local/etc/jitsi/jicofo/jicofo.conf but it is currently ignored as well.\nSo we also adjusted /usr/local/etc/rc.d/jicofo and saved the original file as jicofo.orig:\n... jicofo_start() { daemon -p ${pidfile} -o /var/log/${name}.log ${command} -Xmx3072m \\ -XX:+HeapDumpOnOutOfMemoryError \\ -XX:HeapDumpPath=/tmp \\ -Djava.util.logging.config.file=${jicofo_logging_config} \\ -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/usr/local/etc/jitsi \\ -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=jicofo \\ -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/ \\ -Djavax.net.ssl.trustStore=/usr/local/etc/ssl/java.pem \\ -cp ${jicofo_jar} \\ org.jitsi.jicofo.Main \\ --host=localhost \\ --domain=jitsi.honeyguide.net \\ --port=5347 \\ --secret=3r8wer8DH \\ --user_domain=auth.jitsi.honeyguide.net \\ --user_name=focus \\ --user_password=KEYUSEDINCONFIG ${jicofo_flags} echo \u0026#34;Started\u0026#34; } ... Here, the restart command works as it has been implemented more elegantly in the startup script already.\nFor later on, when the startup script reads the config file correctly, here is our /usr/local/etc/jitsi/jicofo/jicofo.conf:\nJVB_XMPP_HOST=localhost JVB_XMPP_DOMAIN=jitsi.honeyguide.net JVB_XMPP_PORT=5347 JVB_XMPP_SECRET=KEYUSEDINCONFIG JVB_XMPP_USER_DOMAIN=auth.jitsi.honeyguide.net JVB_XMPP_USER_NAME=focus JVB_XMPP_USER_SECRET=KEYUSEDINCONFIG MAX_MEMORY=3072m Setting up nginx # The nginx configuration is in /usr/local/etc/nginx/nginx.conf. Of course it might be done differently, but we set up everything in this one file as it is not complicated.\nWe only need to add two server entries:\n... server { listen 80 default_server; server_name _; return 301 https://$host$request_uri; } server { listen 0.0.0.0:443 ssl http2; ssl_certificate /usr/local/etc/ssl/jitsi.honeyguide.net.cer; ssl_certificate_key /usr/local/etc/ssl/jitsi.honeyguide.net.key; ssl_session_cache shared:SSL:1m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; server_name jitsi.honeyguide.net; # set the root root /usr/local/www/jitsi-meet; index index.html; location ~ ^/([a-zA-Z0-9=\\?]+)$ { rewrite ^/(.*)$ / break; } location / { ssi on; } # BOSH, Bidirectional-streams Over Synchronous HTTP # https://en.wikipedia.org/wiki/BOSH_(protocol) location /http-bind { proxy_pass http://localhost:5280/http-bind; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host $http_host; } # external_api.js must be accessible from the root of the # installation for the electron version of Jitsi Meet to work # https://github.com/jitsi/jitsi-meet-electron location /external_api.js { alias /srv/jitsi-meet/libs/external_api.min.js; } } ... An easy way to maintain Let\u0026rsquo;s Encrypt SSL certificates is acme.sh stand alone mode:\n$ /usr/local/etc/rc.d/nginx stop ... $ cd $ acme.sh --force --issue -d jitsi.honeyguide.net --standalone ... $ cd /root/.acme.sh/jitsi.honeyguide.net/ $ mv * /usr/local/etc/ssl/ $ /usr/local/etc/rc.d/nginx start Setting up jitsi-meet # The jitsi-meet configuration is located in /usr/local/www/jitsi-meet/config.js.\nMost of the values there can remain as they are (though you might want to customise them depending on your needs), but you need to change the first lines of the file to reflect your domain:\nvar config = { // Connection // hosts: { // XMPP domain. domain: \u0026#39;jitsi.honeyguide.net\u0026#39;, // When using authentication, domain for guest users. // anonymousdomain: \u0026#39;guest.example.com\u0026#39;, // Domain for authenticated users. Defaults to \u0026lt;domain\u0026gt;. // authdomain: \u0026#39;jitsi-meet.example.com\u0026#39;, // Jirecon recording component domain. // jirecon: \u0026#39;jirecon.jitsi-meet.example.com\u0026#39;, // Call control component (Jigasi). // call_control: \u0026#39;callcontrol.jitsi-meet.example.com\u0026#39;, bridge: \u0026#39;jitsi-videobridge.jitsi.honeyguide.net\u0026#39;, // Focus component domain. Defaults to focus.\u0026lt;domain\u0026gt;. focus: \u0026#39;focus.jitsi.honeyguide.net\u0026#39;, // XMPP MUC domain. FIXME: use XEP-0030 to discover it. muc: \u0026#39;conference.jitsi.honeyguide.net\u0026#39; }, ... If you want to adjust the frontend and look and feel, look at the content in the directories static, images and at interface_config.js.\nFinishing Up # To make sure everything is started automatically, add the services to /etc/rc.conf:\n... jitsi_videobridge_enable=\u0026#34;YES\u0026#34; jisti_videobridge_flags=\u0026#34;--apis=rest,xmpp\u0026#34; nginx_enable=\u0026#34;YES\u0026#34; prosody_enable=\u0026#34;YES\u0026#34; jicofo_enable=\u0026#34;YES\u0026#34; ... ","date":"May 8, 2020","externalUrl":null,"permalink":"/blog/2020-05-08/","section":"Tech Blog","summary":"\u003ch2 class=\"relative group\"\u003eIntroduction \n    \u003cdiv id=\"introduction\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700 !no-underline\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eDue to the situation with COVID-19 that also lead to people being confined to their homes in South Africa as well, we decided to provide a (freely usable of course) \u003ca\n  href=\"https://github.com/jitsi/jitsi-meet\"\n    target=\"_blank\"\n  \u003eJitsi Meet\u003c/a\u003e instance to the community being hosted in South Africa on our \u003ca\n  href=\"https://www.freebsd.org\"\n    target=\"_blank\"\n  \u003eFreeBSD\u003c/a\u003e environment.\u003c/p\u003e","title":"Running Jitsi-Meet in a FreeBSD Jail","type":"blog"},{"content":"Catch up on all our technology knowledge sharing right here.\n","date":"May 8, 2020","externalUrl":null,"permalink":"/blog/","section":"Tech Blog","summary":"\u003cp\u003eCatch up on all our technology knowledge sharing right here.\u003c/p\u003e","title":"Tech Blog","type":"blog"},{"content":"","date":"May 8, 2020","externalUrl":null,"permalink":"/tags/videoconference/","section":"Tags","summary":"","title":"Videoconference","type":"tags"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","externalUrl":null,"permalink":"/micro/","section":"Microes","summary":"","title":"Microes","type":"micro"}]