Recently I’ve migrated from TrueNAS Core (formerly known as FreeNAS, FreeBSD-based) to TrueNAS Scale (Debian-based). Thus, I had to move web apps like Nextcloud, Plex, etc. from FreeBSD Jails to Kubernetes Pods manually. I would like to briefly introduce these different two virtualization techniques here below.

FreeBSD Jail

First things first — what is FreeBSD? FreeBSD is an operating system which has a bunch of advanced features. One such feature is soft update for the UFS filesystem. Another such feature is called Jail, which is the front runner to the containers we use today. A Jail enables process and its children processes to run in a sandbox, i.e., they have separate file system, hostname, IP address, etc. In other words, jailed process cannot interact with processes in another jail, and each jail has its own superuser.

So… how did they implement Jail? They patched FreeBSD kernel to be aware of if it is inside jail. For example, there is a statement if(superuser) then do (something). After being modified, it should be if(superuser and in jail) then do (something) or more typically if(superuser and not in jail) then do (something).

The features of FreeBSD Jail make it a great candidate for hosting web applications. For example, we have a WordPress service running in one jail with its IP 192.168.0.100, and a Plex service running in another jail with IP 192.168.0.101. They cannot interact with each other and if one crashes, the other service is unaffected.

You would say we can run the applications in virtual machines (bhyve in FreeBSD) but don’t forget such VMs have its own OS layer and take much more resources.

Kubernetes (K8s)

Actually, Kubernetes is not a type of virtualization method, it is a container manager. Wait, what is container? It is more or less like FreeBSD Jail which runs a group of processes in an isolated environment by leveraging kernel features. The most popular one is Docker container. With kernel namespaces, every container has its own file system, users and network stack, and resources can be limited by cgroups (which FreeBSD Jail cannot provide with). We create Pods (a group of one or more containers) via Kubernetes just like we create FreeBSD Jail via iocage. For example, a WordPress pod may have three containers: one for apache web server, another for sql database, and the other for redis. Kubernetes provides some advanced features like scaling and load balancing, but here in TrueNAS Scale we just use its basic functions.

0. Prerequisites

In order to host Nextcloud with your custom domain name, you must have:

1. TrueNAS Scale
2. Domain
3. Cloudflare account
4. Public IP
5. Router with port forwarding and NAT hairpin enabled

1. Setup App environment

If you haven’t setup any app before, you are required to assign a pool for storing app. It might take a while to fetch app catalogs from the Internet.

Although there is an official nextcloud app in available applications tab, we don’t use this one. Instead, to use reverse proxy and integrate with ingress, we have to add TrueCharts catalog manually and use its nextcloud. Head to “manage catalogs” tab, and click “add catalog” button above.

Catalog name can be just truecharts; repository is https://github.com/truecharts/catalog; preferred trains are stable (for Nextcloud) and enterprise (for Traefik); branch remains main. Click save to proceed. Again, it takes a while to fetch and validate catalog. You can click task manager (at upper-right corner) to view tasks in progress.

2023/02/24 Update: Traefik has been moved from train core to enterprise.

After all the tasks are finished, head to “available applications”, you should see a bunch of TrueCharts apps. If not, click “refresh all” button above.

If you are running TrueNAS SCALE 22.02-RC.2, there is an existing issue which causes web interface very slow. There is a patch to resolve this (this is only for RC.2 release, and is not required for SCALE 22.02.0 and future releases):

curl -s https://truecharts.org/rc2patcher | bash

2. Install and configure Traefik reverse proxy

Traefik is a reverse proxy (actually much more than that, but we will just use this function in TrueNAS) service which routes network traffic from different hostnames to belonged apps. To install this, just head to “available applications” tab and search for traefik. Click install to proceed.

After giving this app a name (e.g., traefik), you can just click “next” buttons until step 5 “Nextworking and Services”. Here you will see three numbers: 9000, 9080, and 9443. So, 9000 the port number for Traefik its own web interface (to monitor route setting for example); 9080 is the unsecured (http://) entry point port; 9443 is secured (https://) port. If you want to redirect http to https for your apps, click “show advanced settings” right below 9080, and put 443 in “redirect to port” text field. Note that “redirect to entrypoint” should be empty.

Click “next” to the end, and save the setting to install it!

3. Setup ACME certificates with Cloudflare account

So far TrueNAS only supports Cloudflare and route53 to verify ACME DNS challenges automatically. Here we will go with Cloudflare.

3.1. Fetch Cloudflare API

Let’s suppose you are using Cloudflare to manage your domain name. If not, just migrate to it! We will need Global API Key, which can be viewed in settings -> API Tokens tab.

3.2. Setup ACME DNS challenges

You are required to have email address for root before proceeding.

All certificates have expired dates. We will setup auto renewal for certificates through DNS challenges. Head to “credentials” menu and select “certificates” tab, click “add” in “ACME DNS-Authenticators” block.

Just give it a name (e.g., your domain), select “cloudflare” in authenticator, and type-in your registered email and global API key.

Now we need to create CSR as well: click add in that block. Give it a name again and click next.

In step 3 “Certificate Subject”, be sure to put your domain name in “common name” text field, and all of domains you will use in “subject alternate names” field. Here you can use wildcard like *.windsketch.cc which matches all sub-domains. Click next to save it.

After successfully creating CSR, click the wrench icon 🔧 to create ACME certificate. Give it a name like ACME_CERT and make sure to choose “Let’s Encrypt Production Directory” in ACME Server Directory URI (otherwise certificates will be in testing mode). Select our newly created DNS authenticator below.

If you see a certificate entry shown in “certificates” block with valid expired date, you are all set!

4. Install Nextcloud

In this section, we will install TrueCharts-based nextcloud. Click “refresh all” again to let ingress know you’ve added new certificate. Search application “nextcloud”, and there are two apps. Just choose “TrueCharts” one and click install.

If you want to migrate from existing Nextcloud, you are required to setup mount points (for example, /var/www/html/config is the path for config folder inside the container for TrueCharts-based Nextcloud) and set owner and group to www-data:www-data for existing files. If you are not sure how to do this, feel free to leave a comment!

In step 6 “Ingress”, click “enable ingress”, add one host with the hostname you would like to use for nextcloud (e.g., nextcloud.windsketch.cc). Path should be / with prefix path type.

For TLS setting, put the same information again, in the dropdown list “Select TrueNAS SCALE Certificate“, be sure to select the certificate we created in the previous section.

Other settings remain defaults. Click “save” to deploy it (it will take longer than Traefik since there are three containers in this pod). If all the settings are set correctly, there will be one entry added automatically in Traefik http routes tab:

By the way, you can use above steps to install any TrueCharts app with your custom domain name! There is also an app called “external-service” which enables unsupported pods (like something you cannot find in TrueCharts catalog and have to manually create) to integrate with Ingress. Let’s say we have a ZoneMinder with address http://192.168.88.73:19080, in the setting it will be:

5. Setup DNS and port forwarding

Finally, you should add an “A” type DNS record in Cloudflare to point your domain name to the external IP address of TrueNAS, and add port forwarding entries in your router:

Let’s say if you browse http://your.domain.name/, the IP will be resolved via Cloudflare DNS entry and then will try to establish connection to your router (for example, your public IP is 1.2.3.4 and TrueNAS IP is 192.168.88.72), 1.2.3.4:80 (port 80). By port forwarding rules from your router, it will redirect to 192.168.88.72:9080. Next, with the Traefik “redirect to port” setup, port 9080 will redirect to port 443, so it is 1.2.3.4:443 now. Similarly, router redirects 1.2.3.4:443 to 192.168.88.72:9443. Now Traefik will check the domain name and redirect to specific services (for example, redirection to Nextcloud is 192.168.88.72:10020).

Note that you have to enable NAT hairpin (NAT loopback) in your router. Without that, you cannot connect to Nextcloud (or other apps) with domain name (still can browse it directly with IP and port).

If you want proxy enabled for Cloudflare DNS entry, you have to set SSL/TLS encryption mode to “full” to avoid ERR_TOO_MANY_REDIRECTS error.

From now on, access your Nextcloud via https://your.domain.name/ and enjoy 😀!

---

Update: If you tried to upgrade NextCloud and found that it won’t start anymore, make sure in settings tab of NextCloud app, “Networking and Services” → “Show Advanced settings” → “Target Port” is 8080 (instead of 80). They changed the listening port without updating it while upgrading.