Recently I’ve migrated from TrueNAS Core (formerly known as FreeNAS, FreeBSD-based) to TrueNAS Scale (Debian-based). Thus, I had to move web apps like Nextcloud, Plex, etc. from FreeBSD Jails to Kubernetes Pods manually. I would like to briefly introduce these different two virtualization techniques here below.

---

2023.12.09 Update:

The configuration of TrueCharts and reverse proxy have changed a lot since I wrote this tutorial. I have also had lots of problems with TrueCharts because of frequent changes of app setting / trains. For example, people cannot simply upgrade their TrueCharts apps after they changed the validation [1,2]. Due to the frustration, I have migrated all apps (Nextcloud, Jellyfin, AdGuardHome, code-server, some game servers…) to docker containers I simply created. For reverse proxy, I am using nginx-proxy-manager instead of traefik now. The best practice is either mount your Nextcloud config/data into host (don’t use PVC to store them) or using SMB/NFS mounting functions in Nextcloud (without messing up with permission).

As a result, I removed all the TrueCharts setup and replaced it with tutorial for official docker images.

Join my discord server for discussion / troubleshooting!

---

FreeBSD Jail

First things first — what is FreeBSD? FreeBSD is an operating system which has a bunch of advanced features. One such feature is soft update for the UFS filesystem. Another such feature is called Jail, which is the front runner to the containers we use today. A Jail enables process and its children processes to run in a sandbox, i.e., they have separate file system, hostname, IP address, etc. In other words, jailed process cannot interact with processes in another jail, and each jail has its own superuser.

So… how did they implement Jail? They patched FreeBSD kernel to be aware of if it is inside jail. For example, there is a statement if(superuser) then do (something). After being modified, it should be if(superuser and in jail) then do (something) or more typically if(superuser and not in jail) then do (something).

The features of FreeBSD Jail make it a great candidate for hosting web applications. For example, we have a WordPress service running in one jail with its IP 192.168.0.100, and a Plex service running in another jail with IP 192.168.0.101. They cannot interact with each other and if one crashes, the other service is unaffected.

You would say we can run the applications in virtual machines (bhyve in FreeBSD) but don’t forget such VMs have its own OS layer and take much more resources.

Kubernetes (K8s)

Actually, Kubernetes is not a type of virtualization method, it is a container manager. Wait, what is container? It is more or less like FreeBSD Jail which runs a group of processes in an isolated environment by leveraging kernel features. The most popular one is Docker container. With kernel namespaces, every container has its own file system, users and network stack, and resources can be limited by cgroups (which FreeBSD Jail cannot provide with). We create Pods (a group of one or more containers) via Kubernetes just like we create FreeBSD Jail via iocage. For example, a WordPress pod may have three containers: one for apache web server, another for sql database, and the other for redis. Kubernetes provides some advanced features like scaling and load balancing, but here in TrueNAS Scale we just use its basic functions.

0. Prerequisites

In order to host Nextcloud with your custom domain name, you must have:

1. TrueNAS Scale
2. Domain
3. Cloudflare account (or other supported DNS providers)
4. Public IP
5. Router with port forwarding and NAT hairpin enabled

1. Setup App environment

If you haven’t setup any app before, you are required to assign a pool for storing app. It might take a while to fetch app catalogs from the Internet.

Since we will use nginx-proxy-manager and it’s under TrueNAS Official community train, head to “manage catalogs” tab and click three dots icon on Official to edit it.

After all the tasks are finished, head to “available applications”, you should find nginx-proxy-manager. If not, click “refresh all” button above.

2. Install and configure nginx-proxy-manager reverse proxy

Reverse proxy service routes network traffic from different hostnames to belonged apps. To install this, just head to “available applications” tab and search for nginx-proxy-manager. Click install to proceed.

You can leave port numbers as default. For the storage configuration, if you want to save your certificate and login information even after this app is removed, you can use “Host Path” as shown above. Otherwise, default type (ixVolume) is fine.

Click “next” to the end, and save the setting to install it! After that, you should see nginx-proxy-manager in “installed applications’ tab with clickable “Web Portal” button.

3. Setup ACME certificates with Cloudflare account

Here we use DNS challenge to get our certificate.

3.1. Fetch Cloudflare API

Let’s suppose you are using Cloudflare to manage your domain name. We will need a API Token (not global key), which can be created in settings -> API Tokens tab.

You can just use “Edit zone DNS” template, and for the Zone Resources, select “All zones”

After that, you’ll get a API token, please copy it to somewhere else as it will not be shown again.

3.2. Setup ACME DNS challenges

After login to Nginx Proxy Manager web UI, head to “SSL Certificates” tab and click on “Add SSL Certificate”, “Let’s Encrypt” button.

For the domain names, if you’re going to just host nextcloud, you can enter a specific domain name. Otherwise, you can also use wildcard like *.windsketch.cc which matches all sub-domains as shown above.

Paste your cloudflare API Token after dns_cloudflare_api_token = . After that, click save and wait for a few minutes for certificate validation. If you see a certificate entry with valid expired date, you are all set!

4. Install Nextcloud

In this section, we will install both official Nextcloud and its database (we use mariadb here), so we will create two apps.

3.1. Setup mariadb

Since the official nextcloud docker itself does not come with any database to store any information, we need a create a separate database for that. Here we’re going to use official mariadb image. Click “Launch Docker Image” in TrueNAS Apps tab:

Image repository is mariadb.

For the container environment variables, you are going to add a database and user/password:

Since it uses TCP port 3306, you will need to expose this port by doing port forwarding. For example, I’m using port 13306 as node port here, so MY.TRUENAS.IP:13306 will head to port 3306 of this app.

For the storage, I use host path volumes again here for persistent storage. Click “Save” to launch docker image.

3.2. Setup Nextcloud

Click “Launch Docker Image” in Truenas Apps tab again for creating Nextcloud app.

Image repository is nextcloud.

For the container entrypoint, since I will use SMB as external storage in Nextcloud, smbclient is required and I simply add a command to install it:

(If you’re not going to use SMB, you don’t have to add anything here.)

For environment variables, you will need to add database information:

For MYSQL_HOST shown below, the format of value is IP:PORT. For example if the local IP of your TrueNAS is 192.168.1.254 and “node port” for mariadb (in pervious section) is 13306, then the value is 192.168.1.254:13306 here.

NEXTCLOUD_TRUSTED_DOMAINS is the domain you’re going to use for Nextcloud (which matches the one setup in nginx-proxy-manager). NEXTCLOUD_TRUSTED_PROXIES is you TrueNAS local IP again.

We only want to accept https connections, so OVERWRITEPROTOCOL is set as https here.

For port forwarding, add one with container port 80. I use 18080 as node port here:

For the storage, I use host path volumes again here for persistent storage for potential upgrade/migrate. Click “Save” to launch docker image.

Head to Nginx Proxy Manager web UI to add a proxy host: In “Hosts” tab, click “Add Proxy Host” button.

Select your valid certificate in SSL tab:

5. Setup DNS and port forwarding

Finally, you should add an “A” type DNS record in Cloudflare to point your domain name to the external IP address of TrueNAS, and add port forwarding entries in your router:

Remember the default ports when you were creating nginx-proxy-manager? You will forward port 80/443 coming into WAN to port 30021/30022 in your TrueNAS.

Let’s say if you browse https://your.domain.name/, and for example your public IP is 1.2.3.4 and TrueNAS local IP is 192.168.1.254.

the IP will be resolved via Cloudflare DNS entry and then will try to establish connection to your router, which is 1.2.3.4:443. By port forwarding rules from your router, it will redirect to 192.168.1.54:30022. Next, with the Nginx Proxy Manager inspect the hostname (your.domain.name) and found a match entry in hosts, so it redirects to port 192.168.1.254:18080, which is the local IP and port for your Nextcloud.

Note that you have to enable NAT hairpin (NAT loopback) in your router. Without that, you cannot connect to Nextcloud (or other apps) with domain name when you’re under the same LAN.

If you want proxy enabled for Cloudflare DNS entry, you have to set SSL/TLS encryption mode to “full” to avoid ERR_TOO_MANY_REDIRECTS error.

From now on, access your Nextcloud via https://your.domain.name/ and enjoy 😀!