m3tam3re.com/content/posts/v1-docker-n8n-setup.en.md

title, date, draft, tutorials, tags

title	date	draft	tutorials	tags
#1: Docker, Portainer and n8n Installation	2022-03-28	false	Docker Installations	docker n8n caddy portainer installation tutorial

I don't want to spend too much time on an introduction. I think there are many good videos on the topics I will discuss here. I don't want to show you how to use tools like Docker or similar here. Rather, I will show you a few specific solutions that I use in our company in the same or a modified form.

For a large part of the videos shown here, a setup of Docker, Caddy, Portainer and n8n will be necessary. I will keep the installation and configuration here to the essentials. I think there are more than enough tutorials and videos on each of these topics 😉

Setting up Subdomains & DNS

Later on, I want to call up Portainer and n8n via separate subdomains and sometimes it takes a little while until changes to domains are applied so I set up the subdomains first. I will show this here using the example of GoDaddy, where I have registered my domains.

Creating DNS Entries

You must go to the management of Your domain in the DNS settings. In the case of GoDaddy, after logging in, simply click on your name in the top right and then select My Products. Then you can manage your domain:

Now click on the menu at the domain card or below on DNS.

The DNS setting is simple. Here you only need to specify an A-Record, just the prefix for the subdomain, and the IP address of your server. You can set the TTL value as it makes sense for you, I always take 600 seconds here.

Installation of Docker

# Under Arch Linux, Manjaro etc:
sudo pacman -S docker docker-compose

# Under Debian, Ubuntu etc:
sudo apt-get update && apt-get install docker docker-compose

sudo groupadd docker # Optional: in most distributions the group will be created during installation
sudo systemctl enable --now docker

Setting Permissions and Creating Folders

I like to add my main user to the docker group, so I don't always have to enter the admin password when running Docker commands. Depending on which shell you use, the command is a little different. If you don't know which shell you are using, it will definitely be Bash:

Bash

sudo usermod -a -G docker "$(whoami)"

Or Fish

sudo usermod -a -G docker (whoami)

Now create the folder where Docker configurations etc. should be stored. I have chosen the folder /opt/docker-services here.

sudo mkdir -m 775 /opt/docker-services

And now set the permissions:

Bash

sudo chown $(whoami):docker /opt/docker-services

Fish

sudo chown (whoami):docker /opt/docker-services

Clone and Adapt Repository for Caddy, Portainer and n8

To make the installation easier, I have prepared a repository where you actually only have to adjust a few values and we can start directly:

git clone git@git.sr.ht:~m3tam3re/yt1-docker-caddy-portainer-n8n /opt/docker-services

Here you may replace the last part with the folder where you want to store your data for your Docker services. The structure should look like this:

/opt/docker-services:
├── LICENSE
├── n8n
│   ├── docker-compose.yml
│   ├── DockerFile
│   └── README.md
├── portainer-caddy
│   ├── Caddyfile
│   └── docker-compose.yml
└── README.md

Installation of Caddy and Portainer

Caddy and Portainer are the basis for our setup. We use Caddy as a reverse proxy, which is responsible for forwarding requests to the corresponding Docker container. Alternatively, you could also use Traefik or NGINX here, but compared to those, Caddy is much easier to configure.

Portainer is a web interface that makes managing docker containers very easy. Portainer is basically optional and a good alternative for anyone who doesn't like to work with the terminal.

Caddy install

The Caddy installation is relatively simple. Since my server has Arch Linux as the operating system, Caddy is installed via pacman:

sudo pacman -Syu caddy

In Ubuntu or other Debian-based distributions, the installation is a little more complicated:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo tee /etc/apt/trusted.gpg.d/caddy-stable.asc

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list

sudo apt update

sudo apt install caddy

Basically, Caddy is started automatically after installation, if not just one more time:

caddy start

If Caddy is already running, you will get an error here. You can simply ignore that.

Setup Caddy and Portainer

At the top, under the heading Setting up Subdomains & DNS, I talked about wanting to call up Portainer and n8n via separate subdomains. For this to work, we need to configure Caddy as a reverse proxy. For this, we edit the Caddyfile. In my setup, I have planned the domain ca.m3tam3re.com for Portainer and io.m3tam3re.com for n8n. The Caddyfile then looks like this:

ca.m3tam3re.com {
    reverse_proxy localhost:9000
}
io.m3tam3re.com {
    reverse_proxy localhost:5678
}

As you can see, the Caddy configuration is straightforward. Here, of course, you should use your own domains 😄

Start Container Stack

Now we can create and start the containers. For this, first change the directory where the docker-compose.yml file from the repository is located:

cd /opt/docker-services/portainer-caddy

Since I created an external network in the docker-compose.yml, the web network must be created:

docker network create web

Now start the Container Stack:

docker-compose up -d

When you now type docker ps, it should look something like this:

CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                                                          NAMES
f4b9f9302962   portainer/portainer-ce:latest   "/portainer -H unix:…"   4 minutes ago   Up 4 minutes   8000/tcp, 9443/tcp, 127.0.0.1:9000->9000/tcp   portainer-caddy-portainer-1

Load Caddy Configuration for Container Stack

The Caddyfile that we have previously adjusted can now be loaded by Caddy:

caddy reload

Possibly Set Caddy Permissions

Depending on how the operating system is configured, Caddy may need permission to bind itself to ports 80 and 443. When you execute caddy reload in the terminal, Caddy will output this as an error.

tcp:443: bind: permission denied

You set the permissions as follows:

Bash

sudo setcap CAP_NET_BIND_SERVICE=+eip $(which caddy)

Fish

sudo setcap CAP_NET_BIND_SERVICE=+eip (which caddy)

Again:

caddy reload

The nice thing about Caddy is that I can have Caddyfiles with different settings in different folders and simply load them with caddy reload without having to restart the server service.

Basic setup of Portainer

Now that everything should be running 🚀 you can now call up Portainer in the browser. In my case, it's ca.m3tam3re.com.

You should now be able to set your administrator data for Portainer:

Once you have set up your admin account, you will also be logged in directly and we are now ready to go for 🚀 n8n 🚀.

Installation of n8n

Now that the basic setup is complete, we can finally start our first container. I will show two possibilities here, which one you choose is up to you.

1. With docker-compose via the terminal

Even though we just installed Portainer, I have designed the repository in such a way that the whole thing can be set up easily without Portainer. So, let's get started:

Adapt environment variables

First, we need to go to the n8n folder in our repository:

cd ../n8n

The directory should look like this.

/opt/docker-services/n8n
├── docker-compose.yml
├── DockerFile
└── .env

Open the file .env and adjust the values accordingly. In my case, it looks like this:

# TLD - Your main domain on which n8n will run
DOMAIN_NAME=m3tam3re.com

# The subdomain for n8n
SUBDOMAIN=io

# DOMAIN_NAME and SUBDOMAIN determine in combination how n8n will be accessible
# in the above example, this would be: https://n8n.your-domain-here.com

# The username for logging in - PLEASE CHANGE!
N8N_BASIC_AUTH_USER=admin

# The password for login - PLEASE CHANGE!
N8N_BASIC_AUTH_PASSWORD=noneOfYourBusiness0815

# Here you can set the timezone, this is important for CRON tasks, for example
# If not set, New York will be set
GENERIC_TIMEZONE=Europe/Berlin

# Allows importing and using internal modules in function nodes
NODE_FUNCTION_ALLOW_BUILTIN=*

# Allows importing and using external modules in function nodes
NODE_FUNCTION_ALLOW_EXTERNAL=*

Since n8n has problems with the permissions for regular Docker volumes in this constellation, we have to create another folder for the permanent storage of data from n8n:

mkdir n8n-data

Bash:

chown $(whoami):docker n8n-data

Fish:

chown (whoami):docker n8n-data

That's all there is to it. Now we can start the stack:

docker-compose up -d

2. Set up n8n via Portainer

Create folder for permanent storage of data

First, we need to go to the n8n folder of our repository again in this variant:

cd ../n8n

mkdir n8n-data

Bash:

chown $(whoami):docker n8n-data

Fish:

chown (whoami):docker n8n-data

Create the n8n Stack in Portainer

When you click on Add stack, you will see the configuration window for the stack:

There are only a few things to set up here:

1. Specify the name of the stack.
2. Copy the contents of the file n8n/docker-compose.yml into a web editor or upload the file via the Upload tab.
3. Upload the file n8n/.env.

You don't need to adjust the .env file beforehand. Portainer allows you to adjust the values beforehand.

Now start the stack with a click on Deploy the stack.

Set up n8n

Finally done 🚀 😄. I must admit that I am also running out of steam. Writing all this down takes a lot longer than I thought 🤣.

Now you can call the domain for n8n. In my example it's io.m3tam3re.com and set up the admin account:

Now you can login and land on the page for setting up the workflow. Who wants can simply browse through the templates here. n8n offers lots of ready-made templates for automating tasks 😄

That's it 🚀

Conclusion

Just a few closing remarks 😄:

1. In many videos, the combination Docker and Traefik is currently shown and praised a lot. Traefik is a great program, and I would recommend it without hesitation to anyone. The reason I use Caddy here instead is because Caddy is much easier to configure and simplifies many things. For example, Caddy automatically generates valid SSL certificates for the domains, which has to be configured for Traefik for each domain. In production, of course, it makes sense to start Caddy as a service and do the configuration via Caddy's REST API. But this and the following examples are perfectly adequate for the setting shown here 😄.

2. I deliberately use the n8n-debian image. I know that Alpine-based containers are much smaller, but it's not about storage space here. With the Debian image, I have access to a vast database of binaries that I can install and use in no time. If I were to take the Alpine container, I would probably have to compile a lot of stuff from the source code because there are much fewer packages.