Deploying a Wagtail site with Gunicorn, Nginx and Supervisor

In a previous tutorial we have set up our project to run locally on a DigitalOcean server. We are now going to install Gunicorn and Nginx to serve our site to the internet, as is also described in a DigitalOcean tutorial. However, our setup will be slightly different: we will use a Gunicorn configuration file and use Supervisor to manage both Gunicorn as well as Nginx. There are more tutorials out there, such as this comprehensive guide. Linux tutorials can be found here and here.

Gunicorn will serve as application server, Nginx as web server and reverse proxy and Supervisor as management tool to run these. Gunicorn will run our site and serve dynamic content (from the database) and will be installed in the same virtual environment as our site. Nginx will serve static content (media and images) and will delegate requests for other content to Gunicorn. Nginx will be installed on the server, outside of the virtual environment. Supervisor will also be installed on the server and will be used to start, stop and configure both Nginx and Gunicorn. The steps will be:

1. update production settings
2. install and configure Gunicorn
3. install and configure Nginx
4. install and configure Supervisor
5. run our site

1. Update production settings

Until now we have only used our project's development settings. With Gunicorn and Nginx we will use the production settings, where debug = False. We then need to set the parameter ALLOWED_HOSTS, otherwise Django will generate a Bad Request error. We'll change the settings on our development computer, push them to the repository and on our server pull them from there. In your production.py settings file add the line:

Push the change to the repository:

Go to the server, activate the virtual environment, enter the project's directory and type:

Review the file production.py to check that the change is made. Let's also collect all static files in the directory /static/, so that they can be picked up by the webserver later:

2. Install and configure Gunicorn

Log in at the server as the user that we created (use your own IP address):

Enter the virtual environment and install Gunicorn:

To test that Gunicorn can serve the site, go into your project's directory and type:

Visiting your site with the browser on http://165.22.199.4:8000 (substitute your IP address) should give the same result as in the previous tutorial with the runserver command. Shut down Gunicorn with ctrl-C.

There's several ways to define the configuration for Gunicorn. We could create a service file for the systemd command, or create a bash script. Here we will create a Gunicorn configuration file as described in the Gunicorn docs. We can place the file anywhere; to stay close to Linux's file organization we will create a directory /etc for configuration files with a subdirectory /gunicorn:

The Gunicorn repository contains an example configuration file. We will copy some of the settings that we need. Let's start with a few simple ones, which are explained in the example file. Open conf.py with vi or another editor and add:

We also add information on how we want to log:

The specified files don't exist yet. We don't need to create the files, but we do need to make sure the directory exists, so exit the editor temporarily and create a directory ~/env/var/log. Also create a directory ~/env/run for the socket file (see below).

Return to editing the configuration file conf.py. Essential is the bind parameter, which tells Gunicorn what the interface is with which to communicate to the outside world. Above we used an IP-address, now we will use a file (a unix socket file) via which Gunicorn will interface with Nginx (that in turn will connect to an IP-address). Gunicorn will create that file itself, but the directory needs to exist. The syntax is:

We need a way to tell Gunicorn to use the production settings of our project. The Django parameter for this is DJANGO_SETTINGS_MODULE and we can set it using the Gunicorn parameter raw_env (see also example file):

To make sure Gunicorn can find the Django settings file, we need to specify the Python path:

That concludes the Gunicorn configuration file.

3. Install and configure Nginx

Exit the virtual environment if needed and install Nginx:

Nginx creates a generic configuration file at /etc/nginx/nginx.conf, and two directories /sites-available and /sites-enabled. We will create a server block configuration file in the /sites-available directory. Create the file (we call it pet) and open it with your editor (in our case vi; use sudo for write permission):

In this file we will put a server block with a number of settings (see "Configure Nginx to Proxy Pass to Gunicorn"):

• domain name(s) of the server
• path to log files
• location blocks telling us how to handle different uri's

Put in the following content and save the file; see below for the explanation.

The first line specifies the domain names of our site, the second and third the log files. The first location statement tells Nginx to ignore any potential problems with finding a favicon. The second and third location statement specify the location of the static and media files in our project, so that Nginx can serve these directly if asked for it. The final statement matches all other requests and points to the socket file we created earlier, thereby proxy passing the request to Gunicorn. It includes a parameter file proxy_params created during install in the directory /etc/nginx.

The actual Nginx settings will be picked up from the directory /sites-enabled, so we make a symbolic link from /sites-available to there:

Test the Nginx configuration for errors with:

To check that Nginx is running, use the systemctl command; it should display that Nginx is "active (running)".

We can now check whether Gunicorn and Nginx can exchange data via the socket file, by running Gunicorn with its configuration file:

Here the first path leads to the Gunicorn executable, the argument pet.wsgi:application is the variable application within the file pet.wsgi in your project, and the second path leads to the configuration file that we created above. This command will start Gunicorn in the foreground, temporarily occupying your terminal window. In your browser visit your domain (www.pythoneatstail.com); if all is right you will be able to access your site. You can check that Gunicorn has created the socket file by opening a second terminal and listing the contents of the directory /home/usr_pet/env/run. Stop the Gunicorn process in the first terminal with ctrl-C. The socket file will be gone.

5. Install and configure Supervisor

We will use Supervisor to monitor and control both the Gunicorn and the Nginx process. It is slightly simpler and more user friendly than the built-in Systemd and can delegate to non-root users. Supervisor could be installed in the virtual environment, but since we intend to use it to manage Nginx, which is installed on the server, this is not very logical and requires more configuration. Installing can be done via pip, but we will use apt-get as recommended by DigitalOcean, despite the fact that it gives us an older version:

Supervisor has created a configuration file in /etc/supervisor. If you inspect it, you can see that it includes all application-specific configuration files in the subdirectory /conf.d. Go into that subdirectory, create a file guni-pet.conf and paste the following content:

All configuration parameters are in the docs. The first line specifies the execution command that we used before, the second line specifies the user and the third and fourth line tell Supervisor to automatically start and restart the process after an exit. For Nginx create a file nginx-pet.conf and paste:

In the command we set "daemon off" with the -g switch because Supervisor needs subprocesses to run in the foreground. This time we don't specify a user, since Nginx was installed by root and therefore needs to be run by root. For the same reason we repeat the logfile locations, to make sure the logs end up there.

That completes the setup of Supervisor. The command line client part of Supervisor is supervisorctl. Add the new configurations to Supervisor with:

5. Run the site

This should now be as simple as:

which starts Gunicorn and Nginx. Go to your domain address in your browser (in our case www.pythoneatstail.com) and see if it works. Stop either one of the processes, start them again, check the status, using the various Supervisor commands (with <process-name> either guni-pet or nginx-pet):

If we haven't created any content, then our site is still empty. We can fill it manually, or use a backup; read another tutorial explaining how to do that. We will also set up a firewall, enable https and set ssh access keys.