Complete guide to deploying django on Ubuntu with nginx, FastCGI and MySQL

Posted Monday, November 3rd at 5:38p.m. under Django, Ubuntu, Nginx, Python

I recently deployed my first django project and it was an eye-opening experience to say the least. Coming from Windows, PHP, Wampserver and the cPanel shared hosting culture, starting with a fresh Ubuntu install and having to build the stack myself was something of a culture shock. There were of course a wealth of tutorials out there to help me get it up and running, but the reason I'm writing this is because the information I needed to get from brand new VPS to fully functional django app was split across many different tutorials, some of which were better than others. For my own reference (and for when I inevitably break something and need to start from scratch!), here is how I did it in ten simple steps:

Step 1: Get a slice

It's important to realise that you don't actually have to abandon the shared hosting model to deploy your django application, many shared hosts specialise in django hosting since the work needed to do this yourself can be quite intimidating. This thread over at Stack Overflow has a good outline of the available options.

I chose the route of getting a VPS (Virtual Private Server) because I liked the idea of having total control over the platforms my hosting service provided as well as the desire to get some systems administration experience under my belt. It's hard to get the chance to play with this stuff in a commercial environment when you have live systems with millions of users relying on everything working. So with that in mind, I started looking for a good VPS provider and after overwhelming recommendation, quickly settled on slicehost. Since my django project wouldn't be too high-traffic, I chose the 256MB slice with the default Linux distribution of Ubuntu 8.04.1 (Hardy).

Step 2: SSH into the slice

Windows users use this guide if you're having problems.

Step 3: Set up a non-root user account

I can't stress enough that you follow the two guides on slicehost for best practices for setting up your slice. The most important part is that you do not do anything under the root account. I put that in bold because in my haste I ignored all the stuff about creating a seperate user account for myself as I planned to be the only person logging into the server. This is fine, but when you create files and directories with root the default permissions are extremely strict and that will cause major problems when your server tries to read and/or execute these files later.

Step 4: Install the stack

You may have heard of the term LAMP before - it usually refers to the common server stack consisting of Linux, Apache, MySQL and PHP.  Since we're starting with a nice fresh slice, we're going to have to set this up ourselves (with Python and Django replacing PHP, of course).

The only notable thing here is that I've went with nginx over Apache. This is because on a 256MB slice, I want to make the most of my limited RAM. The two major alternatives to Apache in the lightweight category are lighttpd and nginx. After doing a bit of research, I installed both, played with the virtual hosts and config and decided I liked nginx better. This article and others like it provide a good overview of all the options.

The stack:

Linux distro - Ubuntu 8.04.1
Server - nginx 0.5.33
Database - MySQL 5.0.51
Language - Python 2.5.2 (default with Ubuntu)

Step 4a: Installing nginx

Getting nginx up and running to serve static files is quite straight forward, and the guides on slicehost articles were extremely easy to follow so I'm not going to repeat them here:

1. Install nginx with aptitude
2. Basic configuration of nginx
3. Create a virtual host for your django site

By this point you should have at least the default nginx page displaying under your slice IP address. Getting a domain set up on your slice is quite tricky too but luckily there's an easy guide for that as well.

Step 4b: Installing MySQL

Again, aptitude makes this easy. Run the following command:

sudo apt-get install mysql-server mysql-client

You will be taken through a wizard to create a root account, make sure you do this then test the install by entering the mysql command line client:

mysql -u root -p

Just being prompted for your password tells you that MySQL was installed successfully. You should go ahead and create a database for your django app to use later. I recommend at some point you create a user with restricted permissions for your web client to connect as.

Step 4c: Confirm Python install

Ubuntu comes with Python pre-installed, you can type 'python' into the shell to make sure.

Step 5: Install django

Install django from the latest development branch. We will need to install subversion first:

sudo aptitude install subversion
mkdir /home/username/installers
cd /home/username/installers
sudo svn co http://code.djangoproject.com/svn/django/trunk/
sudo python trunk/setup.py install

This should be straight forward. We install subversion, check out the code somewhere (I recommended checking out installers and projects to one location to keep your slice organised) and then run the set up script. You should get clear feedback from the prompt to let you know how everything went.

Step 6: Install django dependencies

Our goal is to run django under FastCGI with a MySQL database. If you go ahead and try to do this, you'll be told there are missing packages when you run syncdb and later when you try to run django as a FastCGI:  we need to install both flup and python-mysqldb.

Installing python-mysqldb is easy:

sudo apt-get install python-mysqldb

To install flup we first need to install "easy_install" - a Python utility for installing packages. Here are the commands:

cd /home/username/installers
sudo wget http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg
sudo sh setuptools-0.6c9-py2.5.egg
sudo wget http://www.saddi.com/software/flup/dist/flup-0.5-py2.5.egg
sudo easy_install flup-0.5-py2.5.egg

Once again, pay attention to the feedback at the command prompt to make sure everything is (hopefully) installing successfully.

Step 7: Create a test project

Create a directory somewhere on your file system and start a django project with a small default app, then go in and change the settings to look for your MySQL database (you will need to create a new database for django to use if you haven't already done so). If you used the slicehost method of organising your code, you would run commands like this:

cd /home/username/public_html/mydomain.com/private/
django-admin.py startproject myproject
cd myproject
python manage.py startapp myapp
nano settings.py

You then need to enter your database details and paths. Here is a sample for a MySQL connection:

DATABASE_ENGINE = 'mysql'      #
DATABASE_NAME = 'mysite'       #
DATABASE_USER = 'websites'     #
DATABASE_PASSWORD = 'mypasswd' #
DATABASE_HOST = ''             #
DATABASE_PORT = ''             #

Don't worry too much about the other path details (like media and templates) for now, all we're doing here is trying to get that default "welcome to django" page to display so that we can confirm everything else is working.

Step 8: Set up nginx conf for django

When you installed nginx, hopefully you took the trouble to get yourdomain.com running and displaying static content on your nginx install - because we're about to edit the config file to make it serve django content. If you didn't, then follow these steps:

1. Change your domains nameserver records to the slicehost settings. Whoever you registered your domain with will provide the tools to do this. See your slicehost manager control panel for the nameserver domains to point to.
2. Use the DNS wizard in slicehost manager to handle the domain
3. Set up a nginx virtual host which will handle requests to this domain

If you don't have a domain yet and just want to use the IP for the time being, use the default config file located at (/etc/nginx/sites-available/default) in the following examples.

Open your domain config file:

sudo nano /etc/nginx/sites-available/mydomain.com

And change it to look like this, substituting in your project path:

server
{
    listen   80;

    server_name www.yourdomain.com;
    access_log /path/to/yoursite.com/log/access.log;
    error_log /path/to/yoursite.com/log/error.log;
    root /path/to/yoursite.com/public;

    location /site_media
    {
        root /path/to/yoursite.com/public;
    }

    location /
    {
        # host and port to fastcgi server
        fastcgi_pass 127.0.0.1:8081;
        fastcgi_param PATH_INFO $fastcgi_script_name;
        fastcgi_param REQUEST_METHOD $request_method;
        fastcgi_param QUERY_STRING $query_string;
        fastcgi_param CONTENT_TYPE $content_type;
        fastcgi_param CONTENT_LENGTH $content_length;
        fastcgi_pass_header Authorization;
        fastcgi_intercept_errors off;
    }
}

This will direct all web requests to http://www.yoursite.com/ to a FastCGI process (i.e django) except those under http://www.yoursite.com/site_media/. The port you use here has to be different for each instance of django you want to run, so take a note of it here as we'll need it when we run the FastCGI process later.

I had a number of legacy links on my band website that I had to transfer over and I also used a slightly different directory structure from the slicehost articles but it all follows the same pattern. For reference, here are my settings:

server
{
    # redirect all mesaverde.co.uk requests to www.mesaverde.co.uk
    listen 80;
    server_name mesaverde.co.uk;
    rewrite ^/(.*) http://www.mesaverde.co.uk/$1 permanent;
}

server
{
    listen   80;
    server_name www.mesaverde.co.uk;

    access_log /home/david/websites/mesaverde.co.uk/log/access.log;
    error_log /home/david/websites/mesaverde.co.uk/log/error.log;

    root /home/david/websites/mesaverde.co.uk/public;

    location /ui
    {
        root /home/david/websites/mesaverde.co.uk/public;
    }

    location /mp3
    {
        root /home/david/websites/mesaverde.co.uk/public;

    }

    location ~* ^.+\.(jpg|css|jpeg|gif|png|ico|zip|pdf|txt|wav|js|mov|mp3)
    {

       access_log   off;
       expires      30d;
    }

    location /

    {

        # host and port to fastcgi server
        fastcgi_pass 127.0.0.1:8081;
        fastcgi_param PATH_INFO $fastcgi_script_name;
        fastcgi_param REQUEST_METHOD $request_method;
        fastcgi_param QUERY_STRING $query_string;
        fastcgi_param CONTENT_TYPE $content_type;
        fastcgi_param CONTENT_LENGTH $content_length;
        fastcgi_pass_header Authorization;
        fastcgi_intercept_errors off;

    }

}

All my CSS, JavaScript and image files are located inside www.mesaverde.co.uk/ui/ and I also have a page of mp3s that I host at http://www.mesaverde.co.uk/mp3/, using this set up I don't involve django in any static file serving.

Step 9: Run django as a FastCGI

The final step is starting up django as a FastCGI. Remember in the development mode when you had to use runserver to test your code? Well it's the same idea here, with a couple of extra options:

cd /path/to/your/project
python manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings

Note that the port you specify here should match the one you chose in your site conf in step 8. Once you have run this command, you can go to http://www.yourdomain.com and if all is well, you will get the default 'welcome to django' page.  Congratulations, you have deployed django and the worst is over!

Step 10: Settings, uploading projects and other configuration

Setting up an FTP server

Now that django is working, you'll probably want to upload your django app and attempt to get that working. If you come from a shared hosting background like I did, you'll probably take FTP access for granted. Unfortunately it doesn't quite work like that. You'll need to install an FTP server on your slice, but this is a lot easier than it sounds. I chose VSFTP for this:

sudo aptitude install vsftpd

Once it's installed (and automatically started) you'll now need to open up the conf and change a couple of key settings:

sudo nano /etc/vsftpd.conf

Most importantly, you should disable anonymous access and set up access for local users:

anonymous_enable=NO
local_enable=YES
local_umask=022

The two local settings allow you to FTP in as the same user you SSH in as and make sure that the permissions are reasonable. Once you change these settings, restart your FTP server for them to take effect:

sudo /etc/init.d/vsftpd restart

Now you can log into your slice with your usual FTP client and upload your django project files. Use your slice IP, and the username and password you've been using to SSH to your slice. Remember to change settings.py to reflect the new templates path.

Admin Media

If your project is using the automatic admin then you might notice it looks a bit bare. That's because django doesn't automatically load the media for you like it does on the development server. We'll need to set this up.

The admin media is located in a restricted part of your system that you need superuser permissions to access. So first we grant it more liberal permissions:

sudo chmod +rx /usr/lib/python2.5/site-packages/django/contrib/admin/media

Here we've given it read and executable permissions for all users. Next we need to make this directory available inside the directory we chose for our static files when we set up our nginx conf (your site_media directory or whatever you named it). The typical Windows way to do this is just to make a copy, but in UNIX we can use a symbolic link to achieve this.

ln -s /usr/lib/python2.5/site-packages/django/contrib/admin/media /home/username/public_html/yoursite.com/public/site_media/admin

The idea is that you can now access these files through a URL like www.yoursite.com/site_media/admin/dashboard.css. By default, admin media is served through www.yoursite.com/media/ so we need to change the ADMIN_MEDIA_PREFIX setting in settings.py:

ADMIN_MEDIA_PREFIX = '/site_media/admin/'

Restart django and the admin should now look more familiar.

Restarting Django

When we make changes to nginx, there is a handy init script that takes care of it for us:

/etc/init.d/nginx restart

So using information from the official docs let's create a similar one for django. Create the script in your /usr/local/bin directory.

sudo nano /usr/local/bin/yoursite-restart

Then save into this file the following script:

#!/bin/bash
PROJDIR="/home/username/public_html/yoursite.com/private/project"
PIDFILE="$PROJDIR/yoursite.pid"
cd $PROJDIR
if [ -f $PIDFILE ]; then
   kill `cat -- $PIDFILE`
   rm -f -- $PIDFILE
fi
exec python ./manage.py runfcgi host=127.0.0.1 port=8081 pidfile=$PIDFILE --settings=settings

Obviously you need to change the path to point to your project directory and make sure the port setting matches the one in your nginx config. Save the file, give it executable permissions and then make sure django isn't currently running:

sudo chmod +rx /usr/local/bin/yoursite-restart
ps -ef | grep "fcgi"

This will give an output of currently running processes which contain "fcgi" in them. It should look like:

root     12085     1  0 Nov02 ?        00:00:00 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings
root     12086 12085  0 Nov02 ?        00:00:02 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings
root     12087 12085  0 Nov02 ?        00:00:02 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings
root     14818 12085  0 10:29 ?        00:00:00 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings
root     14869 12085  0 12:23 ?        00:00:00 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings
root     16544 12085  0 15:54 ?        00:00:00 python ./manage.py runfcgi host=127.0.0.1 port=8081 --settings=settings

The key process we're interested in is the first in this list. We want to take note of the first number - 12085. This is the process ID. The other runfcgi processes are just children of this process and killing the parent will kill the children processes too:

kill 12085

Django is now stopped. We can test the bash script we just created by running:

sudo yoursite-restart

If you created the file in /usr/local/bin you don't need to specify the full path to the file, which is why we can just type the script name directly. Django should now be running again and each time you make changes to your django settings or models, etc. you can just run this command to restart the django processes.

Congratulations, you're done!

Conclusion

Although there are a lot of small steps, deploying django is simple and a good exercise in beginning to understand the most popular software stack that supports the web. For Windows user in particular, this kind of basic systems administration will be a good introductory exercise to UNIX and set you apart from many applications developers who avoid this kind of thing.

10 Responses

colophon