Thursday, June 16, 2016

Installing Python 3 and Django on Dreamhost

I'm working on a side project for a friend that has zero budget and since I'm already paying for Dreamhost (my favorite and only choice for shared hosting) and totally underusing it, I figured I'd use this project as my opportunity to figure out how to install Python 3 and Django since A) Dreamhost lets you do install things even on their shared platform (one of the reasons I love them!), and B) the default Python on Dreamhost is Python 2.7.3.

1. Install Python 3.5.1

This part's really simple and Dreamhost even documents it nicely on their knowledge base, but for the sake of one-stop shopping I'll include the steps I performed here. Note that I'm leaving the default Python 2.7.3 in place since I'm not sure what havoc it would wreak to blow it away, so Python 3.5.1 will be installed in such a way that it'll be run as python3. (For the curious, you can also read more about Python at Dreamhost in general.)

Before going any further, if you haven't enabled SSH access for your Dreamhost user you need to do that first since if you can't log in you can't install anything.
  1. Log into the Dreamhost web panel
  2. Click on "Users" and then "Manage Users"
  3. Find the relevant user and click the edit button
  4. Under "User Type" choose "Shell user," and I'd recommend also choosing /bin/bash as the shell type
  5. Scroll to the bottom of the page and click "Save Changes"
Next, we'll install Python 3.5.1.
  1. Go to and get the URL for the gzipped source tarball.
  2. SSH into your Dreamhost server
  3. In your user's home directory, wget the tgz file using the URL you grabbed in step 1:
  4. Extract the Python source:
    tar xvf Python-3.5.1.tgz
  5. cd into the extracted source directory:
    cd Python-3.5.1
  6. Configure Python 3.5.1 with a prefix flag so as not to stomp on anything that's already installed:
    ./configure --prefix=$HOME/opt/python-3.5.1
  7. Run make and make install:
    make && make install
  8. Add the new Python installation to your path:
    export PATH=$HOME/opt/python-3.5.1/bin:$PATH
  9. Update your bash profile:
    . ~/.bash_profile
  10. Verify that everything's working (this should output Python 3.5.1):
    python3 -V
If that all looks good at this point, we're ready to proceed and create a virtualenv in which we'll install Django.

2. Create a virtualenv

Python 3 ships with pyvenv, but since it works a bit differently than virtualenv which is what I'm used to using, I decided to install virtualenv and use that. Feel free to use pyenv here if that floats your boat.
  1. Use pip3 to install virtualenv:
    pip3 install virtualenv
  2. Create and/or cd into the directory in which you want to put your virtualenv. I tend to put mine in ~/.virtualenvs, so I did this:
    mkdir ~/.virtualenvs
    cd ~/.virtualenvs
  3. Create a virtualenv to use for our Django project (since we installed virtualenv with pip3 the default Python environment will be Python 3.5.1):
    virtualenv my_django_project
  4. Activate your virtualenv:
    source my_django_project/bin/activate
  5. Confirm everything looks correct at this point (this should output /path/to/virtualenv/bin/python3):
    which python3
If we're still on track we can proceed to install Django.

3. Install Django (and your other requirements)

At this point since we have Python 3.5.1 installed and have successfully created a virtualenv, we can do the usual installation of requirements using pip3, which typically will come from a requirements.txt file. For the sake of illustration we'll go through installing Django into our virtualenv and making sure it works, but pip3 install -r requirements.txt will be the more typical way of installing all the packages you need.
  1. With your virtualenv activated, install Django:
    pip3 install django
  2. Verify everything is working by starting a Python interpreter:
  3. Import Django to make sure it's in your virtualenv (this shouldn't throw any errors):
    >>> import django
Now that we have verified that our virtualenv is working and sane, we can proceed to the rest of the configuration to get a Django project running successfully in this environment on Dreamhost.

4. Enable Passenger on Your Web Site

Rather than the gunicorn or some of the other WSGI servers you may be familiar with, on Dreamhost you can use either Passenger or FastCGI as the application server for Django applications.

For this post I'll be using Passenger, a tool with which I was not familiar until I saw information about it on Dreamhost. Passenger is apparently more common in Ruby on Rails setups, but it serves the same purpose and falls in the same place in the stack as any of the other WSGI tools with which you may have experience. You can read more about Dreamhost's use of Passenger here and here.

Some Notes About Passenger

Worthy to note about using Passenger on Dreamhost is that once Passenger is enabled, all requests that come into your domain will be handed off to Passenger by Apache. Passenger will first look see if there are any files matching the request in your domain's /public directory (and note that the files must be in the /public directory or subdirectories under /public specifically), and if it can't find an appropriate file, Passenger then tries to handle the request itself.

To provide a couple of concrete examples, let's assume you have a file called hello.html in your domain's /public directory, and a request comes in for Here's what happens next:
  1. Apache hands the request off to Passenger immediately. This is important to note since Apache is out of the picture entirely other than serving as a proxy to Passenger.
  2. Passenger looks for a file called hello.html in your domain's /public directory. Since it finds one, it returns that page to the requester.
Now let's assume a request comes in for Here's what happens in that case:
  1. Apache hands the request off to Passenger.
  2. Passenger looks for an appropriate file in /public. This time it doesn't find one.
  3. Passenger then attempts to handle the request itself, and since later we'll be configuring Passenger to serve up our Django application using WSGI, at this point Django would be processing the request.

Enable Passenger

In order to run a Django application on Dreamhost we'll need to enable Passenger on the domain on which we're going to run our Django application.
  1. Log into the Dreamhost web panel
  2. Click "Domains" and then click "Manage Domains"
  3. Locate the domain on which you'll be hosting your Django application and click "Edit"
  4. Scroll down and under "Web Options" you'll see a checkbox labeled "Passenger (Ruby/NodeJS/Python apps only)." Check that box.
  5. Scroll to the bottom of the page and click "Change Settings."
That's it for enabling Passenger. Next we'll throw our Django application at Dreamhost and see what sticks.

5. Create a Django App

For the purposes of this post we'll be creating and configuring a Django application directly on the Dreamhost server. Obviously if you have an application that's already written that you want to run on Dreamhost you'd be uploading your application's files as opposed to doing things from scratch on the server, but this should illustrate some of the nuances of what you need to do with your own Django application to get it running on Dreamhost with Passenger.

  1. ssh into the Dreamhost server
  2. Activate the virtualenv you created above:
    source ~/.virtualenvs/my_django_project/bin/activate
  3. cd into the root directory of the web site on which you'll be running your Django application. Note that this is not the aforementioned /public directory under your web site's directory, but the root of the web site itself.
  4. Start a new Django project using
    python3 startproject my_django_project
  5. Do a quick gut check by firing up the Django development server:
    cd my_django_project
    python3 runserver
At this point you should see the Django development server start up without any errors. We're nearly there! The last step is to configure this newly created Django application to work with Passenger. But before we do that, a quick note about static files.

Serving Static Files and the Static Root Setting

It wouldn't be Django if we didn't have to stop and say a word about serving static files. Luckily this isn't anything out of the ordinary, but given how Passenger works you do need to be aware of how to set your STATIC_ROOT to make sure everything comes together smoothly.

  1. Create a static directory under your domain's public directory, i.e. you'll wind up with ~/
  2. Go into your Django project's settings file and add a STATIC_ROOT setting:
    STATIC_ROOT = os.path.dirname(BASE_DIR) + '/public/static/'
With that you should be all set to run collectstatic and have everything work properly. Now let's move on to configuring Passenger.

6. Configure Passenger

In order to have Passenger load up our Django application, we need to create a file in the root of our domain. As with all WSGI files this is what tells the application server, in this case Passenger, how to load up the application so it can handle requests.

In the root of your domain, create a file with the following contents:

With that file in place, you should be able to hit and see the default "Congratulations" Django page. Congratulations indeed! You now have a Django application running on Python 3.5.1 on Dreamhost.

A common error you might run into is something like "no module named foo" or "no module named foo.settings." If you get something along these lines it means your paths are incorrect, so make sure that your Django project is in a directory directly inside your domain's root, not in the /public directory. If that isn't the issue, double-check all the paths in the file.

Additional Passenger Considerations

It's worth noting that if you change your file you need to run pkill python to reload the Django application since Passenger loads your application into memory (which is one of its advertised advantages since it makes things pretty zippy).

I also saw note of having to run touch tmp/restart.txt but in my experience I didn't have to do that, so I'm not sure if that's more of a Rails thing or if I simply didn't yet run into situations in which that was necessary. I'm noting it here in the event pkill python doesn't do the trick.

Final Notes

There you have it. You now have the latest Python and Django running on Dreamhost. At this point you can load your own Django project, set up a database (note that Dreamhost only supports MySQL, and even though I much prefer Postgres, MySQL works just fine), and do all the usual Django stuff.



Matt Woodward said...

Quick follow-up about STATIC_ROOT and MEDIA_ROOT settings.

I have my Django project set up with a settings module as opposed to a single file, so these settings worked for me to put files in the right place when running collectstatic, or uploading files in the app using FileField or ImageField:
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(os.path.dirname(BASE_DIR), '../', 'public/static/')

MEDIA_ROOT = os.path.join(os.path.dirname(BASE_DIR), '../', 'public/media/')
MEDIA_URL = '/media/'

Sean D. said...

thank you for the article. i just went through the process of standing things up and used your guide for my passenger woes and it helped quite a bit. Cheers!

Matt Woodward said...

Excellent! Glad it was helpful.

SoulSnax said...
This comment has been removed by the author.
SoulSnax said...
This comment has been removed by the author.
SoulSnax said...

For the STATIC_ROOT and the MEDIA_ROOT do you mean that we replace the '../' with your domain name as indicated below?

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(os.path.dirname(BASE_DIR), '', 'public/static/')

MEDIA_ROOT = os.path.join(os.path.dirname(BASE_DIR), '', 'public/media/')
MEDIA_URL = '/media/'

SoulSnax said...

or could it just be like this?:

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(os.path.dirname(BASE_DIR), '')

MEDIA_ROOT = os.path.join(os.path.dirname(BASE_DIR), '')
MEDIA_URL = '/media/'

Blogger said...

DreamHost is the best hosting company with plans for any hosting requirments.