Blog

7 Tips for Python & Django on Heroku, Testing, and more from a Ruby on Rails developer

May 3, 2012 10:41 am / by / 7 comments

While Niall has been coding in Python for years and has worked extensively with both Django and Pyramid, I am new to Python. We have been building Strider primarily in node.js (Strider is our hosted continuous deployment platform for Python and node.js – learn more at StriderCD.com), and in the past I have done some coding in Rails and further back in both Perl and Java, but no Python. To kick things off with Python, I decided to build a basic Django project. The project is called Klingsbo (github).

I started by walking through the excellent Django Project tutorial. The tutorial guides you through the process of creating a very basic project with one app – polls. The tutorial does a great job getting a new user up and running with Django in short order. There were, however, a few additional things that I needed to implement for this sample project to meet my needs.

1. Relative Paths for Templates

One thing that I found surprising in the tutorial was the use of absolute paths for the templates. I can’t imagine why anyone would want to use absolute paths in a project configuration. After searching around a bit, I found the following code to put the templates directory inside the project directory (ie the same directory as settings.py):

TEMPLATE_DIRS = (
  os.path.join(SITE_ROOT, 'templates')
)

2. Testing

There is nothing in the tutorial about automated testing with Django. After some more searching, I managed to find this excellent pair of blog posts by Daniel Lindsley. You can find them here: part one & part 2. Daniel goes further than I did in my project; if you are building a Django project yourself, I would recommend walking through the entirety of both of his posts.

3. Initial data

The tests used fixtures for sample data and I also wanted to load some initial data when the project is first setup. Django will let you provide initial data via fixtures (just like the test data) or via SQL. I opted for fixtures. I created a json file called ‘initial_data.json’ which I put inside the fixtures directory of my app. (Note that this sample data will be re-added every time you run syncdb so you should not put data in here that will be changing. If you want to load data once but not upon subsequent execution of ‘syncdb’, then you should name the file something else and explicitly run ‘manage.py loaddata [filename]’ the first time around).

See the Django documentation on Providing initial data for models for further details.

4. Non-interactive setup

By default, the first time you run ‘manage.py syncdb’, it will ask you interactively to create a superuser. In order for Strider to be able to setup the project programmatically, I needed to turn off the interactivity, but yet still create a superuser. There are a couple of ways to do this. I came across this blog post which recommends dumping the data from the db to json and then adding the superuser json section to an ‘initial_data.json’ file. This is how I am creating the superuser in Klingsbo. (Note: when I tried to put the ‘initial_data.json’ file in the klingsbo (interior) project folder, it didn’t load, so I ended up appending the superuser initial data to the initial data for the polls, ie the initial_data.json file located in ‘polls/fixtures’. There is most likely a better way to do this.)

Another way to create a superuser non-interactively is via the following sequence of commands. For a production app, this might be preferable to creating the superuser via the initial data load:

python manage.py syncdb
python django-admin.py createsuperuser --username admin --email admin@exmaple.com
python django-admin.py changepassword --username admin [password]

5. Data migrations

I was a bit surprised that Django does not have built-in support for data migrations (as Rails does) but it was easy enough to get up and running with South, which seems to be the most popular of the Django migration alternatives. Since I had already created a project and run ‘syncdb’, I followed their instructions on Converting an Existing App. Basically I just had to add ‘south’ to my list of installed apps and then execute the following command:

python manage.py convert_to_south polls

And that’s it. In the future, when I change a model, I will need to run this command before I commit to create a new migration:

python manage.py schemamigration [model] --auto

And then of course after pushing that commit, I will need to run ‘migrate’ on my production server:

python manage.py migrate

6. Deploying to Heroku

Next, I wanted to deploy my project to Heroku. Heroku has a great guide to running Django projects that is very detailed (Getting Started with Django on Heroku/Cedar). The key steps are that you need to install psycopg2 (even if you are using sqlite locally) as well as gunicorn, and you need to create a Procfile. My Procfile looks like this:

web: gunicorn klingsbo.wsgi -b 0.0.0.0:$PORT

Once you have those three items in place, you can then deploy your app to Heroku from the command line (or via Strider of course). After you have deployed your project, you will need to execute two commands via the Heroku toolbelt (cli):

1. heroku run python manage.py syncdb
2. heroku run python manage.py migrate

That’s it! Heroku will add a bunch of stuff to your settings.py to handle the production db configuration so you don’t need to worry about that.

For more details on configuring a Django project for Heroku, see Getting Started with Django on Heroku/Cedar.

7. Converting from sqlite to PostgreSQL

As I noted above, the project can use sqlite locally while Heroku will configure and run PostgreSQL in production. However, I needed to configure the project to run PostgreSQL locally so that I could confirm that Strider would work properly using PostgreSQL as the database. For this I had to change the settings.py DATABASE ENGINE to ‘django.db.backends.postgresql_psycopg2’, and I had to specify a username and password for the database. I also had to explicitly specify ‘localhost’ as the HOST even though the comment says that localhost is the default.

Lastly, one of my tests started failing after I cut over to PostgreSQL. I eventually determined that this is because sqlite always returns results from a table ordered by primary key whereas PostgreSQL does not unless explicitly ordered. I added the following to my polls model to fix this:

class Meta:     
  ordering = ['id']

[UPDATE: As Erik rightly points out in the comments, the reason I needed to add this ordering clause was because in my test, I am querying the db and storing the results in an array (via the ORM), and then checking the elements in the array *by their order in the array* to confirm that values in the other fields of each row are correct. In a real-world application, it is unlikely that the order of the results would matter in this way, so you should not have a test like this, in which case you would not need to add this ordering to the model which will order the results for each and every query. So, in summary, only add this ordering if you in fact really need it for at least many if not most of your queries.]

And that’s it! Do you have a tip for new users that you didn’t see on this list? Add it in the comments.


Stephen Bronstein is a co-founder of BeyondFog, the creators of StriderCD.com, an open source continuous delivery platform for Python and node.js.

 
  • Sean

    Great write up!

    The link for the testing blog post #1 is broken (I think you assumed the date in the URL was the same as post #2)

    • Anonymous

      Doh! Now fixed, thanks Sean.

    • Anonymous

      Doh! Now fixed, thanks Sean.

  • Anonymous

    I’m pretty sure that the tip in #7 about ordering is an anti-pattern.  The problem is that this will sort every single database query (add an ORDER BY to the SQL).  That can amount to quite a bit of extra overhead just for the ability to rely on default natural sorting in your tests (granted their is already an index on the primary key….).

    If I were you, I would remove this from your Model and modify your tests to not rely on any type of default sort (add an order_by to your ORM calls if you need it — explicit is better than implicit in Python

    • Anonymous

      Yeah, that is a valid point and one that I thought about when I was debugging the issue. With a real application and real data, it’s hard to imagine a situation where the key for each row would matter in and of itself as opposed to being used for joins and whatnot.

  • http://twitter.com/rogueleaderr George London

    Great post. If you’re trying to launch a Django site as fast as possible, try Pinax. Here’s a guide I wrote to getting a site up in an hour. http://rogueleaderr.tumblr.com/post/21769585437/setup-a-basic-publicly-accessible-website-in-an-hour

  • http://twitter.com/BlaShadow Luis Black

    nice tutorial!!!