Convention Over Configuration: Django Settings

I’ve been working with Django for over five years. Even though I don’t consider myself as a programmer, one of the advantages I seem to have over less experienced Django developers is that I’ve developed (together with our team) strong conventions on how I work. This makes developing faster and generally more enjoyable.

Conventions are a tool for efficient workflow and to help read and understand code by other people better. Like so many tools, conventions too can be taken to extremes to the point when then they become magic (Rails, anyone), which in Python community is considered a bad practice.

The Python community has a name for code that feels right: that kind of code is pythonic. Good, pythonic conventions are logical and easy to remember but also explicit. In other words good conventions should make sense when you see them first time.

Django has many documented conventions (like that models should live in models.py and views in views.py) and even coding guidelines but there are many places that could use more stronger or better ones. This article is first in a series of discussing conventions related to Django. My goal is not to impose these opinnions to anyone but to share what I’ve learned while working with Django for over five years.

Conventions for Django settings

Settings are not the sharpest tool in the Django shed. This article is not about crying over the design details, but to make working with them easier.

Always use relative paths

One of the first hurdles one comes up with Django settings is that they aren’t very portable. The MEDIA_ROOT, STATIC_ROOT and TEMPLATE_DIRS settings are problematic because the given examples are absolute paths (like "/home/media/media.lawrence.com/media/"). Simply by defining your paths as relative you can free yourself from this and move your projects around without breaking your settings.

A good convention is to define a PROJECT_ROOT setting and make every path in your settings relative to that root. Django settings file is just an ordinary Python module (well, not really, but in this respect it is) so you can use Python to do this:

 import os  PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../')) MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'media') 

Turn settings into a package

Second problem you almost immediately have is a problem of multiple settings files. You can make your life a lot easier by moving settings from ´settings.py´ into several different files in a directory called settings. (Remember that to python /path/to/your_project/settings.py and /path/to/your_project/settings/__init__.py are the same thing.)

Also, because there usually is at least two almost equal settings files (for development and production), it’s smart to differentiate common settings and the ones that change for every machine. For this, you can move common settings to common_settings.py and then just import that to your other settings files:

 from common_settings import * 

Now you can either add an empty __init__.py to the settings directory and refer to settings as DJANGO_SETTINGS_MODULE=myproject.settings.myenv or you can symlink the current environment settings as __init__.py.)

You could end up with a directory structure like this:

myproject/     manage.py     urls.py     settings/         __init__.py         common_settings.py         dev_adrian.py         dev_jacob.py         dev_simon.py         production.py         staging.py 

Now you have one file for common settings and one for each individual environment and they all can live together in version control.

Store sensitive data outside of settings files

Another problematic thing about Django settings is that you’re supposed to put things like passwords and API-keys in there. For development environment this seldomly is a problem but what about version control and different settings for staging and production environments?

At this point it is a no-brainer: just put all sensitive data to a file outside of version control and import that:

 from secrets import *

Now we don’t need to think about storing sensitive data in the version control and we just need to make sure that secrets.py is properly protected in the production environment.

Conclusion

Conventions are good and we need them. By using these few simple conventions you can make your Django projects more portable, secure and pythonic.

More important than the implemention details is that you follow some conventions. I’m a strong believer of customization and that you should make your own conventions that best fit you yourself or your team. (This, obviously, is not something you shouldn’t do when writing your first Django apps but instead when you have been working with Django a while so that you have a general feeling of what works for you and what doesn’t.)

Next in the series I’ll talk about arranging the overall project structure and the tools that help keeping your project evolving.

Meanwhile, any comments or tips that you’d like to share?

Offline Development With Django

Coming to Django from the PHP-world, running a local development server (as opposed to Apache or a full LAMP-machine set up for just testing) and doing real offline development is something that takes a little bit of learning. After two years of active development with Django, I’d like to share some of my learnings.

Why Offline?

There are many benefits for developing your site someplace other than the same server which powers the site. I’m sure most of us do development this way.

Generally speaking offline development could mean any development that doesn’t happen on the production server. The meaning for offline development in this article is more literal: by offline I mean literally offline, that is without [requiring] a connection to the Web.

A well configured development environment helps you write better code efficiently — anywhere.

On a side note, don’t blame me if you end up coding Django your whole winter vacation at an idyllic remote cottage 😉

Best Practices

In the same way that Django lets you separate models, templates and views, it also lets you easily separate production and testing environments. Django also offers several tools for local development, such as the built in Web server and DEBUG-mode. In short, Django encourages you to follow best practices.

One thing that I’ve been trying to unlearn is the PHP-esque way of doing small modifications on the production server and at the same time accidentally breaking the site from two other places. Luckily when using Apache and modpython the lure of doing this is a bit smaller because every modification to a python file needs a server restart. By keeping the development strictly off the production server, the probability of breaking something on the live site reduces significantly. (Because you _do want to test your changes before deploying, right?)

Loosely related to local development is version control. When working with version control, you generally don’t want to check in non-functional code. That means that you must test the code before you check it in. Having a local development environment helps with this 🙂

Prepare Your Site — Thoroughly

I’ve been doing web development for nearly ten years now. Everything I do goes trough dedicated testing servers and version control. I’ve always thought my sites to be well prepared for offline development. Then, in spring 2007, I had some problems switching ISPs and I was cut off from web for two weeks. (What a long two weeks they were 🙂 Turns out this was a very good thing since I discovered tons of problems while trying to work really offline.

Use the Settings, Luke

Using different settings for development and production makes it possible to do truly offline development. At work we keep different settings for every development machine in the project-root and separate them by naming convention of settings-hostname.py and settings-production.py. We then symlink the appropriate settings on the machine as settings.py and everything Just Works. You might also want to learn to keep things portable.

Furthermore, there is a good tip in Django documentation about limiting serving static files to DEBUG=True. The given example is a bit un-DRY, though. And it also adds the static url as last element of the URLconf, which sometimes just doesn’t work. Here is what I use:

 from django.conf.urls.defaults import * from django.conf import settings  if settings.DEBUG:     # Serve all local files from MEDIA_ROOT below /localmedia/     urlpatterns = patterns('',         (r'^localmedia/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.MEDIA_ROOT, 'show_indexes': True}),     ) else:     urlpatterns = patterns('',)  urlpatterns += patterns('',     # your urlpatterns here )

I don’t like the else-part because it looks ugly, but it works. Point is that you want to add the media path as a first urlpattern so it won’t get overwritten by any of your other urls.

Also remember to configure INTERNAL_IPS so you’ll get debug-variable to your templates. This template snippet is a good example how to use that debug data.

In addition to these built-in settings, I also use a custom LOCALDEV boolean for explicitly handling situations that may not work well without internet connection. This way I can just ignore things that should do something over the web when I want without breaking the site. (Ie. if DEBUG and LOCALDEV, don’t fetch this over the Web but use this fixed variable instead, etc.) The combination of these two settings add up to very easy to use and flexible development environment.

Avoid Hardcoding Media Files

For me, most typical problems with offline development come from media links (that is links to images, CSS and JavaScript). Often these files are stored on a separate server(s), like recommended in Django documentation. Instead of hardcoding the links (relative or absolute) to media files, you should use your settings file and let Django take care of the rest. If you’re using SVN-version of Django, you probably already have a default context processor that sets MEDIA_URL in your RequestContext.

If you’re using older version of Django, here is how to do this yourself:

 # myprojects/misc/mediacontext.py from django.conf import settings  def media_url(request):     """ Returns MEDIA_URL url to context."""     return { 'MEDIA_URL': settings.MEDIA_URL }  # and in settings.py TEMPLATE_CONTEXT_PROCESSORS = ("django.core.context_processors.auth", "django.core.context_processors.debug", "django.core.context_processors.i18n", "myproject.misc.mediacontext.media_url")

Now you have MEDIA_URL variable in all your templates that have been rendered with RequestContext instance. Generic views use RequestContext, but unfortunately helpers like render_to_response use Context, not RequestContext, so we’re out of luck there. Luckily these kind of things are really easy to come by in Django. One easy solution is to use a simple wrapper to render_to_response method.

To get your media links work offline, just put MEDIA_URL it in front of any media links like this: <img src="{{ MEDIA_URL }}images/myimage.png" alt="" />. That way the media links Just Work — also offline (if you have the needed media files locally on your machine, of course).

Note that this technique works only for content that is rendered via Djangos template-engine. This means that with CSS-files, for example, you have to separate the parts that have media URLs in them and render them with Django templates. This works great for medium and small sites, but on a high-volume sites you’ll definitely want to make different arrangements to let those static elements be delivered via a separate media server.

Use Sample Data

Local developing means also that you don’t have access to the production database. It’s often necessary to have some data in the database before you can do any development at all. Django provides you a way for setting basic initial data automatically after syncdb-command, and it also helps you move all your data across different databases via fixtures.

A fixture is a collection of data that Django knows how to import into a database. You can export your whole production (Django-)database (or just one app) to a fixture with dumpdata-command. You can then move this fixture to your local machine and import it with loaddata. This way you can easily make copies of your production database and use them in your local development.

I’ve found two compelling reasons for using fixtures with development:

  • Using a copy of real data from a live production server in development is great because that way you’ll be able to work with those kind of inputs from real users that you’d never dream up writing yourself in your tests — before something breaks and you have to. It also feels nice to work on a site with real content instead of endless lorem ipsum paragraphs.
  • Also, lately I’ve been developing small sites entirely offline and deployed them via fixtures; I start up with an empty site, test and iterate it offline, add data and finally dump it (mostly from sqlite), and then load it to MySQL or PostgreSQL on the production server. Being able to move data easily from one database backend to another is great!

Conclusions

Offline-development is all about agility and portability. By keeping things not dependent on any specific database, media server or development machine, you’re giving yourself more freedom. In addition to easier development, portability adds to easier deployment, too.

Django provides great tools for fully offline-development. Hopefully this post gave you some ideas why you’d want to do it. I’ve been trying to better my own developing practices with Django for over two years. I’ve learned a lot, but I also believe that there’s still much to learn. Your tips and experiences are more than welcome in the comments!