Quick Guide To Using python-twitter

Twitter doesn’t have an official Python client for it’s REST API (which requires authentication for every query) but there are plenty of third party options. One of them is python-twitter, which is implements full API but lacks a decent documentation.

Here’s a full and working code on how to get a list of public tweets for a given screen name and an example of how to convert tweet dates from strings to datetime objects:

 import twitter  # You'll find these from https://apps.twitter.com/ access_token_key = ''      # Twitter API key access_token_secret = ''   # Twitter API secret consumer_secret = ''       # Twitter Access token consumer_key = ''          # Twitter Access token secret  # this functions needs 4 parameters or it will raise following exception: # AttributeError: 'Api' object has no attribute '_Api__auth' api = twitter.Api(     consumer_key=consumer_key,     consumer_secret=consumer_secret,     access_token_key=access_token_key,     access_token_secret=access_token_secret )  # this follows the REST API reference: https://dev.twitter.com/rest/public timeline = api.GetUserTimeline(screen_name='djUninen', exclude_replies=True)  for tweet in timeline:     print tweet.text  # finally, tweet.created_at returns a string, not a timestamp # to convert the string into a datetime object, do this: from datetime import datetime tweet_timestamp = datetime.strptime(tweet.created_at, "%a %b %d %H:%M:%S +0000 %Y") 

One last exotic error to be aware of is an exception which says “Timestamp out of bounds”. It means that the environment clock is too far away from the actual time. For example if you’re running a local virtual machine which messes up its clock when the host goes to sleep, you might see this. (Note to self: how about finding a permanent fix for the god damn clock.)

(This post was originally posted to my other blog called Spinning Code.)

Why I Love Python And Hate Java

Java and Python processes

These backgroundtasks running Java processes — after tuning for minimun memory consumption — eat 20 times more memory than my whole Python Web app. Nice.

(This post was originally published on my other blog called Spinning Code.)

Convention Over Configuration: Django Settings

I’ve been working with Django for over five years. Even though I don’t consider myself as a programmer, one of the advantages I seem to have over less experienced Django developers is that I’ve developed (together with our team) strong conventions on how I work. This makes developing faster and generally more enjoyable.

Conventions are a tool for efficient workflow and to help read and understand code by other people better. Like so many tools, conventions too can be taken to extremes to the point when then they become magic (Rails, anyone), which in Python community is considered a bad practice.

The Python community has a name for code that feels right: that kind of code is pythonic. Good, pythonic conventions are logical and easy to remember but also explicit. In other words good conventions should make sense when you see them first time.

Django has many documented conventions (like that models should live in models.py and views in views.py) and even coding guidelines but there are many places that could use more stronger or better ones. This article is first in a series of discussing conventions related to Django. My goal is not to impose these opinnions to anyone but to share what I’ve learned while working with Django for over five years.

Conventions for Django settings

Settings are not the sharpest tool in the Django shed. This article is not about crying over the design details, but to make working with them easier.

Always use relative paths

One of the first hurdles one comes up with Django settings is that they aren’t very portable. The MEDIA_ROOT, STATIC_ROOT and TEMPLATE_DIRS settings are problematic because the given examples are absolute paths (like "/home/media/media.lawrence.com/media/"). Simply by defining your paths as relative you can free yourself from this and move your projects around without breaking your settings.

A good convention is to define a PROJECT_ROOT setting and make every path in your settings relative to that root. Django settings file is just an ordinary Python module (well, not really, but in this respect it is) so you can use Python to do this:

 import os  PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../')) MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'media') 

Turn settings into a package

Second problem you almost immediately have is a problem of multiple settings files. You can make your life a lot easier by moving settings from ´settings.py´ into several different files in a directory called settings. (Remember that to python /path/to/your_project/settings.py and /path/to/your_project/settings/__init__.py are the same thing.)

Also, because there usually is at least two almost equal settings files (for development and production), it’s smart to differentiate common settings and the ones that change for every machine. For this, you can move common settings to common_settings.py and then just import that to your other settings files:

 from common_settings import * 

Now you can either add an empty __init__.py to the settings directory and refer to settings as DJANGO_SETTINGS_MODULE=myproject.settings.myenv or you can symlink the current environment settings as __init__.py.)

You could end up with a directory structure like this:

myproject/     manage.py     urls.py     settings/         __init__.py         common_settings.py         dev_adrian.py         dev_jacob.py         dev_simon.py         production.py         staging.py 

Now you have one file for common settings and one for each individual environment and they all can live together in version control.

Store sensitive data outside of settings files

Another problematic thing about Django settings is that you’re supposed to put things like passwords and API-keys in there. For development environment this seldomly is a problem but what about version control and different settings for staging and production environments?

At this point it is a no-brainer: just put all sensitive data to a file outside of version control and import that:

 from secrets import *

Now we don’t need to think about storing sensitive data in the version control and we just need to make sure that secrets.py is properly protected in the production environment.


Conventions are good and we need them. By using these few simple conventions you can make your Django projects more portable, secure and pythonic.

More important than the implemention details is that you follow some conventions. I’m a strong believer of customization and that you should make your own conventions that best fit you yourself or your team. (This, obviously, is not something you shouldn’t do when writing your first Django apps but instead when you have been working with Django a while so that you have a general feeling of what works for you and what doesn’t.)

Next in the series I’ll talk about arranging the overall project structure and the tools that help keeping your project evolving.

Meanwhile, any comments or tips that you’d like to share?

Controlling iTunes from Python

django-tunesBack in 2008 I stumbled on then new API on Mac OS X called Scripting Bridge that enables easy controlling of Cocoa apps (like iTunes) from scripting languages like Python.

I wrote simple proof-of-concept Django app that turned my iPod touch into a remote control. It was cool, even though it didn’t do very much. Now there’s of course native iOS apps for this but I was somewhat surprised that there are still not much code for controlling iTunes via Web. I misplaced my original code so I decided to write it again — just for fun.

So here’s a working class for controlling iTunes, in 58 lines of Python:

 # -*- coding: utf-8 -*- from Foundation import * from ScriptingBridge import *  class iTunes(object):     """     A helper class for interacting with iTunes on Mac OS X via Scripting     Bridge framework.      To use this, launch iTunes and make sure a playlist or an album is ready.      Usage:      >>> player = iTunes()     >>> player.status     'playing'     >>> player.current_track     u'Maison Rilax'     >>> player.current_album     u'Maison Rilax'     >>> player.current_artist     u'Lemonator'     >>> player.pause()     >>> player.status     'paused'     >>> player.play()     >>> player.next()     >>> player.current_track     u'Not Your Game'      """      def __init__(self):         self.app = SBApplication.applicationWithBundleIdentifier_("com.apple.iTunes")      def _get_status(self):         if self.app.playerState() == 1800426320:             return "playing"         elif self.app.playerState() == 1800426352:             return "paused"         else:             return "unknown"     status = property(_get_status)      def _get_current_track(self):         return self.app.currentTrack().name()     current_track = property(_get_current_track)      def _get_current_artist(self):         return self.app.currentTrack().artist()     current_artist = property(_get_current_artist)      def _get_current_album(self):         return self.app.currentTrack().album()     current_album = property(_get_current_album)      def _set_volume(self, level):         """         level should be an integer between 0-100.         """         self.app.setSoundVolume_(level)      def _get_volume(self):         return self.app.soundVolume()     volume = property(_get_volume, _set_volume)      def pause(self):         self.app.pause()      def play(self):         # According to AppleScript documentatin there should be a .play()         # method, but apparently there isn't. So we fake it :)         if self.status == "paused":             self.app.playpause()      def next(self):         self.app.nextTrack()      def previous(self):         self.app.previousTrack() 

This could be easily refined into something potentially interesting like a native REST API for iTunes. The downside, of course, is that it only works on a Mac. (The code is available also on BitBucket.)

A somewhat related is a project called Mopidy, which is a Python powered MPD music server for the awesome Spotify music service (that is suposed to finally be launching in US very soon, I hear.). I’m not at all familiar with MPD but I think it wouldn’t be very difficult to build a Django frontend to a server like Mopidy to get a very slick Web-based interface to your music. Now that would be cool.

EuroPython 2008, day 2

I skipped most of the sessions for the day as they didn’t seem even remotely interesting to me. But, the ones I did attend where really good.

Last nights partying kept me pretty much in bed for better part of the morning. Descriptor tutorial by Raymond D. Hettinger was very good and I think it was actually the first conference tutorial ever that actually teached me something about programming and Python in general.

There was quite a bit testing-related material on the Lightning talks. One thing I wrote down on my notes was the phrase “Given enough tests, all bugs are shallow” (originally from Linus Torvalds in a form “Given enough eyeballs, all bugs are shallow”).

The keynote talk for the day was something that absolutely blew my mind. Hans Rosling talked about Gapminder and how statstical data and databases should be free. First thing that crossed my mind when listening to him was that “I wonder if Adrian Holovaty has ever talked to this guy — they’d have much to talk about”. If you havent heard about Gapminder or Hans Rosling before, you should definitely see his talk on last years TED conference. In Django-terms, thats some cool shit 🙂

EuroPartyPython 2008, night 2


After Roslings inspiring keynote there was the conference dinner. We ate well and headed again to down town Vilnius. (Todays dream team was Edgars, Nicolas and me.) The two girls from last night showed up as they had promised. We had great time with them for a couple of hours untill they left early. We stayed at the bar and eventually hooked up with three cute Lithuanian girls. Lots of dancing, beer and fun until the bar closed at 3 am. What a great night — again!

Setting Up Mac OS X Leopard for Web Development (in just two steps)

Some time ago I wrote how to set up Mac OS X Tiger for Web development. Since the entry has been a good starting point for myself, I decided to re-write it for Leopard.

One step forward, two steps back

The OS X cat has evolved hugely from developers point of view. Things like Apache 2, PHP 5, Python 2.5, easy_install, Subversion and Capistrano work out of the box. So we’re almost there, right? Wrong.

Unfortunately some aspects of the operating system are as bad as before, so in lack of working package management almost everything on top of the default setup means manual compling. And like that wouldn’t be bad enough, Leopards ingenious mixture of 64 bit and 32 bit binaries makes getting diferent programs and libraries work together unbelievably hard and complicated.

Installing MySQL from source is easy. Installing mod_python is a bit more complicated, but still possible. Trying to get Apache, mod_python and MySQL work together without segfaulting is nearly impossible — but still doable. And after that there is more PITA everywhere. Compiling things and trying to fix several dependencies by hand is just plain stupid. It’s so last decade that even a Mac-using developer shouldn’t need to do it.

And, luckily, you don’t have to!

Installing the perfect development environment

So after testing various diferent setups on my new iMac, here is the “there is no step three” formula for setting up a decent development environment for your Leopard:

  1. Buy VMWare Fusion or Parallels Desktop (They’re both great, just choose which one you like best.)
  2. Install Ubuntu on it.
  3. There is no step three!

Now you can mount your real Linux box disk to the crappy Finder and use it like it was the real thing. This way you make both Mark Pilgrim and yourself happy!

PS. Happy New Year!
PPS. Yes, I still love my Mac 🙂 But I hate it when it tries to make me do stuff that I really don’t want to. Like installing and configuring stuff from source. Ewww.

Using Jaiku API with Python

I’ve slowly fallen in love with Jaiku. It’s a microblogging platform much like Twitter, but with fun features like ability to add feeds from other services (like Flickr, Ma.gnolia.com, Last.fm or any arbitrary source) to your stream and a nifty client for Series 60 phones. Jaiku also has an open API for easy hacking. I was amazed how easy it was to use.

My goal was to get my current presence data and location from Jaiku. After 30 minutes of coding (of which I watched the Simpsons about 20 minutes) I had the following 25 lines of code in my jaiku.py:

 from django.utils import simplejson  from urllib import urlopen from time import strptime from datetime import datetime, timedelta  JAIKU_USERNAME = 'uninen'  def get_jaiku_presence():     """Returns user Jaiku presence as dict or False if errors."""      url = 'http://%s.jaiku.com/presence/last/json' % JAIKU_USERNAME      try:         result = simplejson.load(urlopen(url))          # pythonize needed fields         time = strptime(result['created_at'], "%Y-%m-%dT%H:%M:%S GMT")         result['created_at'] = datetime(*time[:5]) + timedelta(hours=3)         result['comments'] = len(result['comments']) or False          return result      except IOError:         return False

It’s very simple and dead easy to use:

In [1]: from unessanet.misc.jaiku import get_jaiku_presence  In [2]: get_jaiku_presence() Out[2]:  {u'comments': False,  u'content': u'Savitehtaankatu, Turku, Finland',  u'created_at': datetime.datetime(2007, 8, 17, 0, 20),  u'created_at_relative': u'3 hours, 8 minutes ago',  u'icon': u'',  u'id': u'9552365',  u'location': u'Savitehtaankatu, Turku, Finland',  u'title': u'Enjoying the rain',  u'url': u'http://Uninen.jaiku.com/presence/9552365',  u'user': {u'avatar': u'http://jaiku.com/image/13/avatar_32113_t.jpg',            u'first_name': u'Ville',            u'last_name': u'S\xe4\xe4vuori',            u'nick': u'Uninen',            u'url': u'http://Uninen.jaiku.com'}}

Feel free to modify this to your needs.

Next up is a system for storing these presences to a local database and drawing a location trail to Google Maps or something similar…

PS. My Jaiku stream is at uninen.jaiku.com 🙂

It Could Be Worse

After a long-awaited summer vacation (including an unforgettable visit to Norway) I’m back to sitting indoors during these amazingly beautiful summer days.

This monday felt truly a monday until I saw this blogpost revealing the code for Facebook home page. Man, that gives one perspective. It’s been only about eighteen months since I dropped PHP for Python. The switch has been almost as fulfilling as the switch to my first Mac back in the day. Seeing that ugly and messy PHP code just made me smile; at least I don’t have to put up with anything like that anymore!

This is not to bash PHP or PCs (although they both suck) but to remind me that it’s really great to be able to work on interesting projects with Python and Django. It could be much worse. I just forget it much too often.

iPhone for Nerds

Remember my rant from January about how Nokia sucks? I was basically really upset about the fact that Apple could enter the phone business with something as cool as iPhone when Nokia and others have been making these really crappy phones for decades. Well, couple a weeks ago I got a new phone; Nokia N95.

Two Alike

There are not so many real competitors for the iPhone. By real competitor I mean a phone that could be seriously thought as an “iPod-killer”, a multimedia (video, imaging) device and a smartphone with good networking capabilities. Nokia N95 is definitely one of those few devices. While it doesn’t have a pretty UI like the iPhone, it does, however, pack an impressive feature set that outperforms iPhone in many areas.

Much of the criticism against the iPhone is about its price. Compared to N95, iPhone seems almost cheap. Well, almost. The less expensive iPhone is yours for $500 bucks. The high-end model sets you back $600. And Nokia N95? It’s around \$750. That’s sevenhundredandfifty dollars — for a phone. So please, iPhone is not the only expensive phone on the market.

(For the record, I’m not that rich. In Finland you can get a N95 with a two year contract for zero euros. I love contracts 🙂

In addition to rich feature set and high price point, these phones do have at least a one more thing in common: their battery life is ridiculously low. Of course, no one knows exactly how long the battery will last on the iPhone, but it is obvious that during intensive use it’s not going to be days but hours. And when it’s not days, it pretty much means that you must charge it every single day. Which sucks big time.

Nokia N95 In A Nutshell

  • 2.6″ screen (320×240)
  • 160 Mb onboard memory, support for up to 4 Gb microSD card
  • Dedicated keys for controlling media playback
  • Really Good 5 Mpix camera
  • Wlan, 3G (UMTS)
  • Built in GPS (with Nokia Maps)
  • Runs on Series 60 3rd Edition FeaturePack 1

For a nerd, the most significant differences to iPhone (other than the UI, of course) are N95s better connectivity features, built in GPS and the fact that Series 60 platform is absolutely great for hacking. (And there are lots of free and open source software available. Real software.)

By the way, the Safari browser on iPhone is using the same open source Webkit-engine that is powering Series 60 browser too.

Mobile Hacking

Apple issued a press release earlier this month stating that “evolutionary iPhone™ will run applications created with Web 2.0 Internet standards”. Yippee! So, for iPhone you can build shitt.. err. shiny interfaces on web applications and link phone numbers from them. (Want to know a Big Secret? href="tel:+5551234"). Man, that’s so cool! Or at least it was back in 2000.

I really don’t know which puts me off more; Nokia and others making really bad UIs and half-baked software on top of a decent operating system, or Apple making uber cool “runs on OS X” iPhone and crippling it with no SDK.

Series 60 phones have their downsides, but for developers the platform is wide open. In addition of dozens of open source programs that you can learn from, there are things like Python for Series 60 that lower the barrier for mobile development significantly. Furthermore, projects like Apache 2 port for Series 60 have opened amazing possibilities of which almost none have been explored yet.

Nokia alone has many other cool open source projects running and there are lots of people working on all sorts of cool hacks. (For example the RaccoonOnMap, which is a Mobile Web Server and Google Maps mashup that puts your phone on a map).

Getting a developer certificate for Series 60 3rd edition is, albeit free, a bit pain in the butt. But now that I have one (after few hours of installing stupid win-only software on Virtual PC and fighting trough the registration process), I’ll definitely try some Python hacking on my N95. I still think it’s a shame that you cannot do anything like this on an iPhone. Sure, it’s a smartphone “for the rest of us”, but if it really runs OS X, why not open it up for developers?

Until that happens, I guess the N95 is the iPhone for nerds.

Setting Up Mac OS X Tiger for Web Development

Setting up a new Mac as a web development workstation takes a little bit more work than just installing Textmate. These are my notes of the process (mainly for myself to make future installations faster).

The Basics

  • Install TextMate
    • Configure Finder to open ALL possible coding-related documents with TM
    • Install also the terminal extention. Test by running mate foo.txt in terminal
    • Install or sync (with other machines) all necassary bundles. They are located in ~/Library/Application Support/Textmate
  • Install Transmit
    • Copy settings from other machines ~/Library/Preferences/com.panic.Transmit3.plist
    • Sync bookmarks
  • Install SSHKeychain
    • The Universal binary is there. Just not very well in sight.
  • Install Subversion
  • Install MySQL
  • PostgreSQL, if needed
  • Install PHP for bundled Apache 1.3

Setting Up Python

Setting Up Django

I like to live dangerously. So:

 svn co http://code.djangoproject.com/svn/django/trunk/ django_src ln -s /path/to/installation_directory/django_src/django /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/django

That’s it. Django works. Now add export PYTHONPATH=/where/you/develop to your .bash_profile and finally copy django_src/django/bin/django-admin.py to somewhere in your path (for example in /usr/local/bin).

Localization issues

Are We There Yet?

Unfortunately that’s just the nerdy part of the setup-process. Next up is installing browsers, browser plugins (did I hear Firebug?) and all that jazz. I’m not going to get into those in this post, though.

I recenty listed all my favourite apps in iusethis.com (you can log in via OpenID). Please share your comments, suggestions or own favourite apps in the comments.