The Hidden Value of Magic (and Tailwind CSS)

As I’ve begun the full rewrite process of my Slipmat.io hobby project, I’m astounded how much I’ve been learning from all kinds of things every step of the way, by carefully analysing the old, and making lots of effort in engineering and designing the new. What follows is one of my latest learned lessons that connected some dots between 15 years of working with the Web.

Python is my favourite programming language, and I’ve always stayed firmly with the Django camp as opposed to Flask or any other gazillion alternatives. In the early days of Django (like 15 years ago) I also had lots of reservations against Rails, mostly because I was young and stupid, but also because it had too much magic. The more I’ve worked with JavaScript the more I’ve learned about good architecture and best practises from bad examples, and my views on magic have changed somewhat.

JavaScript is a fascinating language. ECMAscript 2020 is the ultimate lipstick on a pig. The core language is such an awful mess, yet the new syntax is pretty powerful and the modern tooling built around it is truly amazing and wonderful to work with. And as the tools have gotten so much better in a very rapid pace, the community is starting to get the libraries and coding conventions in a better shape as well.

Vue 3 is a great example of this. It’s built totally from ground up using TypeScript, designed with an open RFC process taking feedback from the whole community, and Evan You has done a lot of work shaping the end result into something that’s actually designed to be used and not only something that looks good.

The work on my new Vue 3 project template has advanced to list screens which need various list components, preferably in a form of a powerful datatable component. I had to completely drop the project for a while to get a better understanding what I’d really want here. Today I finally had an epiphany; the thing that’s most wrong in the current Slipmat design are the numerous different random third-party components that even though styled to look somewhat (but not exactly) similar, they all feel different. And this exactly is my issue with glue-in frameworks like Flask as well. Even though the various parts can be easily used together, there’s always something missing compared to tools like Django where everything is built to work together.

So now we get dangerously close to the infamous not invented here syndrome. If gluing up various third-party solutions yields bad results, then surely we should write all components ourselves? Well, no. Firstly, always prefer an existing ui library (like Vuetify for Vue/Material Design) over building anything yourself. Secondly, –and this is again something I realised today– properly architectured third-party components can be not just almost but precisely tweaked to fit your design. And whenever using them, you should take the time to do that design work properly. The main problem with inconsistent Web UIs is handwritten CSS. Whenever a non-designer needs to invent their own styles, the end result will end up looking something that almost matches the thing they did yesterday, but only almost. And this is where Tailwind CSS shows again how brilliant the utility-first idea really is; there is close to zero handwritten css needed for most Tailwind projects which automatically restricts you to a relatively small amount of professionally curated choices. The end result looks automatically not-very-bad and with some tweaking it’s not hard to get absolutely spot-on. (To be fair, it’s not easy either, but unlike trying to manually come up with CSS, it’s way easier!)

After I watched Adam Wathan build this select component, I was convinced that this would be the way I should build the few components (button, toggle-switch, modal, toast, data table / list component, datepicker, tag input) I need for the Slipmat rewrite.

(Yes. NIH. I’d love to use some third-party library here but at the moment there just aren’t too much to choose from for Vue 3 and Tailwind.)

Datatable experiment

The first iterations don’t need to be anything too fancy so I’m confident that building all of these from scratch instead of trying to modify third-party alternatives won’t be an issue. I’m also fully prepared to switch to using @tailwindui/vue or some other library in the future if maintaining these eats too much time. Just to get a feeling of what the API would maybe feel like I created an experiment project and pushed it to GitHub (not GitLab because on GH I get Dependabot updates automatically).

So this story has been a bit of a rambling journey from back in the early days of Web frameworks to Vue 3 and Tailwind CSS. What I’m finding common between all these projects is the meticulous attention to detail regarding almost every aspect of the tool, especially to API design. Great tools aren’t born by accident, they are engineered. And after working with JavaScript for few years now I’m starting to appreciate the metaprogramming powers of Python which makes it possible to write super clean —and somewhat magical— APIs. A good amount of magic really makes a difference in developer experience when it’s done right. I still do think that having a pluralization engine that automatically fetches one Octopus and many Octopi is way too much but I also do appreciate the attention to detail in Rails the more I get to see language restricted not-so-beautiful designs in JavaScript land.

So the lesson of the day: invest in API design and the most value from magic comes from those situations where you can’t see it.

Django Tip: Smarter Output From Management Commands

Have you ever tried to add a helpful output for showing the progress of a slow management command but all the output frustratingly appears in one go after the script has finished? You’ve tripped into something called output buffering and luckily it’s very easy to fix.

Use self.stdout instead of print in management commands

You should never use print inside Django management commands as they have a special self.stdout method that makes testing easier. This method frustratingly takes slightly different arguments than print, but it also has more features like styling. Here’s how you would do a simple row of dots that grows iteratively:

self.stdout(".", ending="", flush=True) self.stdout.flush()

Note the flush(). This is wat forces the output to be shown to the user before the script finishes. And the same with print:

print(".", end="", flush=True)

Both self.stdout and print will add newline to the output by default but you can disable that by explicitly setting the ending as an empty string. And when you flush your output manually, you’ll get nice and interactive console output for your management commands — voilà!

(Btw. If you have ever wondered about ENV PYTHONUNBUFFERED 1 in some of the Docker images for Python, this is why; disabling output buffering makes reading Docker logs way easier.)

On Flying Starts

What a start for a blog to end up on the homepage of Hacker News on the first day (with a short post about Django UI). The good: it’s surprisingly easy to install a cache plugin to WordPress. The bad: this blog is not hosted on the Slipmat.io server (because I don’t want to install PHP on it) so I missed a great chance to stress-test it.254 Concurrent Users in Google Realtime Analytics

To stay with the topic on flying starts, I’ve been quietly readying the first Slipmat.io tool, a site for taking and making requests in Mixify events, for its official launch. I opened the site for public three weeks ago and I’ve been working on it ever since. I’m keeping detailed public release notes to keep the users updated and also for myself as a way to get a feel of the progress.

Most non-DJs and furthermore non-Mixify users probably have no idea what this tool actually does, so let me explain it quickly. Mixify is a Web site where DJs can play for a live audience by streaming sound (and possibly also video) from their computer to Mixify which distributes it to listeners. Listeners can join these live events and chat with the DJ and other listeners. Many DJs take requests but it can sometimes be cumbersome trough the small chat window that can update really fast if there are lots of listeners actively chatting. MixifyRequests is a site where a DJ can make a request event and get the requests in a nice, organised way with a easy to use admin controls for marking requests played etc. Users can also vote for each others requests and there’s a bunch of other features, too. So that’s the idea in a nutshell.

MixifyRequests Homepage v0.9a

I did the first part of the development in one 10 day sprint, with the idea of getting the first alpha version online in whatever state it would be after 10 days. (Because real developers ship.) The first release had one big showstopper-bug[^1] which was due to my lazy testing of Facebook sign up process, but otherwise everything worked fine.

[^1]: Where users who signed up everywhere but from the homepage would get a 500 error after first login because the login view respected ‘next’ parameter that would redirect them past the view that created an app-spesific user profile of which everything on the site assumed to be there. Thanks to Sentry I was informed about the bug immediately and got a hotfix in place in about 20 minutes but it had already affected dozens of users of which few of them didn’t come back that first night.

I decided to do kind of a soft launch by only telling about the site to few selected DJs and by placing a disclaimer on top of every page with a note saying “This site is not officially launched yet. Use it and have fun, but please do not advertise this yet in bigger scale. Thanks!“. So far this has worked really well and the userbase has been steadily growing at a rate of about 1-3 new users every day. Thanks to a small number of real users actively using the site, I’ve gotten loads of good usage data, and most importantly, I’ve been able to find and fix good number of bugs emerging from situations only real users can get to. (It never ceases to amaze me how actual users always find ways to break code that was supposed to be tested really well.)

After the first sprint and the initial production push I changed the development cycle more towards daily productions pushes with the help of feature switches. Last week I sent out an email to all active DJs of the site asking volunteers as future betatesters. Got a few and now I can test and roll out new features in a controlled manner independently of production pushes. Feature switches enable me to keep the development branch in Mercurial very close to stable branch and push bugfixes,  not yet polished code and new features on the site daily without exposing anything to the public before the code is ready and tested (live with betatesters). When I want to publish a new feature, I go to the admin, flip a switch, and it’s live. Boom! Also, if there are any problems, I can just as easily flip the feature off again, and fix the problem with no hurry. Feature switches are awesome. There are lots of different apps for Django for doing this, I chose Gargoyle (mainly because couldn’t figure out how to get the newer version called Gutter working). Big props to Disqus folks for yet another awesome Open Source app.

If you’re interested in following the development of Slipmat.io site by this lone music loving Django developer, check out @slipmatio on Twitter, Slipmat.io on Facebook and, of course, keep in touch with this blog. And if you have been, thanks for reading this far!

(This post was originally posted to my other blog called Spinning Code.)

Django UI in 2005 vs Django UI in 2014

I stumbled across my favourite ever tech talk, the legendary Snakes and Rubies event where Django met Rails, back in 2005. One thing caught my eye while re-watching every minute of the 3-hour event: the Django Web site and the Admin Site app look almost exactly the same today than they did 9 years ago.

Screen capture image from Djangoproject.com as it was in 2005

Still image of Djangoproject.com from the Snakes and Rubies video, 2005.

 

Screen Shot 2014-09-17 at 22.50.28

Djangoproject.com, 2014.

 

Still image from Snakes and Rubies video

Still image of Django Admin Site from the Snakes and Rubies video, 2005.

 

Screen Shot 2014-09-17 at 22.58.44

A screencapture of one Django Admin app (in Finnish — it still talks also Welsh, among others), 2014.

 

I don’t want to comment this in any other way but to notice that it’s quite amazing that Djangoproject.com still looks so good and I actually like using it every single day — it works really well. Way better than most newer homepages of other open source projects.

Granted, the Admin app looks a bit more dated, but not too much. And for its purpose, it too works still really well.

Wilson Miner is one kick-ass designer.

And Wilson, it’s not just a Web site. 🙂

(This post was originally published on my other blog called Spinning Code.)

Redirect to Custom URL After Saving in Django Admin

Here’s a quick and dirty admin hack that can save you lots of time if you’re used to browsing between editing objects in admin and vieving the results on a live site.

Put following change_view-method in your admin.py and now you can make links to admin change page that return you back to where you came from:

 class BlogEntryAdmin(admin.ModelAdmin):      ...      def change_view(self, request, object_id, form_url='', extra_context=None):         result = super(BlogEntryAdmin, self).change_view(request, object_id, form_url, extra_context)         if request.GET.get('return_to', False):                         result['Location'] = request.GET['return_to']         return result 

In your template you’d have something like:

 {% if request.user.is_staff %}     <a href="/admin/blog/blogentry/{{ entry.pk }}/?return_to={{ entry.get_absolute_url }}">Edit in admin</a> {% endif %} 

Of course, if you’re smart, you should convert any places that need this kind of functionality with frontend editing tools so you can forget going to the Admin in the first place. But meanwhile, those few lines of code may come in handy!

Convention Over Configuration: Django Settings

I’ve been working with Django for over five years. Even though I don’t consider myself as a programmer, one of the advantages I seem to have over less experienced Django developers is that I’ve developed (together with our team) strong conventions on how I work. This makes developing faster and generally more enjoyable.

Conventions are a tool for efficient workflow and to help read and understand code by other people better. Like so many tools, conventions too can be taken to extremes to the point when then they become magic (Rails, anyone), which in Python community is considered a bad practice.

The Python community has a name for code that feels right: that kind of code is pythonic. Good, pythonic conventions are logical and easy to remember but also explicit. In other words good conventions should make sense when you see them first time.

Django has many documented conventions (like that models should live in models.py and views in views.py) and even coding guidelines but there are many places that could use more stronger or better ones. This article is first in a series of discussing conventions related to Django. My goal is not to impose these opinnions to anyone but to share what I’ve learned while working with Django for over five years.

Conventions for Django settings

Settings are not the sharpest tool in the Django shed. This article is not about crying over the design details, but to make working with them easier.

Always use relative paths

One of the first hurdles one comes up with Django settings is that they aren’t very portable. The MEDIA_ROOT, STATIC_ROOT and TEMPLATE_DIRS settings are problematic because the given examples are absolute paths (like "/home/media/media.lawrence.com/media/"). Simply by defining your paths as relative you can free yourself from this and move your projects around without breaking your settings.

A good convention is to define a PROJECT_ROOT setting and make every path in your settings relative to that root. Django settings file is just an ordinary Python module (well, not really, but in this respect it is) so you can use Python to do this:

 import os  PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../')) MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'media') 

Turn settings into a package

Second problem you almost immediately have is a problem of multiple settings files. You can make your life a lot easier by moving settings from ´settings.py´ into several different files in a directory called settings. (Remember that to python /path/to/your_project/settings.py and /path/to/your_project/settings/__init__.py are the same thing.)

Also, because there usually is at least two almost equal settings files (for development and production), it’s smart to differentiate common settings and the ones that change for every machine. For this, you can move common settings to common_settings.py and then just import that to your other settings files:

 from common_settings import * 

Now you can either add an empty __init__.py to the settings directory and refer to settings as DJANGO_SETTINGS_MODULE=myproject.settings.myenv or you can symlink the current environment settings as __init__.py.)

You could end up with a directory structure like this:

myproject/     manage.py     urls.py     settings/         __init__.py         common_settings.py         dev_adrian.py         dev_jacob.py         dev_simon.py         production.py         staging.py 

Now you have one file for common settings and one for each individual environment and they all can live together in version control.

Store sensitive data outside of settings files

Another problematic thing about Django settings is that you’re supposed to put things like passwords and API-keys in there. For development environment this seldomly is a problem but what about version control and different settings for staging and production environments?

At this point it is a no-brainer: just put all sensitive data to a file outside of version control and import that:

 from secrets import *

Now we don’t need to think about storing sensitive data in the version control and we just need to make sure that secrets.py is properly protected in the production environment.

Conclusion

Conventions are good and we need them. By using these few simple conventions you can make your Django projects more portable, secure and pythonic.

More important than the implemention details is that you follow some conventions. I’m a strong believer of customization and that you should make your own conventions that best fit you yourself or your team. (This, obviously, is not something you shouldn’t do when writing your first Django apps but instead when you have been working with Django a while so that you have a general feeling of what works for you and what doesn’t.)

Next in the series I’ll talk about arranging the overall project structure and the tools that help keeping your project evolving.

Meanwhile, any comments or tips that you’d like to share?

Controlling iTunes from Python

django-tunesBack in 2008 I stumbled on then new API on Mac OS X called Scripting Bridge that enables easy controlling of Cocoa apps (like iTunes) from scripting languages like Python.

I wrote simple proof-of-concept Django app that turned my iPod touch into a remote control. It was cool, even though it didn’t do very much. Now there’s of course native iOS apps for this but I was somewhat surprised that there are still not much code for controlling iTunes via Web. I misplaced my original code so I decided to write it again — just for fun.

So here’s a working class for controlling iTunes, in 58 lines of Python:

 # -*- coding: utf-8 -*- from Foundation import * from ScriptingBridge import *  class iTunes(object):     """     A helper class for interacting with iTunes on Mac OS X via Scripting     Bridge framework.      To use this, launch iTunes and make sure a playlist or an album is ready.      Usage:      >>> player = iTunes()     >>> player.status     'playing'     >>> player.current_track     u'Maison Rilax'     >>> player.current_album     u'Maison Rilax'     >>> player.current_artist     u'Lemonator'     >>> player.pause()     >>> player.status     'paused'     >>> player.play()     >>> player.next()     >>> player.current_track     u'Not Your Game'      """      def __init__(self):         self.app = SBApplication.applicationWithBundleIdentifier_("com.apple.iTunes")      def _get_status(self):         if self.app.playerState() == 1800426320:             return "playing"         elif self.app.playerState() == 1800426352:             return "paused"         else:             return "unknown"     status = property(_get_status)      def _get_current_track(self):         return self.app.currentTrack().name()     current_track = property(_get_current_track)      def _get_current_artist(self):         return self.app.currentTrack().artist()     current_artist = property(_get_current_artist)      def _get_current_album(self):         return self.app.currentTrack().album()     current_album = property(_get_current_album)      def _set_volume(self, level):         """         level should be an integer between 0-100.         """         self.app.setSoundVolume_(level)      def _get_volume(self):         return self.app.soundVolume()     volume = property(_get_volume, _set_volume)      def pause(self):         self.app.pause()      def play(self):         # According to AppleScript documentatin there should be a .play()         # method, but apparently there isn't. So we fake it :)         if self.status == "paused":             self.app.playpause()      def next(self):         self.app.nextTrack()      def previous(self):         self.app.previousTrack() 

This could be easily refined into something potentially interesting like a native REST API for iTunes. The downside, of course, is that it only works on a Mac. (The code is available also on BitBucket.)

A somewhat related is a project called Mopidy, which is a Python powered MPD music server for the awesome Spotify music service (that is suposed to finally be launching in US very soon, I hear.). I’m not at all familiar with MPD but I think it wouldn’t be very difficult to build a Django frontend to a server like Mopidy to get a very slick Web-based interface to your music. Now that would be cool.

EuroPython 2008, day 3

For me, the main theme for the third and last day of the conference was Django. Day started with a session by Honza Král about Django newforms admin, which gave me a pretty good idea what I have to do in preparing for the migration to the new admin interface. Later on the day I had many interesting discussions about varions Django-related things, including ideas about having something like Django release party here in Europe, too.

At Vilnius TV TowerTurns out there are quite a lot Djangonauts here in EuroPython, we just arent organized very well so most of us don’t know about each other. Lively after-conference discussions about career choises, Web Design and working with Django in general were definitely the high point of the conference so far.

A couple of hour later on the evening, decisions about quiet evening at the Sky Bar (at the 22nd floor of the conference hotel) were forgotten and, again, we headed for the old town. This time we ended up in some night club and, well, yeah, we had a great time 🙂

EuroPython 2008, day 2

I skipped most of the sessions for the day as they didn’t seem even remotely interesting to me. But, the ones I did attend where really good.

Last nights partying kept me pretty much in bed for better part of the morning. Descriptor tutorial by Raymond D. Hettinger was very good and I think it was actually the first conference tutorial ever that actually teached me something about programming and Python in general.

There was quite a bit testing-related material on the Lightning talks. One thing I wrote down on my notes was the phrase “Given enough tests, all bugs are shallow” (originally from Linus Torvalds in a form “Given enough eyeballs, all bugs are shallow”).

The keynote talk for the day was something that absolutely blew my mind. Hans Rosling talked about Gapminder and how statstical data and databases should be free. First thing that crossed my mind when listening to him was that “I wonder if Adrian Holovaty has ever talked to this guy — they’d have much to talk about”. If you havent heard about Gapminder or Hans Rosling before, you should definitely see his talk on last years TED conference. In Django-terms, thats some cool shit 🙂

EuroPartyPython 2008, night 2

Smile!

After Roslings inspiring keynote there was the conference dinner. We ate well and headed again to down town Vilnius. (Todays dream team was Edgars, Nicolas and me.) The two girls from last night showed up as they had promised. We had great time with them for a couple of hours untill they left early. We stayed at the bar and eventually hooked up with three cute Lithuanian girls. Lots of dancing, beer and fun until the bar closed at 3 am. What a great night — again!