Building An Open Graph Image Builder (Part 1)

I’ve been slowly updating this blog to the modern world from the ages when people wrote their XML-feeds by hand (yes, it was a thing). One of the “new” things Web sites nowadays have in their meta tags are Open Graph preview images (or social previews). These images are used to make preview cards in various places, for example when posting links to Twitter and other sites.

There are tools like Pablo for creating these images online. Recently I’ve switched to creating them manually with Pixelmator but I’ve long been looking for a Web-based solution that could be integrated better with my other blogging workflows.

Two days ago I happened to stumble on to ZEIT Now’s blog post Social Cards as a Service where they dogfood their own serverless platform by building a simple image generator tool that uses Puppeteer to render the images. I immediately dove in to the source code hoping that I could modify it to my own needs but it looked way too complicated for my taste. (In other words, I didn’t understand most of it.) The UI got me thinking that building a simple framework around Tailwind would probably be super easy and would allow lots more control. So I sat down for two evenings and came up with Open Graph Image Builder.

This is still a work in progress and I’m not sure how far I want to develop this but so far it has been a fun little project.

The Toolchain

My absolutely favourite frontend tool for simple projects like this has been Vue CLI. Problem with that is that as soon as you start thinking about publishing something, you’ll have gazillion things to glue together and configure before you have a production-ready site. (I should probably invest some time into a proper cookiecutter template…) This exactly is why I recently wrote Tulip, a simple one page starter template for Gridsome (which is just Vue with some JAMstack flavor).

Tulip ships with Tailwind CSS preconfigured so I was immediately ready to build something interesting.

The third weapon of choise was ZEIT Now. I’ve been following their journey for a long time because of Simon Willison but never actually tried the platform. As it was their inspiration that led me to this path it was only appropriate to try out the service with this project. The first installation and publishing workflow via the deploy button in Tulip repository was unbelievably easy, even easier than Netlify if it’s even possible. (Although I did have an issue with GitLab import. It didn’t work so I pushed my code in GitHub instead and after that everything has been smooth sailing.)

Example output:

Tulip starter for Gridsome preview

To Be Continued…

The project is just getting started and I want to learn the ZEIT platform a bit more to be able to write about it better. I’ll also probably implement a proper backend rendering for the images so it’ll be a bit more useful in practise.

In part 2 of this series I’ll dive deeper into the code, going through some of the challenges and design decisions along the way.

In the mean time, check out the project source code and follow me on Twitter!

Howto: Draft Posts With Gridsome

Gridsome is a Vue-based static site generator inspired by Gatsby. As of version 0.7 there is no official way to have a publishing flow that separates draft and published posts. For me this is a big deal as I like to iterate on my writings for several days or weeks before publishing them. Luckily, there is an easy way to add support for draft posts yourself.

Adding a New GraphQL Property

At the heart of a Gridsome site is a GraphQL data layer that you can use and manipulate in development mode. Gridsome fetches any sources you define into this GraphQL storage in development mode and during the build process writes the data into static files as instructed in your configuration.

The naive way of importing any data into Gridsome just adds it into the storage and it then respectively gets published as you build the site. A super simple way to implement draft functionality is to simply add an attribute to your data object and then filtering by this attribute when building the site.

Add the following javascript in a file called gridsome.server.js in the root of your Gridsome project:

module.exports = function(api) {   api.loadSource(store => {     if (process.env.NODE_ENV === 'production') {       const posts = store.getContentType('Post')        posts.data().forEach(node => {         if (node.published !== true) {           posts.removeNode(node.id)         }       })     }   }) }

(Change the Post ContentType to match whatever your source plugin typeName has been configured to in gridsome.configjs plugins section. The default for source-filesystem-plugin is FileNode.)

Now, when you run gridsome build, the API that loads your sources at build time filters out each post that hasn’t got a published-attribute that evaluates to true. (Note that if you have a blog with lots of content and you don’t want to add a new attribute to all of them, you might want to reverse the logic, for example to if (node.draft !== true). I like to explicitly mark published posts, YMMV.)

For Markdown and Front Matter content you can now add published: false to your post meta to be able to filter out draft posts in production. (Modifying other content sources is left to the reader.)

My Goals for 2020

I don’t usually write (publicly) down any new year’s resolutions but as I feel like I’ve been finding more and more excuses of writing less and finishing fewer hobby projects, some public peer pressure would help me to maintain some focus. So here we go; my tech-related goals for the year 2020.

1. Publish Weekly Blog Posts

On this one I have to take a somewhat late start as January almost flew by already, but I’m targetting to write to this blog at least ~50 new posts this year. I love writing and I do write quite a lot but I’ve lost the habbit of doing it in the form of blog posts — and I want to get that habbit back.

I have tons of stuff in various draft posts and even more ideas about things to write about and I’m looking forward to make myself find a good publishing rhythm after a long hiatus. So stay tuned! 🙂

2. Release / Document More Hobby Projects

One of my biggest problems is that I love digging into new technologies and starting something but then never actually getting anything published (nor documenting the journey). I want to learn to be better especially in the documenting part as I know that it’s where I can contribute the most (compared to actual coding, for example).

3. Release At Least Two Serious Side Projects

I’ve been working on a side project called Slipmat.io for almost four years without actually committing to getting it properly published. I’ve also sat on a good idea (in my opinnion) without moving it forward for a couple of years now and I really want to get both of these projects in a better place.

While Slipmat is more of a hobby project, the second idea is something that could actually be turned into a proper SaaS business if done correctly. Both of these require a lot of time to complete so I really need to learn to split things up in smaller iterations and move faster if I’m going to pull this off. I’ll be documenting both of these journeys here, so again, stay tuned.

4. Attend To a New Tech Conference

I’ve been attending to Python-related conferences since 2007. This year I want to find a new non-Python conference to find different perspectives into the world of modern software development.

5. Fix My Tooling

When I was younger, I was constantly obsessing over my tooling to make my workflows faster and better. Lately I’ve noticed that my hatred towards computers and technology has grown so big that I’m usually just happy if my tools even start. The Web tooling has gotten so big and complex that it takes serious time and effort to keep everything up to date — not to mention of trying to optimize things.

I want to make some time and rebuild some very basic stuff (starting with dotfiles) around my personal tooling, document them better and start to get back on top of things again. Continuous improvement, Kaizen, is something that I’m really passionate about so this is again a goal that is hopefully going to lead to some interesting and fullfilling paths.

So, there are my five fairly ambitious goals for 2020. Now, it’s time for some action!

Django Tip: Smarter Output From Management Commands

Have you ever tried to add a helpful output for showing the progress of a slow management command but all the output frustratingly appears in one go after the script has finished? You’ve tripped into something called output buffering and luckily it’s very easy to fix.

Use self.stdout instead of print in management commands

You should never use print inside Django management commands as they have a special self.stdout method that makes testing easier. This method frustratingly takes slightly different arguments than print, but it also has more features like styling. Here’s how you would do a simple row of dots that grows iteratively:

self.stdout(".", ending="", flush=True) self.stdout.flush()

Note the flush(). This is wat forces the output to be shown to the user before the script finishes. And the same with print:

print(".", end="", flush=True)

Both self.stdout and print will add newline to the output by default but you can disable that by explicitly setting the ending as an empty string. And when you flush your output manually, you’ll get nice and interactive console output for your management commands — voilà!

(Btw. If you have ever wondered about ENV PYTHONUNBUFFERED 1 in some of the Docker images for Python, this is why; disabling output buffering makes reading Docker logs way easier.)

Hello Gridsome!

The publishing frequency of this blog has been somewhat .. irregular to say the least. The previous attempt to bring it to life didn’t quite catch on in the end as a positive problem called life got in the way. But now there’s been just too long break from blogging in English for me so I had to revitalize this project once more. Cue Gridsome.

Gridsome is an interesting static site generator built for Vue. It can read all kinds of data sources from REST APIs to Markdown files and then build the end result as a fully static site that can be deployed very easily pretty much anywhere. I opted for Netlify as I’ve found it super easy and reliable way to host static sites straight from GitLab.

The migration from a fully dynamic Django site to a fully static Vue site was a somewhat interesting experience, I’m planning to write about it in near future. But for now, here’s to yet another awakening of the Hoyci Blog!

Awakenings

This site and this blog is back from a four year vacation! Unfortunately this was not an easy one; I had a pretty bad burnout. Turns out I’m not the only one in the Python and Webdev community either, which is pretty sad. But, I’m back again 🙂

Old site ran on a dedicated server and was built with so old version of Django that I figured out it was easier to start from scratch and rewrite everything with 1.11 and Python 3. So here we are, a little bit rough around the edges, but mostly there. Stories about porting this stuff probably coming up in near future.

I’ve also got two new hobbies during this time: DJing and AI. Well, the first one is a proper hobby, latter is just something I’ve started to follow really closely. So closely in fact, that I decided to put it in the blog description. Some AI-related things to be expected here as well then.

Small Things Matter

Usability is a big topic but sometimes it can be very small things. Say, one line of JavaScript (well, jQuery in this case):

$('my_important_input_field').focus()

This is a small big thing that is ridiculously easy to implement and yet so underused.

I’m currently slowing down the feature development on the MixifyRequests-site and concentrating on little finishing touches like this in preparation of an official 1.0 launch. I already did one release concentrating on reducing technical debt (by improving the codebase in general, adding more tests, etc.) and now I’m kind of doing the same with UI and copy.

This is usually the part of software projects that gets neglected the most due to tight deadlines and budgets, so it’s fun to be able to do it with no hurry.

(This post was originally posted to my other blog called Spinning Code.)

How Real Life Data Can Kill An Idea

I had an idea to add tweets from registered DJs to the site homepage to give it more fresh and dynamic feel. So I hacked together a small script that fetched latest tweets from all of the DJs using Twitter API. After taking a quick look of the data, it was quite obvious that in practise, this wouldn’t be a very good addition to the homepage.

This is what I saw:

Database Screenshot of Tweets

When you think about it, it makes sense. Most DJs aren’t really interested in Twitter or other social media (except sites like SoundCloud and Mixcloud, which in contrast are very important and much used), they just feel that they need to be there because everyone else is and promoting yourself is hard. Their tweets are mostly just plain adverts of upcoming gigs and very little actual content. DJs create their content as music, not as tweets.

This is actually a good example why it’s important to have deep understanding of the field in order to being able to create good services for it. I’m not yet very familiar with the whole DJ scene yet, which is why I often ask dumb questions from experienced DJs who have been in the business for decades. But I’m learning every day.

(This post was originally posted to my other blog called Spinning Code.)

Quick Guide To Using python-twitter

Twitter doesn’t have an official Python client for it’s REST API (which requires authentication for every query) but there are plenty of third party options. One of them is python-twitter, which is implements full API but lacks a decent documentation.

Here’s a full and working code on how to get a list of public tweets for a given screen name and an example of how to convert tweet dates from strings to datetime objects:

 import twitter  # You'll find these from https://apps.twitter.com/ access_token_key = ''      # Twitter API key access_token_secret = ''   # Twitter API secret consumer_secret = ''       # Twitter Access token consumer_key = ''          # Twitter Access token secret  # this functions needs 4 parameters or it will raise following exception: # AttributeError: 'Api' object has no attribute '_Api__auth' api = twitter.Api(     consumer_key=consumer_key,     consumer_secret=consumer_secret,     access_token_key=access_token_key,     access_token_secret=access_token_secret )  # this follows the REST API reference: https://dev.twitter.com/rest/public timeline = api.GetUserTimeline(screen_name='djUninen', exclude_replies=True)  for tweet in timeline:     print tweet.text  # finally, tweet.created_at returns a string, not a timestamp # to convert the string into a datetime object, do this: from datetime import datetime tweet_timestamp = datetime.strptime(tweet.created_at, "%a %b %d %H:%M:%S +0000 %Y") 

One last exotic error to be aware of is an exception which says “Timestamp out of bounds”. It means that the environment clock is too far away from the actual time. For example if you’re running a local virtual machine which messes up its clock when the host goes to sleep, you might see this. (Note to self: how about finding a permanent fix for the god damn clock.)

(This post was originally posted to my other blog called Spinning Code.)

On Flying Starts

What a start for a blog to end up on the homepage of Hacker News on the first day (with a short post about Django UI). The good: it’s surprisingly easy to install a cache plugin to WordPress. The bad: this blog is not hosted on the Slipmat.io server (because I don’t want to install PHP on it) so I missed a great chance to stress-test it.254 Concurrent Users in Google Realtime Analytics

To stay with the topic on flying starts, I’ve been quietly readying the first Slipmat.io tool, a site for taking and making requests in Mixify events, for its official launch. I opened the site for public three weeks ago and I’ve been working on it ever since. I’m keeping detailed public release notes to keep the users updated and also for myself as a way to get a feel of the progress.

Most non-DJs and furthermore non-Mixify users probably have no idea what this tool actually does, so let me explain it quickly. Mixify is a Web site where DJs can play for a live audience by streaming sound (and possibly also video) from their computer to Mixify which distributes it to listeners. Listeners can join these live events and chat with the DJ and other listeners. Many DJs take requests but it can sometimes be cumbersome trough the small chat window that can update really fast if there are lots of listeners actively chatting. MixifyRequests is a site where a DJ can make a request event and get the requests in a nice, organised way with a easy to use admin controls for marking requests played etc. Users can also vote for each others requests and there’s a bunch of other features, too. So that’s the idea in a nutshell.

MixifyRequests Homepage v0.9a

I did the first part of the development in one 10 day sprint, with the idea of getting the first alpha version online in whatever state it would be after 10 days. (Because real developers ship.) The first release had one big showstopper-bug[^1] which was due to my lazy testing of Facebook sign up process, but otherwise everything worked fine.

[^1]: Where users who signed up everywhere but from the homepage would get a 500 error after first login because the login view respected ‘next’ parameter that would redirect them past the view that created an app-spesific user profile of which everything on the site assumed to be there. Thanks to Sentry I was informed about the bug immediately and got a hotfix in place in about 20 minutes but it had already affected dozens of users of which few of them didn’t come back that first night.

I decided to do kind of a soft launch by only telling about the site to few selected DJs and by placing a disclaimer on top of every page with a note saying “This site is not officially launched yet. Use it and have fun, but please do not advertise this yet in bigger scale. Thanks!“. So far this has worked really well and the userbase has been steadily growing at a rate of about 1-3 new users every day. Thanks to a small number of real users actively using the site, I’ve gotten loads of good usage data, and most importantly, I’ve been able to find and fix good number of bugs emerging from situations only real users can get to. (It never ceases to amaze me how actual users always find ways to break code that was supposed to be tested really well.)

After the first sprint and the initial production push I changed the development cycle more towards daily productions pushes with the help of feature switches. Last week I sent out an email to all active DJs of the site asking volunteers as future betatesters. Got a few and now I can test and roll out new features in a controlled manner independently of production pushes. Feature switches enable me to keep the development branch in Mercurial very close to stable branch and push bugfixes,  not yet polished code and new features on the site daily without exposing anything to the public before the code is ready and tested (live with betatesters). When I want to publish a new feature, I go to the admin, flip a switch, and it’s live. Boom! Also, if there are any problems, I can just as easily flip the feature off again, and fix the problem with no hurry. Feature switches are awesome. There are lots of different apps for Django for doing this, I chose Gargoyle (mainly because couldn’t figure out how to get the newer version called Gutter working). Big props to Disqus folks for yet another awesome Open Source app.

If you’re interested in following the development of Slipmat.io site by this lone music loving Django developer, check out @slipmatio on Twitter, Slipmat.io on Facebook and, of course, keep in touch with this blog. And if you have been, thanks for reading this far!

(This post was originally posted to my other blog called Spinning Code.)