The many faces of reubano

For those of you keeping track, I’ve now redesigned my website for the second time since its inception. I created my first website with the static site generator (SSG) Octopress. This first site contained a blog, project portfolio, archives, and several pages (home, about, and contact).

Despite Octopress’ popularity and breadth of plugins, the Pythonista in me couldn’t stand its Ruby centric tooling. For the first rewrite, I decide to try a new approach with a Backbone.js based single page app (SPA).

To reduce the required boilerplate and provide my SPA a bit of structure, I used Chaplin.js in combination with the asset compiler Brunch. Using Brunch and two custom plugins, I was able to emulate Octopress’ markdown workflow. In addition, I gained the ability create pages from JSON data sources. I eagerly used this new power to generate a project portfolio from the GitHub API. Keeping with the JSON theme, I also generated a photo gallery from the Flickr API.

Client side JavaScript really shone here. I was able to use cached JSON files in development, but fetch data directly from the APIs in the live version of the site. This meant that any repos I created on GitHub or photos I uploaded to Flickr would instantly appear on the site without additional intervention. I also loved that you could navigate the entire site without page refreshes or server calls after initialization. This meant the site worked offline once it was loaded.

Sadly, all good things must come to an end. Chaplin’s maintainers effectively ceased development on the framework. This, in combination with overall JS fatigue, motivated me to reconsider static sites.

SSG selection

Since I had recently given a talk on SSGs, I had a decent idea of what I was looking for. I was also determined to make the new site as lightweight as possible. This meant I was going to abandon Bootstrap, jquery, Google Analytics, and web fonts.

The contenders

I’ll save the gory details on the pros and cons of each SSG that I tried for a later post. But after trying out Roots, Hexo, Wintersmith, and Hugo, I decided to go with Metalsmith. Metalsmith was the only SSG that gave me flexibility, a comprehensive feature set, and ability to write plugins in a decent language (CoffeeScript in this case).

I briefly considered a few Python SSGs such as Nikola and MkDocs, but found their integration with my frontend stack (stylus and pug, aka jade) to be lacking at best.

This isn’t to say Metalsmith is a panacea, not in the least. What irks me the most is that Metalsmith is very picky about the order you load plugins in, and suffers from slow compile times. Hexo did a great job of alleviating the first issue (and possibly the second) with their plugin registration system.

Slow compilation

Internally, Metalsmith passes the complete list of files to each plugin, and almost every plugin needs to iterate through the entire list in order to do anything meaningful. The following is an example plugin that removes all draft posts from the pipeline.

plugin.coffee

module.exports = (options) ->
  (files, metalsmith, done) ->
    setImmediate(done)

    for file, data of files
      if (data.draft) 
        delete files[file]

The overhead of this architecture becomes present as you scale the number of pages built. Using dumbbench I retrieved the following timings in development mode:

And the following timings are the result from production mode which performs file minification and compression steps:

As you can see in the following graph, the development run displays a time complexity on the order of log(N), while the production run crosses over into linear territory on the order of N. Yet, even with logarithmic growth in the best case, Metalsmith still takes an eternity (almost 15 secs) to rebuild my site. And as I point out below, I have to wait for a full rebuild after each change.

The ugly

Additionally, JavaScript’s inherent mutability is a major source of pain. And despite my best efforts, I was never able to get either the metalsmith-watch or metalsmith-changed plugins to work.

Future work

The ideal SSG would iterate through the list of files a fixed number of times and then pass each file to every plugin in parallel. This would be most effective with immutable data structures like Python’s MappingProxyType or Clojure’s Maps.

I eventually want to look under the hood of Nikola since it may relieve some of issues stated above. Plus with the help of python-stylus and pyjade, I may even be able to reuse my templates.

Ultimately, I don’t think I will ever be completely satisfied without building my own SSG. Something that combines the speed of Hugo, the elegance of Hexo, and the flexibilty of Metalsmith. My stream processing library riko would provide a good foundation on which to build such a framework. If only I had the time…

Do you have any SSG experiences to share? Tweet your story to me @reubano.