How I publish my wiki with org-publish

I keep my [[hyper commonplace garden wiki]] (AKA [[digital garden]]) at https://commonplace.doubleloop.net.

I use [[org-roam]] to maintain my wiki locally, and I use [[org-publish]] to render my wiki as a site of static html files. This page is specifically to describe how I use org-publish on my org-roam files to get them up on my public site.

⥅ [[images/miller-wiki.png]]

Why org-publish?

org-publish feels a bit slow and clunky to me sometimes, but it's also very hackable and has gotten me pretty far so far. I'm not an expert on it by any means. One might prefer to use something like ox-hugo, firn, or similar (see [[options for publishing an org-roam-based digital garden]]). I do like the idea of just using the built-in org-publish capabilities, but I have started to get to the point where I've customised it so much, I wonder if it would be better to use something else. But, it works for me for now.

Just to note that I've built this all up incrementally - I'm an advocate for starting simple, and you don't need half the stuff I'm describing here just to get started. See the history section at the end to see the progression, and how to start simple.

My org-publish setup

I have the various publish-related functions in a [[publish.el]] file. I have a [[Makefile]] that I can use to call these functions - this to avoid the annoying way in which org-publish completely blocks emacs when you call it interactively. I used to publish from my local machine via the Makefile. Then, for a while, I used have the publish functions called after every commit in a gitlab pipeline ([[Publishing org-roam via GitLab CI]]). As my garden grew the publish process got so slow that it used to time out on Gitlab. As of April 2024, the 'publish to web' process runs on my own server, once a night, via a cron job. I also cross-publish my garden to the [[Agora]].

As far as the org-publish configuration goes, it's generally a fairly standard org-publish setup I think, with a couple of extra bits for [[org-roam]] purposes (e.g. backlinks and graph), along with a bit of extra JS and CSS for [[Miller columns]].

Using a makefile

As mentioned above, I used to use a Makefile to publish from my local machine. I don't publish from my local machine, but this section left here for posterity (or until properly gardened and tidied up…)

I first read about how to publish with a Makefile here: How to blog with Emacs Org mode. All this does is call through to various publish functions in a new process - avoiding blocking Emacs while I'm publishing.

So for example, if I'm in a file in my org-roam project, I could do SPC c c (which triggers helm-make-projectile) and then the publish option. This triggers the publish make target. (You could also just run make publish from the command line of course).

The publish target is:

publish: publish.el
  @echo "Publishing... with current Emacs configuration."
  emacs --batch --load publish.el --funcall org-publish-all

This run emacs in batch mode, loading my publish.el as a library file, calling org-publish-all, and then exiting once finished.

In the previous case I'm calling one of org-publish's built-in functions. I can also call a custom function, e.g. with my graph target

graph: publish.el
  @echo "Graphing..."
  emacs --batch --load publish.el --funcall ngm/build-graph
	@chmod 755 graph.svg

This calls my own ngm/build-graph function in my publish.el.

I deploy the html files to my server with rsync via a make target, too.

rsync: publish.el
	@echo "rsyncing published site to hosting..."
	rsync -chavz /var/www/html/commonplace/ dloop:/var/www/commonplace/

You could have any kind of deployment step here, like a push to Netlify.

So, to re-iterate, the Makefile is just a non-blocking way to call various publishing functionality. With an extra benefit that I can have an 'all' target, that does a few steps in a row, e.g:

all: graph republish rsync

You can find my full Makefile here on Gitlab.

Configuring and extending org-publish in your publish.el

My publish.el file is where I put all of the bits of org-publish configuration, as well as custom publish functions I've written.

It's worth pointing out that this is a constant work-in-progress - you can always find the latest version here on Gitlab.

I have a few differents types of things in my publish.el.

standard org-publish config

This is the standard stuff you'll find on a million posts out there for publishing a blog with org-mode. I'm not really doing anything special here for org-roam/wiki purposes.

I've tweaked head-extra a little to include various css/js files, and preamble and postamble a little to link to my graph and my sitemap.

org-roam backlinks

I've got a couple of functions for including org-roam's backlinks during the publish process.

org-roam graph

I've got some functions for publishing the graph that org-roam produces to my digital garden. I also tweaked the process a little bit so as to allow the links on the nodes of the graph to link to the HTML pages on my site, as opposed to the org-protocol links they link to out of the box.

Hacking the HTML output

This is a bit messy and I'd like to improve on it. I'm basically overriding org-html-template, which is a bit of a no-no, but it's the only way I could see so far to tweak the HTML enough to let me do some of the custom page layout that I wanted.

gitlab pipeline for automatic publishing

In February 2021 I set up a publishing pipeline to allow me to simply commit the org files to my git content repo, and have it all published from there. See: [[Publishing org-roam via GitLab CI]].

Custom CSS and JS

You can use your own custom CSS and JS as you like - just include the links in your html-head-extra in org-publish-project-alist.

CSS

Right at the beginning I started with the very simple CSS from Better Motherfucking Website. Since then I've tinkered with it a bit to make it look a little closer to my WordPress theme at doubleloop.net, and to get the [[Miller Columns]] stuff working.

JS

To get the [[Miller Columns]] bits working, I copied and tweaked a bunch of stuff from Jethro's cortex ox-hugo theme that he uses for his braindump. It's mostly in page.js and URI.js.

Extra bits and pieces

Some other things I've picked up along the way.

Doing some things interactively

One downside of putting stuff in a publish.el is that it isn't loaded by default as it would be if it's in your .emacs config. So you can't out-of-the-box then call stuff like org-publish-current-file. What I'm doing is just quickly opening publish.el and then evaling the buffer with , e b, then all the config and functions are available for use. That works for now, but I wonder if I can set it as part of dir-locals or in some kind of hook to get it eval'ed automatically whenever I'm in the project.

Publishing single files

I actually don't run make all that often on my local mcahine. It's waaaaay too slow.

What I tend to do is org-publish-current-file on any new files I'm working on, and SPC c c and rsync. Then occasionally I'll do make all to rebuild and republish everything (which will include a new graph, sitemap, and update the backlinks on all the pages).

htmlize

org-publish uses htmlize to do syntax highlighting of code blocks.

I found that this doesn't work immediately if you're publishing via –batch. htmlize is a package that needs to be included. It's included by default if you're publishing within the project when you're running Emacs, but not from –batch.

I learned from here https://gjhenrique.com/meta.html how to get htmlize to work from –batch.

Publishing unchanged files

Call org-publish-current-file with the universal argument (i.e. do SPC u first if you're in spacemacs) - that will force a republish of the file even if its actual content hasn't changed. So for example you'd do this if you want to update the backlinks on a published page.

A little bit of history

I think It's useful to have the progression of how I got to where I currently am, because you might not need all the bits and pieces as I have them now.

I started with all my publishing options in my .spacemacs config file. It did the job, but with two issues. Firstly, I wanted to share the full files more easily for others to reference (I do have my .dotfiles in a public repo, but I wouldn't expect people to look there and find the relevant bits).

Secondly, I found the interactive org-publish to be really slow, and it blocks Emacs, so I couldn't do anything during the publish process.

So I then moved them in to the publish.el file in the same folder as all the files, and set up the makefile.

As far as the look'n'feel goes, I used to have just a single page at once on the screen, before I switched to the Miller columns style. I think I like the Miller columns navigation for a personal wiki, but not 100% convinced - I may just go back to single page again at some point.

Some handy resources

• Building a personal website with org-mode
• Blogging with Org publishing
• Org Export Hooks
• Filters: Advanced Export Configuration (The Org Manual)