The title of this blog post refers to two things: first of all, it has been more than 4 years since my last post here. Incredible… Looking back, I think “too busy” is the main reason for this. Both work and private life have eaten up a lot of my ‘spare’ time. So, as many irregular bloggers do, I hereby declare that I will try to post more frequently :-).

The second reason why the title of this post is appropriate, is because the site has been down for several months (since April this year). This was because I made the stupid mistake of wanting to do too many things at the same time. Let me explain.

The webserver that served this blog was a KVM virtual machine running on my home server. The VM was running a version of Ubuntu that was EOL. Moreover, I wasn’t happy with the fact that I used the Ubuntu WordPress package, instead of using the official WordPress installation method. The main reason for my dissatisfaction came from the fact that by using the Ubuntu package I was depending on the Ubuntu packages to update the package. Which didn’t happen often enough to my liking. By using the official WordPress installation method you can configure WordPress to (semi-)automatically update itself. Much better from a security perspective.

I had already migrated my other KVM VMs to LXD containers, with those providing web servers now running behind an Apache reverse proxy setup. I wanted the same for this last remaining VM.

On top of that, the site wasn’t properly configured to use HTTPS (I think I had a self-signed certificate, but no automatic redirection of HTTP to HTTPS). Thanks to Let’s Encrypt there is no excuse for running a proper HTTPS server so users can browse your site without any eavesdropping.

So, one fine weekend in April I bit the bullet. But I bit off too much by doing it all at once: Install an Apache in a fresh LXD container, put it behind a reverse proxy, install the latest WordPress, restore the files from the old VM, all the while trying to implement an HTTP to HTTPS redirect.

Of course, that didn’t work out. The Debian/Ubuntu configuration of WordPress is different from just editing the default WordPress config file, instead Ubuntu has its own config file in /etc/, which is included in the one from WordPress. Also, it turns out that simply redirecting HTTP to HTTPS won’t work because I had ‘mixed content’ warnings all over the place. To fix those I had to do a search-replace of HTTP to the secure variant in the site’s database and to add some extra lines to the reverse proxy config (more details hopefully soon in a separate post). And then it turned out I had forgotten to enable Apache’s mod_rewrite in the LXC container… Before you know it, you loose sight of what you did, when and why. Which config file you changed, what you disabled, enabled, etc. Especially because all the digging, reading and fixing had to be spread across multiple evenings.

Nevertheless, it was a good learning experience :-). Don’t do this if you don’t have ample time for follow-up, do these kind of upgrades in steps and make a proper plan so that dependencies between the steps are clear and you have a functional setup between each step.

Looking back, the following would have been a much better battle plan then “let’s go, this shouldn’t be hard, the migrations of other VMs/services went fine”:

  • put the VM behind the reverse proxy
  • transition to Let’s Encrypt SSL certificates and set up HTTP to HTTPS redirection
  • Configure the container with the upgraded WordPress installation and migrate the data from the old VM.

There might have been some issues with outdated PHP versions in the old VM, but those could have been (probably) mitigated with some extra Ubuntu upgrades focusing only on keeping the web server alive or the use of a PPA with updated PHP package. All the while I could have at least kept the VM running (only disabling it during the times I actually worked on the upgrade). And, of course, I could have put a backup of the VM back while trying and documenting the upgrade path, but every time I nailed an issue, I expected it to be the last one.

Keep on learning!

Related Images: