TechDistortion

From GitLab to GitHub

I previously wrote about a new website publishing workflow that used Netlify CDN as the front end, using my own Private GitLab repository, hosted on an OpenVZ 2GB SSD VPS from Hosted Simply. I wanted to have my own fully private repo for projects and to host the websites I maintain for as little expense as possible.

So about that.

After having my VPS shut off by the Hosting company due to high CPU usage, and experiencing multiple build failures and web interface non-responsiveness, I tweaked every setting that I could, disabled everything non-essential to running GitLab, and finally gave in.

Turns out that following GitHubs acquisition by Microsoft in late 2018 they decided to make private repos completely free and announced that in January this year. By that time I’d already built my GitLab instance, but with the issues I was having I decided to switch between the two. Turns out that took all of about one hour, Netlify integrated without complaint and my GitLab is now disabled.

I don’t blame the hosting company for protecting other users in the OpenVZ Shared environment, that’s totally fine. Ultimately the 2GB VPS simply wasn’t enough for the GitLab instance to function on. Looking back there were some updates applied that fixed a bug I was experiencing but bundled with that bug fix was new functionality that caused higher memory and CPU usage. Hence what used to work (just barely) on my VPS, would no longer function reliably without a higher spec.

GitLab has a lot of enterprise-type features that ran in the background and consumed all of the memory with a lot of performance issues on the VPS I had available. If I didn’t mind spending more money I could have reinstalled it (and maybe in future I will do that) but for now GitHub is working much better with Netlify and technically it’s free - so there’s that.

Engineered Space Take 2

Some time ago I started my own Mastodon server and opened it up for invites. What I learned quickly is that I wasn’t alone and plenty of others were doing the exact same thing. There was no shortage of options for anyone wishing to join the Fediverse, including lots of bigger servers with far more funding than mine. I then learned more about the problems Mastodon faces on a server - there are a LOT of moving parts and the gentleman driving the Mastodon standard was (and still is) having some trouble with direction now its popularity has exploded. For my little server it had only a handful of users and the VPS it was installed on was struggling, with constant delays and timeouts and an overall lack of reliability I started looking for other options. My original instance used the address @[email protected] which was associated with Mastodon and part of the spoken outro-ductions of all of my podcast episodes as the way to get in touch with me.

I investigated and fell in love with Pleroma and wrote about how you can Own Your Social Graph late last year, mentioning Pleroma as my now preferred Fediverse server and at the time it easily outperformed Mastodon on a VPS with only 256MB of RAM (Mastodon was slow even with 1.6GB of RAM). I tried it briefly to confirm it’s functionality on a sub-domain: @[email protected] and after a few weeks tried a backend switch (move Pleroma underneath the original Mastodon address and domain) only to discover that followers wouldn’t and couldn’t be migrated between the servers. Messaging was a complete mess and I was unable to follow or be followed by anyone that had followed me previously. I hence ended up sticking with my “new” pleroma sub-domain in the longer term than I’d planned, and asked people to follow me there instead.

Since I wrote the social graph article there have been a few incidents with Pleroma as they progress towards a formal release. The first was a series of backend changes that meant it would no longer operate as reliably on low-spec VPSs like mine. The second was when the Pleroma team changed the ID data type for posts, which broke a lot of apps and scripts that I had come to rely on for various things (Auto-Posting, iOS apps, MacOS apps). Given how unreliable it had become at that point I decided it was time to shift to a newer, bigger VPS, and to try shifting back to my original domain again.

Now I have a freshly installed Pleroma instance, on my original Mastodon domain: @[email protected] and my pleroma-sub-domain will be deactivated by the end of this month. In order for people to do what I’ve done, effectively switch the backend but keep their domain remains impossible to perform without losing followers. Interestingly, it’s the only way I know of to pull that off. The sequence:

1. Start on Domain X
2. Create a different Domain Y, then ask followers to follow you there instead
3. Re-create your original Domain X, then ask followers to follow you there again

There’s currently no option to import, or auto-re-follow if you swap out the server side components that I’m aware of. I have exports of my original timeline posts for both the Mastodon and first Pleroma accounts, but to date I have not been able to successfully import them. On the plus side the broken apps and scripts have now been mostly fixed with everything I need to use back up and running, fast and reliably again.

So in the end, apologies to all, but I’m done shifting servers and instances around. I think that for the broader fediverse these sorts of issues moving servers will inevitably lead to the same problems as EMail addresses. There is no one true EMail address for most people and knowing someones address on the Fediverse will never be as simple as a single siloed solution because it can not be. Coming from a siloed world it’s annoying but a small price to pay for more control over your own social graph.

If you’re looking for me I’m back at @[email protected] and you can follow me there on the Fediverse.

The Need For Speed

After a lot of deliberation and consideration I’ve decided it’s time to push the web-front end further forward for all of my sites. Not happy with just going Static with Hugo, and after many months of pushing local caching as far as I could on NGinx, I’ve finally joined the rest of the web developers from 3 or so years ago. All of my sites are now backed by the Netlify CDN:

• TechDistortion
• TEN
• Control System Space

Ultimately you just can’t beat a distributed high-performance, low-latency Content Delivery Network. The website tests vary from a 5x to a 11x pageload improvement on average from multiple points around the globe. Locally for me it’s been amazing, but then packets for me generally traverse the Pacific to get to my backyard so that’s not really surprising.

Wishing to have control of my environment (yeah I know) I snagged a OpenVZ 2GB SSD VPS from Hosted Simply for a New Years special of $15USD/yr and built my own Private GitLab repository, then linked that to Netlify. I’m now using a well-known developer workflow with each site it’s own self-contained Git Repository, with the GitLab remote origin mapped to the Netlify CDN with a Webhook for auto-deployment whenever I commit a file to the repo. In addition since it’s Hugo and I want to publish pages into the future, I’ve also added a webhook to trigger a page rebuild periodically.

On the Mac I’m using the passable SourceTree App for Source Control and the awesome Sublime Text 3 for text editing, and on iOS I’m using the excellent Working Copy App with Textastic for text editing. To be honest I feel a lot more in control of what I’m doing now, and being able to run up my changes locally with Hugo, create custom development branches for public/private testing through Netlify and with the ability to rollback changes at the source code level, well, it makes web page maintenance and blogging a lot more like programming.

And as a programmer, I have to say I finally get why so many others do the same workflow. It’s pretty neat :)

Fediverse Series: Definition

This third post in a series about the Fediverse focuses on micro-blogging platforms. My first introduction was to Mastodon, then Pleroma and finally most recently to Misskey. Let’s look briefly at each in turn.

Mastodon

Currently the most popular in terms of active users Mastodon (approximately 2,500+ servers) originated in late 2016 and is a complex application that uses a long list of frameworks and components to deliver what is considered to be the best Web-user interface experience at the moment for both end users and adminstators. However scaling the platform remains a concern and it is driven effectively by a single developer. It originally supported OStatus, but in v1.6 about a year after it launched it added ActivityPub support.

Pleroma

Launched informally in 2017 and like Mastodon, originally supported OStatus but later adopted ActivityPub though a tighter subset known as LitePub, in March 2018 and at time of writing, despite there being over 400 instances they are still running pre-v1 software with no formal release to date. Installation however is much simpler than Mastodon and can run on extremely low-capacity low-performance hardware as a result. It has a native web user interface which is similar to Twitter in some aspects, however also comes with the Mastodon-FE (Front-End) and supports the Mastodon v1 API allowing most Mastodon compliant client and server applications to work with it seamlessly.

Misskey

Reaching v1.0 in April 2018, Misskey is developed predominantly with a strong Japanese influence and elegant styling, conforming to the ActivityPub protocol and a very tidy web interface design. It has similar installation requirements to Mastodon though is considered easier to install and maintain and at time of writing has only 40 servers in operation with posts predominantly in Japanese, but gaining in popularity in other regions.

Server vs Instance and Application vs Fediverse

Let’s be clear, a server running the software for any of these three platforms on it, is “an instance” of that software. Hence you can consider an instance to be a server usually, but technically if you’re load-balancing then things get more hazy. Each instance is for a single domain or subdomain so it still makes sense to think of an instance by its domain name and not call it a server (technically).

It’s also better to separate the application names such as Misskey, Pleroma and Mastodon from the Federated protocol they utilise, such as OStatus and ActivityPub. During the OStatus era (which technically we’re still in however OStatus use is on the decline in favour of ActivityPub/LitePub) the term “Fediverse” was coined to describe the network of federated messaging between different platforms and applications using a common protocol. As naming goes it seems to have stuck, despite a suggestion to use IndieWeb and ActivityWeb and alternative naming conventions based on their current protocol names respectively.

What I’ve Installed

Well the Fediverse sees all, including my Mastodon and my Pleroma servers and tells the story. I’ve had no end of problems with my Mastodon server, with its higher VPS specification to run it, problematic upgrades and poor availability I decided to give Pleroma a shot and haven’t regretted it. They recently added web push notifications which was really great and my script authetication issues also work now so my automation scripts are behaving at last. Having said that make no mistake, they aren’t claiming it’s done yet and their current optimistic GitHub tag of v0.9.9 tells the story indirectly, though the Pleroma development team are keen to ensure it’s as solid as possible before touting a 1.0 release.

Under-the-hood Migration

To date swapping the server and messages under-the-hood as it were it isn’t supported. Meaning if you start up an instance using Mastodon with posts/toots/messages from that instance, becuase how messages are represented by the software on the server, it’s currently not possible to take a message list from one server, migrate the entire lot to a Pleroma server running different software on the same domain. I tried this and too many things broke.

I suspect migration may someday be possible but for now at least shifting to a different domain (or in my case, sub-domain) was the next best option.

No More Mastodon: FEDIVERSE

The truth is that I might set up a Misskey server someday, I might set up a blog that federates using Plume, WriteFreely or a Hugo-ActivityPub bridge might be developed, and I want to be able to describe the means to find me, NOT the technology. In modern conversation we might say “Send me an EMail”, we don’t say “Send me an Outlook” or “Send me a Thunderbird” which, well, could be interesting. In the same fashion I no longer intend to tell people to find me on Mastodon, or Pleroma, or whichever platform I’m using since they all Federate. You can find me now, on the Fediverse.

Updates Across the Board

To reflect this I’m adopting the proposed Fediverse iconongraphy on all of my sites, will be updating URLs, podcast intros/outtros you name it to reflect the Fediverse so when you hear me mention it you’ll know what and why. TEN was updated recently to reflect this.

So if you’re looking to get in touch, you can follow me on the Fediverse @[email protected], just log into your Fediverse account on any instance of Misskey, Pleroma or Mastodon, type that into the search box and you’ll find me, follow/remote follow me and say ‘Hello’.

Catch you on the Fediverse everyone :)