Web Excursions 2021-09-06
Building Calliope: A Technical Journey Through MacStories’ Big Software Project
Calliope was built from the ground up to work in this paradigm, and
our full platform consists of eight separate (micro)services and three separate database deployments,
all managed seamlessly by Kubernetes.
To manage incoming traffic and load-balancing between services,
we run the incredible Ambassador Edge Stack.
This Kubernetes-native, open-source software project manages mapping URL paths to particular back-end services, and
load-balancing incoming traffic across multiple service instances (among other things).
Internally, we’ve built our platform on five services: an authentication service, a proxy service, an API service, a web server, and a Discord bot
We built our back-end services on Node.js, and our front-end website and web server on React via Next.js
Linode launched Linode Kubernetes Engine, or LKE, in mid 2020.
they don’t charge their customers for the server that runs the Kubernetes controllers.
You only pay for what you directly run your services on.
We use the open-source project Prometheus to collect metrics from throughout our system, and
the open-source project Grafana to graph these metrics in an understandable way
One of Calliope’s big features is the ability to create posts with a parent-child relationship.
For example, our weekly newsletter for Club MacStories members is made up of a variety of different sections.
Each of these sections are written independently, and generally by different authors
I’ve never liked the idea of “single-page web apps.”
Many React web apps these days are really just a single JavaScript-generated page
which fakes browser navigation and dynamically pulls content from the server after arriving on the client’s browser
I built an authentication proxy service which lives at the public URLs for each of our new websites
I deployed our Next.js instance onto a private network in our Kubernetes back end.
There’s no way to access the service from the public Internet, and
thus it can exist in a happy land of no authentication
Using some very simple syntax directly in their Markdown text, authors are able to target blocks of content at particular tiers of subscribers.
This means that within the same post, a Premier member might get some extra content that simply does not appear when viewed by a standard Club member.
Migrating from MailChimp
If you’ve ever had the misfortune of needing to inspect the HTML content of Mailchimp emails, you’ll know that it’s a terrifying mess.
Your text is buried under layers upon layers of
<table>
,<tbody>
,<tr>
and<td>
elements.Amongst these are clusters of ID-less and class-less
<div>
s and<span>
s
For each issue of MacStories Weekly or our Monthly Log, we would forward the URL of the Mailchimp-generated web archive to my back-end parser service (this was all done using a custom front-end interface for parsing these issues).
The parser would pull the archive’s HTML contents and pass it to Cheerio,
which would transform it into a traversable virtual DOM tree.
Getting Wild with Faux Dynamism:
I wanted to have my cake and eat it too: develop in React but deploy as a static website.
Thankfully, this is exactly what the Next.js framework is designed to do.
Technically it still hydrates the page with JavaScript,
but with careful design you can cut down on the amount that you’re depending on JS.
As we continued building Calliope and found more and more needs for the project, it became clear that
we had use cases for automatically applying filters to our Markdown text as it went into the system, as well as
filters to our HTML text that was generated from the Markdown.
We even needed to apply filters to the HTML as it was coming out of the CMS,
in order to insert various dynamic elements based on the subscription tier of the requesting user