A cloudy Ghost - Part 1

This is the journey that sparked my "What does cloud really mean?" post. We'll start with a bit of background.

What is Ghost?

Ghost is a blogging platform, the very same that hosts the blog you're reading. It's an open source Node.js app, but the team who write it also have a paid hosting sceme that funds the development of Ghost.

I've used Ghost for a while (for the first blog that I started). My wife has been using it for her blog Taste the Tea since I suggested it when she first started. By a happy coincidence it is also the platform running our blog at work.
I could have started this blog on Medium (and it would have saved me a load of effort promoting it), but setting it up was part of the fun and that means hosting it myself.

What were the goals for this project?

First and foremost, I wanted to build this blog in as cloudy a way as I can, though without spending loads of money. This means it should be very quickly deployable and I should be able to deploy additional blogs with minimal effort.

The money aspect was quickly sorted out by signing up to a free trial of Google Cloud Platform. They gave me $300 credit for a year.

How was I going to achieve the cloud-i-ness?

• Put Ghost in a Docker container
• Put the Nginx frontend in a Docker container
• Use Docker Compose and automate as much of the build as I can
• Put the images in an object store

The rest of this post will focus on object storage, while the next post will look at Docker.

Why object store?

Ghost lets you insert images into your posts, and has images used in the theme and as "feature images" for posts (like on the front page, or on Twitter cards).

The inline images in posts can be be from any URL you enter, but unfortunately you cannot use external image files for the feature images and theme images, these have to be stored by Ghost itself. This works well enough for most people. The uploader is nice and easy and people generally either don't care enough to change it or are running their blog behind a CDN or something anyway.

The main drawbacks are:

• The images don't get included in the standard export (backup), so to backup the images you have to have access to the server file system
• It is difficult to manage multiple load-balanced front ends, as all front ends need live, read/write access to the images
• If you run a non-persistent install of Ghost, such as my Docker containers, your images will be lost whenever the container is destoryed and a new one deployed!

So, I could keep my images on a share of some kind. I could set up rsync and let that keep my nodes in sync. I could pass the content directory through as a volume from my Docker host...

Or I could just use an object store.

So what is an object store?

An object store is a way of storing files more programmatically than the traditional methods.
I'm talking about the traditional method of having a physical disk, making partitions on the disk a filesystem on the partition. Then splitting the filesystem up into blocks and having an allocation table to track what things are in what blocks. When you write a file to disk it gets broken up into little blocks and the blocks get put wherever there is room for them. When you read a file you find which blocks it is on and read them and stick them back together.

We call this a "block level" filesystem, and it has long been a point of pain in IT. The problem? Block level storage is difficult to keep reliably in sync when you have multiple writers. There are solutions out there that replicate every little block-level change to other devices, but you don't need to look for very long to find someone who has had a complete nightmare trying to recover from a vSAN gone wrong!

An Object Store is different. It's a REST API that deals with whole files. You say 'store this file for me please' and it responds saying "I stored your file at this URL and you can come back with a GET request when you want the file again". We can store the URL in our database and all the hosts can get access to whenever they need. If we set the file to public, even the client can get the file, directly from the Object Store.
Gone is the necessity to mount an entire filesystem. Gone are the complexities of sharing. Computers don't understand sharing, there are too many opportunities for one of them to get out of contact and mess things up for everybody.

So object storage is basically an abstraction of the underlying block storage, that lets you read and write files through simple HTTP requests.

So, doesn't Ghost need to know what to do with this object store thing?

Well, yes. Fortunately there are several projects that can help with this.

Ghost lets us add a "storage adapter", so it can talk to types of storage it doesn't knwo about. Anyone can make a storage adapter and plug it in to Ghost so when Ghost writes a file, it turns it into a request to Dropbox, writes it to a private server somewhere or even writes it to tape in a tape drive!

What object stores are out there?

There are a few object stores out there, and most of the major cloud players offer one.
DigitalOcean Spaces is the new object store launched by DigitalOcean. They're a popular VPS provider who have recently been getting more and more 'proper cloud'. This object store is API compatible with AWS S3, meaning we can use S3 plugins to interact with it.
If you want to use DigitalOcean Spaces, you can get $10 free credit using this link!
Google Cloud Platform Cloud Storage is the Chcolate Factory's own object store. It has an S3 compatibility mode but there is also a Ghost storage plugin made especially for Google Cloud Storage. This is the one I use, because I am using the free trial of GCP.
Amazon Web Services S3 has beena around since 2006 (that's 4 years ahead of Google and Microsoft). In case you were wondering, S3 means the Simple Storage Service. There's a Ghost storage adapter for S3 (of course there is).

Finishing up

That's al for today. In the next post of this series I'll describe how I packaged up Ghost, and one of those steps will be installing the storage adapter.