You are here

AWS, Drupal and Caching: pt.1 Intro and Plan

Wednesday, October 15, 2014

So, here we are at another series of blog posts by your bearded badger storyteller.  The reason this one is going to be another of those long series is just because it breaks up into separate pieces nicely, and is altogether a fairly complex topic if you haven't played with it all before.

But, first things first, why am I doing this?  Well, there are many reasons, one is that I am sick of all the excuses you hear when on shared hosting such as 'adding caching will cause us performance issues'.  Another, more sensible, reason is just that whilst I know of a lot of the AWS elements, until you have really broken it apart and played, you can only know so much.  So I want to see what it takes to build a server to host my site, what I can do to it, and how much of my time it will take to maintain such a behemoth.


Let's Plan

Before doing anything with AWS, you kind of need to have a feel for what you want, and understand the parts to make it up.  Usually the most satisfying approach, is to create a nice diagram such as the beautiful ones you can find on the AWS documentation pages (  There's also lot of reading on cloud computing, and how it allows us to modularize our servers, but ultimately I wanted a single web node running Debian that I could modify as much as needed.  I also wanted this to have a fixed IP and not be over expensive, yet giving me a good base to build upon (due to inexperience in devops field).  Oh, of course, we want this to be based in the EU as most of my users are likely to be in the UK.  Although of course, if this changes, it's nothing a good CDN cant help with.  This said, a diagram wouldn't help me all that much, so sadly I decided to save time and not draw one, instead I just listed the tools I want to sit on my web node.

This final point of using a single web node came about once I realised that whilst AWS comes with some cool technology like RDS (Relational Database Service), these things aren't necessarily cheap.  Another standard of web architecture is to seperate caching layers, and introduce a load balancer, which would also add cost, and until I can absorb this in some way, was not worth it yet.  Therefore, I chose to use the Turnkey lampstack (I tried Bitnami first, but will go into that in a later chapter) as seemed a nice base (

Now we have a base OS decided upon, we need to think about what is going to go on there.  Turnkey gave us the base lampstack, but there are several things I am used to on a server, and so will have to make sure these are added to the server as one of my first tasks:

  • bash
  • Individual user account(s)
  • DB user(s) for each site
  • sudo
  • drush
  • memcache
  • apc
  • varnish
  • slow query logs (always handy to receive)

Alongside the above tools, a backup scheduler is required.  Now, there are several online systems you can pay for, or even use free trials for.  But tbh, this isn't necessary at all so long as you put in a little time up-front.  This is because AWS comes with a nice CLI interface, part of which allows you to trigger off backups.  If only there was some system tool to run AWS CLI functions via some chronological peri... okay, yes, we plug cron in to use this.  Very little code, very powerful backup system FTW.

So, is this it?  Not quite, one of the requirements was to have a static IP for my server.  There are many reasons for this, the short-term reason is that my DNS is hosted elsewhere, so I don't want to have to repoint it everytime AWS restarts my server (which can happen due to server issues, or Amazon updates).  Other reasons include scaling up the server and needing to know which varnish servers to purge, aswell as point a load balancer at.  So AWS comes with the concept of EIPs (Elastic IPs) which are free, so long as they are in use, which is exactly what I need.

The final step in most cases is cost, how much do we think this will come to.  TBH that is part of why I am running this test.  All the docs imply this will come to something around the £200 p/a mark, but will no more once the free trial period of AWS is over.

Oh yeah, did I mention that, if you are new to AWS, a lot of the basic parts are free for a set period of time.  It's not a lot, but it meant I did not get charged whilst I was configuring the server.  Bonus!

Final point which I imagine several people I know will whine about, saying things like 'why didn't you use chef/puppet you FOOL'.  Now, whilst I am not a chef expert, I know the parts, and have it running on a local vagrant box nicely.  In the long term, if my server needs multiple boxes due to how popular things get (heh) I will up it to use chef.  The main reason I did not use chef in this though was that it hides a lot of the config and settings (by design), and as this was a learning process for me to understand servers better, it seemed counter productive to use such an automation tool.


Blog & News

Monday, March 7, 2016

Okay, I know it is awful practice to put 2 things into one article, but I didn't want to write two tbh, so I will keep this tidy, and you can grab out what you need.


Subscribe to Blog & News


Cultivating a better future, with the seeds of better websites

For several years I have evolved through one of the largest media charities in the UK to the role of Senior developer.  I loved it there, but have to stepped out on my own as a freelance developer.

My main skills are with the Drupal CMS, developing code for it, ensuring standards through projects (whether that design, development, or planning), and leading teams to build applications they can be proud of.

Digital Consultancy, Web Development and Project Architecture are where I try to focus my skills, supporting my passion and desire to create stunning websites on time, in budget, and meeting your objectives.

To build a powerful web presence, I combine my creative, technical, and managerial experience (alongside a good splash of passion) which I have cultivated over 10 years in the industry, creating websites people want to shout about.

More on how I can help