Rails on Kubernetes

Overview

This is a lengthy topic that is split into multiple parts:

Part 1: Rails and Docker Compose
Part 2: Kubernetes
Part 3: Deployments, Rolling updates and Scaling

Updates

2019-09-29: Upgrade gems, few error fixes and Docker image versions
2017-11-13: Using Alpine linux as base image and using unified entrypoint bash script. Now our docker images are half the size!

Do I need this ?

The short answer is: It depends™.

The traditional way to deploy Rails looks more or less like this:

Provision your servers with your favourite automation tool like Puppet, Chet, Ansible or Bash :)
Capistrano deploys
Nginx as a reverse proxy
Puma, unicorn, thin etc. as your app server

Capistrano makes it easy to deploy on multiple servers but you still need to provision those manually.

If you throw in a queue for background work, use caching as much as possible and create proper DB indexes you'll have a legit setup.

Complete deployment

If you want to skip straight to the code you can access the sample Rails app on Github.

Kubernetes

I won't spend time explaining what Kubernetes is in this post as there are plenty of resources out there. Using Docker or rkt to run your applications makes your infrastructure much more portable. Instead of having to provision servers and deploy the code there, you package your system libraries and your application code in a container image. There are so many official and community images to choose from you will not have to worry about configuring your own servers from scratch. Need an Nginx reverse proxy ? Easy: FROM nginx:latest. Handling things like major security updates for your OS can be as easy as pulling from a newer image.

Since you will now need to package your app in a Docker image you will have a few more moving parts. You will need a private registry and the tooling around building and pushing new images.

This is definitely a trade-off: you will now have to use kubectl for your deployments instead of cap deploy. But look at the pros:

Development using containers guarantees a consistent environment for every developer.
Need to upgrade Ruby ? It's as easy as packaging your app using the latest ruby base, test that and use rolling updates to push it live.
When you're scaling horizontally to more Rails servers you don't need to worry about updating Nginx configs. Ingresses and Ingress Controllers will handle that.

Take this sample setup for example:

We have a few separate services here: Rails application, Redis instance, Sidekiq workers and a Postgres instance.

Depending on what your bottlenecks are you may want to easily spin up more Sidekiq workers for example. Or maybe you get a lot authenticated traffic to your main Rails app and want more app servers. With Kubernetes this is as easy as running:

kubectl scale --replicas=3 rc/rails

The change in the numbers of replicas will propagate all the way to the Ingress Controller. Taking an Nginx IG as an example, the update will mean adding the new services to the upstream config block in the virtual host.

The rest of this article will go through a hypothetical setup step-by-step.

Prerequisites

If you intend to follow the steps here on your machine here's what you need to have installed:

Rails and Docker Compose

You could skip this step if you prefer to keep your current development workflow. You can always run the Rails app using the bundled puma server locally. Having said that, if you get the docker-compose setup running, switching to Kubernetes is going to be a breeze.

I will use ruby-2.6.4 and Rails 5.1.1 for this write-up. The simple Rails app I will create for this article looks like this:

Devise for authentication
Postgresql database
Redis instance
Sidekiq worker for sending emails in the background.

Let's go ahead and create the app:

rails new rails-kube-demo --database=postgresql

And add devise to our Gemfile:

gem 'devise'
gem 'sidekiq'
gem 'tzinfo-data'

We can run bundle install locally, followed by the default Devise setup steps:

rails generate devise:install
rails generate devise user

This will create the user.rb model along with the required migrations and routes. We don't need to run the migrations now as we will configure that in the next steps.

Docker-compose

First let's define the Dockerfile for our Rails application. This is take straight from the docker-compose guides on Rails. I added the netcat package to help us detect when services inside containers are up and running (more on that later).

FROM ruby:2.6.4-alpine3.10

RUN apk --update add nodejs netcat-openbsd postgresql-dev
RUN apk --update add --virtual build-dependencies make g++

RUN mkdir /myapp

WORKDIR /myapp

ADD Gemfile /myapp/Gemfile
ADD Gemfile.lock /myapp/Gemfile.lock

RUN bundle install
RUN apk del build-dependencies && rm -rf /var/cache/apk/*

ADD . /myapp

COPY docker-entrypoint.sh /usr/local/bin

ENTRYPOINT ["docker-entrypoint.sh"]

Create the docker-entrypoint.sh script:

#!/bin/sh

set -e

if [ -f tmp/pids/server.pid ]; then
  rm tmp/pids/server.pid
fi

echo "Waiting for Postgres to start..."
while ! nc -z db 5432; do sleep 0.1; done
echo "Postgres is up"

echo "Waiting for Redis to start..."
while ! nc -z redis 6379; do sleep 0.1; done
echo "Redis is up - execuring command"

exec bundle exec "$@"

This script does a few things. First, it waits until Redis and Postgres containers are up. The 'redis' and 'postgres' will be mapped IP addresses of their respective docker containers. We are adding this script to all our Rails containers. If, let's say, we want to run our latest migrations, we'll need the Postgres server up and running prior to that. Another alternative to rolling your own bash script would be something like wait-for-it or roll your own bash script.

Let's make our script executable:

chmod +x docker-entrypoint.sh

Now we'll define docker-compose.yml and a few services. We have a setup service that will run our migrations, a db service running the Postgres database, a db_data service with a data only container and the web service running the Rails app.

version: '3'
services:
  setup:
    build: .
    depends_on:
      - db
    environment:
      - RAILS_ENV=development
    command: "bin/rails db:migrate"
  db:
    image: postgres:9.6-alpine
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=mysecurepass
      - POSTGRES_DB=rails-kube-demo_development
      - PGDATA=/var/lib/postgresql/data
  db_data:
    image: postgres:9.6-alpine
    volumes:
      - /var/lib/postgresql/data
    command: "/bin/true"
  web:
    build: .
    command: "bin/bundle exec rails s -p 3000 -b 0.0.0.0"
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    depends_on:
      - db

The first thing I always to is to check the image's documentation for the environment variables I can pass through. The ones we are interested in are:

POSTGRES_USER
POSTGRES_PASSWORD
POSTGRES_DB
PGDATA

We created a volume-only image that we mount on the postgresql container to persist the db data.

Now we update our database.yml file correspondingly:

default: &defaul
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: postgres
  password: mysecurepass
  host: db
  port: 5432

development:
  <<: *default
  database: rails-kube-demo_development

test:
  <<: *default
  database: rails-kube-demo_test

Before building our images let's create a .dockerignore file and add our log and tmp folders:

tmp/*
log/*

Let's test this out:

docker-compose build

$ docker-compose up
setup_1    | Waiting for Postgres to start...
setup_1    | Postgres is up - executing command
setup_1    | == 20170524183835 DeviseCreateUsers: migrating ================================
setup_1    | -- create_table(:users)
setup_1    |    -> 0.0294s
setup_1    | -- add_index(:users, :email, {:unique=>true})
setup_1    |    -> 0.0166s
setup_1    | -- add_index(:users, :reset_password_token, {:unique=>true})
setup_1    |    -> 0.0152s
setup_1    | == 20170524183835 DeviseCreateUsers: migrated (0.0613s) =======================
setup_1    |

Note that the sample output is out of order. I just wanted to emphasize that our script and migrations worked.

We should now be able to register a new account at: http://localhost:3000/users/sign_up

Sidekiq and Redis

Let's move on to the Sidekiq queue. I configured the default_url_options so that ActiveMailer doesn't complain about the host and port to my config/environments/development.rb:

config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }

Next, let's create config/initializers/sidekiq.rb and configure our Redis connection. Since our sidekiq worker will not sit on the same server with Rails we have to configure both the client and the server here. We're using two ENV variables to pass on the Redis URL and PORT:

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end
  
Sidekiq.configure_server do |config|
  config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end

This is my config/sidekiq.yml that we will start the sidekiq worker with. Pretty standard except the mailers queue. ActiveJob will push jobs in the mailers queue and this is how we tell Sidekiq to also listen in for jobs.:

---
:concurrency: 25
:queues:
  - default
  - mailers

I configured ActiveJob to use Sidekiq in my config/application.rb:

config.active_job.queue_adapter = :sidekiq

And added an override to force Devise to send emails through the queue (in app/models/user.rb):

class User < ApplicationRecord
  # Include default devise modules. Others available are:
  # :confirmable, :lockable, :timeoutable and :omniauthable
  devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :trackable, :validatable

  def send_devise_notification(notification, *args)
    devise_mailer.send(notification, self, *args).deliver_later
  end
end

I updated the routes.rb to include the Sidekiq Dashboard engine:

  require 'sidekiq/web'
  mount Sidekiq::Web => '/sidekiq'

This will make it easier to test once we deploy everything.

We also need to start the worker as soon as Redis becomes available. Remember the docker-entrypoint.sh script we created earlier ? That script will wait for Postgres and Redis to be online and run whatever command we pass in docker-compose. In our case: command: "bin/bundle exec sidekiq -C config/sidekiq.yml"

And finally, update the docker-compose.yml with the the sidekiq and redis containers:

version: '3'
services:
  setup:
    build: .
    depends_on:
      - db
    environment:
      - RAILS_ENV=development
    command: "bin/rails db:migrate"
  db:
    image: postgres:9.6-alpine
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=mysecurepass
      - POSTGRES_DB=rails-kube-demo_development
      - PGDATA=/var/lib/postgresql/data
  db_data:
    image: postgres:9.6-alpine
    volumes:
      - /var/lib/postgresql/data
    command: /bin/true
  sidekiq:
    build: .
    environment:
      - REDIS_URL=redis
      - REDIS_PORT=6379
    depends_on:
      - redis
    command: "bin/bundle exec sidekiq -C config/sidekiq.yml"
  redis:
    image: redis:3.2-alpine
    ports:
      - "6379:6379"
  web:
    build: .
    depends_on:
      - redis
      - db
      - setup
    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    environment:
      - REDIS_URL=redis
      - REDIS_PORT=6379
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    depends_on:
      - db

We can run docker-compose build && docker-compose up to update our deploy.

Check out http://localhost:3000/sidekiq for the Sidekiq dashboard.

The easiest way to test that Sidekiq is running those tasks is to create a user at http://localhost:3000/users/sign_up and then request a new password at http://localhost:3000/users/password/new. You should be able to see the job processed in the Sidekiq Dashboard.

Next Steps

We now have a working Rails/Sidekiq application running in Docker. In Part 2 we will take our docker-compose.yaml file and translate it into Kubernetes resources.