Generic Health Check for Shimmer

[justinallenmarsh]

• inspired by the health check controllers we should come up with a generic controller and a dsl to define checks for the application (e.g. "is blender running?")
• there should probably be some predefined health checks that can simply be used (is there a sidekiq worker running? is the database reachable?). make them opt-in!

[justinallenmarsh]

Step 1: Create a Health Check DSL

Create a new file health_check.rb in the models/ directory:

# models/health_check.rb

class HealthCheck
  def self.define(&block)
    dsl = new
    dsl.instance_eval(&block)
    dsl
  end

  def initialize
    @checks = []
  end

  def check(name, &block)
    @checks << { name: name, block: block }
  end

  def run_checks
    results = {}
    @checks.each do |check|
      results[check[:name]] = run_check(check)
    end
    results
  end

  private

  def run_check(check)
    begin
      check[:block].call
      "ok"
    rescue => e
      e.message
    end
  end
end

Step 2: Define Predefined Health Checks

Create a new file predefined_health_checks.rb in the models/ directory:

# models/predefined_health_checks.rb

module PredefinedHealthChecks
  def database_check
    check 'postgres' do
      ActiveRecord::Migrator.current_version
    end
  end

  def sidekiq_check
    check 'sidekiq' do
      raise StandardError, 'High worker latency' if Sidekiq::Queue.new('dispatch').latency.seconds > 1.minute
    end
  end

  def blender_check
    check 'blender' do
      raise StandardError, 'High blender latency' if Sidekiq::Queue.new('blender').latency.seconds > 3.minutes
    end
  end
end

Step 3: Define Health Checks using DSL

Create a new file system_health_check.rb in the models/ directory:

# models/system_health_check.rb
require 'health_check'
require 'predefined_health_checks'

class SystemHealthCheck < HealthCheck
  include PredefinedHealthChecks

  def initialize
    super

    # Predefined Checks (Opt-in)
    database_check
    sidekiq_check

    # Custom Checks
    check 'custom_check' do
      raise 'Something is wrong' if 1 + 1 != 2
    end
  end
end

Step 4: Create System Status Controller

# app/controllers/system_status_controller.rb

class SystemStatusController < ApplicationController
  skip_before_action :require_login
  skip_before_action :check_locale
  skip_after_action :verify_authorized

  def show
    health_check = SystemHealthCheck.define
    statuses = health_check.run_checks
    render plain: statuses.map { |k, v| [k, v].join(": ") }.join("\n"), status: statuses.values.uniq == ['ok'] ? :ok : :service_unavailable
  end
end

Step 5: Add Route for System Status

In config/routes.rb:

# config/routes.rb
get 'status', to: 'system_status#show'

Step 6: Test the System Status

In your RSpec file:

# spec/system/system_status_spec.rb

require "system_helper"

RSpec.describe "System Status" do
  it "responds with an OK status code" do
    visit "/status"
    expect(page).to have_content("all: ok")
    expect(page.status_code).to eq 200
  end
end

Step 7: Load the Health Check DSL

In config/application.rb, add:

# config/application.rb

config.autoload_paths << Rails.root.join('models')