From Sidekiq and Redis to Solid Queue

Sidekiq is excellent software. If it’s running happily and Redis isn’t costing you anything you care about, you can stop reading here.

But plenty of teams run Redis for one reason only: Sidekiq needs it. That’s one more service to pay for, monitor, upgrade and back up. Rails 8 made Solid Queue the default job backend, and for most apps it does the job with the database you already have.

Here’s how we move an app across without dropping jobs, and how we decide whether to move at all.

Should you switch?

Switch if most of these are true:

• Redis exists only for Sidekiq (and maybe caching, which Solid Cache can take over).
• You process thousands of jobs a minute, not hundreds of thousands.
• You don’t depend on Sidekiq Pro or Enterprise features you’d have to rebuild, such as batches, unique jobs or rate limiters.
• Your jobs are written against Active Job, not Sidekiq::Job directly. (If they’re native Sidekiq jobs, budget time to convert them.)

Stay on Sidekiq if your job volume is genuinely huge, if your database is already your bottleneck, or if a paid Sidekiq feature is load-bearing. There’s no prize for switching.

How Solid Queue works, in one paragraph

Jobs are rows in database tables. Workers poll for ready jobs and claim them using FOR UPDATE SKIP LOCKED (on PostgreSQL and MySQL 8+), so many workers can pull from the same table without blocking each other. A dispatcher moves scheduled jobs into the ready set when they’re due. A scheduler runs recurring tasks. A supervisor (bin/jobs) starts and watches all of them. That’s it — no separate server.

Step 1: Install Solid Queue next to Sidekiq

On a Rails 7.1+ app (and Ruby 3.2+):

bundle add solid_queue
bin/rails solid_queue:install

The installer creates config/queue.yml, config/recurring.yml, db/queue_schema.rb and a bin/jobs binstub. It also sets Solid Queue as the production Active Job adapter — undo that line for now. During the migration we want Sidekiq to stay the default.

# config/environments/production.rb
# config.active_job.queue_adapter = :solid_queue   # not yet
config.solid_queue.connects_to = { database: { writing: :queue } }

Step 2: Decide where the tables live

By default Solid Queue wants its own database, configured in config/database.yml:

production:
  primary:
    <<: *default
    database: myapp_production
  queue:
    <<: *default
    database: myapp_production_queue
    migrations_paths: db/queue_migrate

Then run bin/rails db:prepare to create it.

A separate database keeps job churn away from your main tables and makes it easy to move later. If you’d rather keep one database, that’s supported too: move the contents of db/queue_schema.rb into a normal migration, delete the schema file, and remove the connects_to line. We use a separate database on PostgreSQL apps and a single database for small apps on one server.

Step 3: Map your Sidekiq config to queue.yml

A typical sidekiq.yml:

:concurrency: 10
:queues:
  - critical
  - default
  - low

Becomes:

# config/queue.yml
production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: [critical]
      threads: 5
      processes: 1
    - queues: [default, low]
      threads: 5
      processes: 2

Two differences to know:

• Queue order is priority. A worker with queues: [default, low] drains default before taking from low. If you relied on Sidekiq’s weighted queues, give your important queues their own workers instead.
• Threads use database connections. Every worker thread needs a connection to the queue database. Make sure the pool size for that database is at least your threads value plus a little headroom.

Run bin/jobs check to validate the file before deploying.

Step 4: Move recurring jobs

If you use sidekiq-cron or sidekiq-scheduler, move those schedules into config/recurring.yml:

# config/recurring.yml
production:
  nightly_cleanup:
    class: CleanupJob
    queue: low
    schedule: every day at 3am
  sync_invoices:
    class: SyncInvoicesJob
    args: [ "incremental" ]
    schedule: every 15 minutes

Schedules are parsed by Fugit, so both natural language (“every day at 3am”) and cron syntax work. Times use your config.time_zone unless you set one.

Important: turn off the Sidekiq version of a recurring job in the same deploy you turn on the Solid Queue version, or it will run twice.

Step 5: Recreate retries, uniqueness and limits

Solid Queue has no retry system of its own. That’s deliberate — it relies on Active Job:

class SyncInvoicesJob < ApplicationJob
  retry_on Net::OpenTimeout, wait: :polynomially_longer, attempts: 10
  discard_on ActiveJob::DeserializationError

  def perform(mode = "incremental")
    # ...
  end
end

If you used sidekiq_options retry: 5, translate it into retry_on for the errors you actually expect. Jobs that fail permanently land in solid_queue_failed_executions, where you can retry or discard them (or use Mission Control – Jobs for a UI).

For “only one of these at a time” jobs, use concurrency controls:

class RebuildSearchIndexJob < ApplicationJob
  limits_concurrency to: 1, key: ->(account) { account.id }, duration: 15.minutes

  def perform(account)
    # ...
  end
end

Conflicting jobs wait by default (on_conflict: :block); use on_conflict: :discard to drop them instead. One caveat: concurrency-limited jobs are enqueued one at a time, so they don’t benefit from perform_all_later.

Step 6: Run both, and move jobs one class at a time

This is the step that makes the migration safe. Start Solid Queue alongside Sidekiq in production. On a single server, the Puma plugin is the simplest way:

# config/puma.rb
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"]

On bigger setups, run bin/jobs as its own process (with Kamal, as a job role).

Then switch individual job classes over while Sidekiq stays the app-wide default:

class SendWelcomeEmailJob < ApplicationJob
  self.queue_adapter = :solid_queue
end

Start with low-stakes, high-volume jobs (emails, webhooks). Watch them for a day. Move the next batch. Leave the scariest jobs — payments, anything with side effects outside your app — for last, once you trust the setup.

Step 7: Cut over and drain Redis

When every job class has moved:

1. Set config.active_job.queue_adapter = :solid_queue and remove the per-class overrides.
2. Deploy. New jobs now go to Solid Queue.
3. Keep Sidekiq running until its queues, scheduled set and retry set are all empty. Scheduled jobs and retries can sit there for days, so check the Sidekiq dashboard rather than guessing.
4. Once everything has drained, stop Sidekiq, remove the gem and its config, and only then shut down Redis.

Skipping step 3 is how teams lose jobs. Don’t skip step 3.

What to watch after the switch

• Queue latency. How long jobs wait before starting. If it creeps up, add threads or processes.
• Database load. Polling is cheap, but it isn’t free. The default worker polling interval is 0.1 seconds; raise it for queues that don’t need near-instant pickup.
• Crashed processes. Jobs claimed by a process that dies are marked failed with ProcessPrunedError, not retried automatically. Decide how you want to handle those.

Is it worth it?

For most apps we work on, yes: one less service, one less bill, one less thing to page someone about, and jobs that live in the same database you already back up. For the few apps where it isn’t, Sidekiq is still right there, and still great.

New to Rails 8 overall? Start with what’s new in Rails 8. If you’d rather have someone run this migration with you, that’s what we do.

Sources

• Solid Queue README — GitHub
• Rails 8.0: No PaaS Required — rubyonrails.org
• Active Job Basics — Rails Guides