Ruby on Rails

Set up Monit for Sidekiq on Rails Server

Monit is a handy utility that can be used to monitor memory, processes or services on your Unix machine or server. It can send mail alerts or automatically restart processes as per configuration.

Monit can be installed with a simple command:
sudo apt-get install monit

Start/Stop commands for Monit are:

sudo monit
sudo monit quit

Monit’s behaviour is defined in a control file called monitrc which can be usually found in
sudo vi /etc/monit/monitrc
or
sudo vi /var/monit/monitrc

We’ll be editing this file to configure monit for Sidekiq monitoring.
(A comprehensive list of examples can be found here)

Set up Mail server

You can set up Gmail to send notifications

set mailserver smtp.gmail.com PORT 587
    username "abc@gmail.com" password "xyz123"
    using tlsv1
    with timeout 30 seconds

set alert abc@gmail.com

Set up Sidekiq Service Monitoring

The structure looks like this:

check process sidekiq
    with pidfile path_to_sidekiq_pid
    start program = start_command with timeout x
    stop program = stop_command with timeout y

The path_to_sidekiq_pid is usually found in rails_project/tmp/pids/sidekiq.pid
The start_command and stop_command are series of commands to navigate to project folder and run sidekiq from bundler. The timeout option lets you define the amount of time Monit will wait for the start/stop action to finish before giving up and reporting an error. The default is 30 seconds.

Assuming your home folder is  /home/ubuntu and the path to your rails project is /home/ubuntu/rails_project  , our code now looks like:


check process sidekiq
    with pidfile /home/ubuntu/rails_project/tmp/pids/sidekiq.pid
    start program = "/bin/bash -c ' cd /home/ubuntu/rails_project ;bundle exec sidekiq -e production -L /home/ubuntu/rails_project/log/sidekiq.log -C /home/ubuntu/rails_project/config/sidekiq.yml -P /home/ubuntu/rails_project/tmp/pids/sidekiq.pid'"
    stop program = "/bin/bash -c ' cd /home/ubuntu/rails_project/ ;bundle exec sidekiqctl stop /home/ubuntu/rails_project/tmp/pids/sidekiq.pid 10'"

You can test out the start and stop commands in your terminal to see that they start and stop the correct sidekiq process, without creating multiple processes. If multiple processes are being created, you should check whether the path to sidekiq pidfile is correct and the file is being updated properly.

Next, you should check the syntax of your monit file using

sudo monit -t

To restart the monit daemon with updated configuration, run

sudo monit reload

Now, when you kill the sidekiq process on your server, it should restart automatically.
You can view the monit logfile at  /var/log/monit.log
If the process does not restart, and you see errors like these in log file –

[UTC Apr 13 09:32:28] error : 'sidekiq' process is not running
[UTC Apr 13 09:32:28] info : 'sidekiq' trying to restart
[UTC Apr 13 09:32:28] info : 'sidekiq' start: /bin/bash
[UTC Apr 13 09:32:59] error : 'sidekiq' failed to start

You may have to export your rvm_path variable and run the commands above as the user.
The code will now look like:


check process sidekiq
    with pidfile /home/ubuntu/rails_project/tmp/pids/sidekiq.pid
    start program = "/bin/bash -c 'export rvm_path=/home/ubuntu/.rvm; . $rvm_path/scripts/rvm; cd /home/ubuntu/rails_project ;bundle exec sidekiq -e production -L /home/ubuntu/rails_project/log/sidekiq.log -C /home/ubuntu/rails_project/config/sidekiq.yml -P /home/ubuntu/rails_project/tmp/pids/sidekiq.pid'"  as uid ubuntu as gid ubuntu
    stop program = "/bin/bash -c 'export rvm_path=/home/ubuntu/.rvm; . $rvm_path/scripts/rvm; cd /home/ubuntu/rails_project/ ;bundle exec sidekiqctl stop /home/ubuntu/rails_project/tmp/pids/sidekiq.pid 10'" as uid ubuntu as gid ubuntu

Now, reload monit and try killing the sidekiq process. Monit should restart the process in 30 seconds.
If you still face any problems, drop a comment below, I’d be happy to help.

2 thoughts on “Set up Monit for Sidekiq on Rails Server

  1. Thanks a lot for this awesome blog post, saved a ton of my time. Debugging monit start stop commands can be a real pain sometimes.
    Just a tip for others since monit daemon runs as super user so try with super user environment and run the start/stop commands in command line if it works there it should work in monitrc too, btw in my case I had rvm path set in command line but I still had to add rvm to path in monitrc which was weird.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s