Asynchronous processing with Sidekiq gem in Rails by Srikant M. April 11, 2014 3 Minutes Read Introduction to Sidekiq Sidekiq gem is used to move long running jobs to background for asynchronous processing. It is more efficient, with respect to memory usage, than delayed_job and Resque as it uses threads instead of forks. Need of background process For example, in a mailing application you need to send emails to a large list of recipients. It will take large time to process with the large list of recipients, which is too long for a User to wait for a response after clicking a link. So, it’s better to move the long running task as the background process by using Sidekiq gem. Integration of Sidekiq in a Rails application Steps to integrate Sidekiq gem in a Rails Application : Step#1 – Add the gem to the Gemfile to use Sidekiq with Active Record gem 'sidekiq' You can also view the web interface to manage the background jobs. It isn’t included by default with Sidekiq. To use the web interface we need to add couple of gems to the Gemfile like ‘sinatra‘ and ‘slim‘ gem 'sidekiq', '2.9.0' gem 'sinatra', '1.3.6', require: false gem 'slim', '1.3.6' Run “bundle install” to install the sidekiq gem Step#2 – Install Redis Server to manage its job queue Redis is an open-source, advanced key-value pair store. It is often referred to as a data structure server. Sidekiq uses Redis to manage its job queue. If you’re running OS X follow the steps as below: Logged in as a Admin user or add sudo prefix on terminal $ brew install redis After installation completed starts the Redis server up with the following command $ redis-server /usr/local/etc/redis.conf if you’re running CentOS 5.6 follow the steps as below: yum install make gcc wget telnet wget http://redis.googlecode.com/files/redis-2.2.12.tar.gz tar -xf redis-2.2.12.tar.gz cd redis-2.2.12 make && make install Change the default redis.conf file to daemonize it. mkdir /etc/redis /var/lib/redis sed -e "s/^daemonize no$/daemonize yes/" -e "s/^dir \.\//dir \/var\/lib\/redis\//" -e "s/^loglevel debug$/loglevel notice/" -e "s/^logfile stdout$/logfile \/var\/log\/redis.log/" redis.conf > /etc/redis/redis.conf To make management easy, use an init script by using sed to update it. Wget https://raw.github.com/gist/257849/9f1e627e0b7dbe68882fa2b7bdb1b2b263522004/redis-server sed -i "s/usr\/local\/sbin\/redis/usr\/local\/bin\/redis/" redis-server chmod u+x redis-server mv redis-server /etc/init.d /sbin/chkconfig --add redis-server /sbin/chkconfig --level 345 redis-server on /sbin/service redis-server start Step#3 – Start up the jobs process There are couple of ways to do this, If application is in development mode, we should use the below rake task file instead. bundle exec sidekiq If application is in production mode, then it is preferred to daemonizes the job process and allows multiple background processes to be spawned. To use this, follow the following step: 1. Add gem “daemon-spawn”, “0.4.2” to your Gemfile 2. Run bundle install 3. Run the below command to run it in background bundle exec sidekiq -d -L log/delayed_job.log 4. You have to pass path of log file while using above command. Step#4 – Add task to run in background We need to add the background work in a separate class and store it in a new directory, let’s say app/background_jobs and ensures that it’s auto-loaded by the application. We need to include the Sidekiq::Worker module. class NotifyUsers include Sidekiq::Worker def perform User.find_each(is_subscribed: true) do |user| NewsMailer.newsletter_mail(user).deliver end end end This class must have a perform method. Add the code that need be to run in the background. Then call NotifyUsers.perform_async method inside the controller which will add the job to Redis and then it will call function perform asynchronously. For Example: Before background job in UsersController class UsersController < ApplicationController def send_email User.find_each(is_subscribed: true) do |user| NewsMailer.newsletter_mail(user).deliver flash[:notice] = "Mail delivered" redirect_to root_path end end end After implementation of UsersController after adding to background job class UsersController < ApplicationController def send_email NotifyUsers.perform_async flash[:notice] = "Mail is being delivered" redirect_to root_path end end end Conclusion No need to wait for a response after clicking a link to do a big stuff Sidekiq has client-side middleware which runs before a job is inserted into Redis It has server-side middleware which runs before the job is being processed. This middleware retry jobs, log them and handle exceptions SEE ALSO: How to Install and Configure Redis-Server on Centos/Fedora Server Sidekiq handles the processing of the jobs after they’re pulled from Redis and multithreaded behavior. It is also providing a web interface there are a couple of gems that we need to add to the gem file too. To visit the web interface just go to the /sidekiq path It tells us how many jobs have been processed, the number of failures, the currently-active workers and the number of jobs in queues. Like this blog? I’d love to hear about your thoughts on this. Thanks for sharing your comments. Related Posts: 4 Simple Steps to Implement “Delayed Job” in Rails You’re just 2 steps away from implementing ‘Backbone’ in Rails 3 How to Install & Configure Redis-Server on Centos/Fedora Server Srikant M. Mohapatra anurag.pattnaik@andolasoft.com No Comments Post A Comment Cancel Reply Save my name, email, and website in this browser for the next time I comment. Current ye@r * Leave this field empty