Ruby & Rails: Making sure rake task won’t slow the site down

If you don’t have multiple cores and/or you have a small VPN, you may end up with a huge slow down of your web app, when rake tasks are executed. This can be a big issue especially when you use something like whenever to perform periodic tasks. Luckily there’s a nice program:

nice is a program found on Unix and Unix-like operating systems such as Linux. It directly maps to a kernel call of the same name. nice is used to invoke a utility or shell script with a particular priority, thus giving the process more or less CPU time than other processes. A niceness of −20 is the highest priority and 19 or 20 is the lowest priority. The default niceness for processes is inherited from its parent process and is usually 0.

With nice you can tell your Linux kernel to give your rake tasks the lowest priority possible. Thanks to that, you can make sure, that any task with higher priority (like web UI server) won’t get stuck because of the background rake task.

Here’s how you can use it:

RAILS_ENV=:environment nice -n 19 bundle exec rake :your_task 

If you use a gem like whenever, you can easily integrate nice with it:

job_type :runner, "cd :path && nice -n 19 script/rails runner -e :environment ':task' :output"

every 30.minutes do
  runner 'Worker.new.perform'
end

or for a rake task with whenever, you can use this:

job_type :runner, "cd :path && :environment_variable=:environment nice -19 bundle exec rake :task --silent :output"

every 15.minutes do
  runner 'your_task'
end

Note: Of course this will help, when CPU is your bottleneck. Keep in mind that you’re rake task might slow down your website because of many more reasons (IO, DB, etc).

10
Sep 2014
POSTED BY
DISCUSSION 5 Comments
  • Another interesting approach is to not run rake task from cron ;-) More precisely: to avoid loading again and again the full rails stack sometimes what you can do is to have your rails application listening on an internal IP address and have your cron pinging the URL to start the processing.

  • Well it is and it isn’t. If you use something like Rack::Timeout, your “internal” controller tasks might get killed in the middle. Of course loading rails stack can be heavy, but on the other hand that’s what make nice nice. As long as you have some CPU power, it won’t cost you a bit ;)

  • I think in your second example you wanted to write “job_type :rake” and “rake ‘your_task'”

  • No – I didn’t want to overwrite “native” rake method. This might be useful for tasks that should hae default CPU priority.

  • Pingback: Making Errbit work faster by keeping it clean and tidy | Running with Rails - Dev Blog not only for Rails fanatics!()