Optimizing background jobs execution with Delayed::UniqueJob

Efficiency in deferred job execution is crucial for any web application, particularly when dealing with time-consuming or resource-intensive tasks. In this context, the Delayed Job gem proves to be an invaluable tool for handling background tasks effectively. However, we often face a common challenge: how do we ensure that a job is enqueued only once?

Unveiling the issue

Picture this common scenario: a user performs an action through a dashboard, which results in enqueuing a job. Simultaneously, an automatic cron job schedules the same task. This is where the predicament arises - a job could potentially be enqueued multiple times.

Moreover, consider a situation where a job, once enqueued, might take a considerable amount of time to complete. If the job is inadvertently enqueued more than once, either by user interaction or automated processes, it could lead to redundancy and wasted resources.

Existing solutions to this problem

Delayed Job doesn’t have concurrency controls out of the box. If we need to ensure unique jobs are enqueued, we need to find other options:

• good_job comes with concurrency controls out of the box. You can prevent duplicate jobs from being enqueued or performed.
• solid_queue comes with concurrency controls out of the box. You can prevent duplicate jobs from being performed at the same time, but they are never discarded, just blocked.
• sidekiq comes with concurrency controls, but only the enterprise version of it.

While these solutions effectively address the issue of unique job enqueuing, there are cases where sticking with Delayed Job becomes necessary, either due to project constraints or a preference for Delayed Job mature and stable foundations.

Analysis of Delayed Job

Delayed Job, being a mature and established library, has its strengths and limitations. While it lacks certain features like built-in support for concurrency controls, it remains a reliable choice with a straightforward design. However, it’s essential to acknowledge that features available out-of-the-box in other job processing frameworks, such as Sidekiq or good_job, might require additional gems when using Delayed Job.

In our specific case, we have project constraints to stick with Delayed Job, so we need to find a solution.

Introducing Delayed::UniqueJob

Delayed::UniqueJob provides a seamless solution for handling unique jobs.

This plugin introduces the #enqueue_once method, designed to ensure that a job is enqueued only once, regardless of how many attempts are made. To do this, the job must define a #unique_key method that #enqueue_once will use to determine the uniqueness of a job. This could be as simple as a string, or include data from objects.

Here’s a quick guide on how to integrate and use this gem in your Ruby on Rails application:

class MyUniqueJob
  def perform
    # Your job logic
  end

  def unique_key
    "my-unique-job"
    # or you could constrain the job to run once per object, returning something like
    # "my-unique-job-#{user.id}"
  end
end

# #enqueue_once returns true if the job was enqueued, or false if it was already enqueued
if Delayed::Job.enqueue_once(MyUniqueJob.new)
  # job enqueued
else
  # job was already enqueued
end

Next Steps

As you embrace the efficiency brought by Delayed::UniqueJob, consider exploring further optimizations in your job execution workflow. Evaluate additional features provided by related gems to tailor the solution to your application’s unique needs.

There are many alternatives, we need to analyze which is the best fit for our project.