sprinboot scheduling using shedlock and quartz

Efficient Job Scheduling and Locking in Spring Boot Applications: Quartz vs. ShedLock

In modern Spring Boot applications running in Kubernetes clusters, it’s crucial to efficiently schedule and manage recurring jobs while ensuring they run smoothly in a highly concurrent and parallel environment. In this article, we will explore two popular options for job scheduling and locking: Quartz and ShedLock. We’ll discuss their implementation and demonstrate how they can help maintain job execution integrity in a distributed environment.

1. Quartz Scheduler

Quartz is a widely-used job scheduling library that provides robust features for scheduling tasks in Java applications. It offers powerful scheduling capabilities, making it a great choice for managing cron-like tasks within your Spring Boot application.

Setting Up Quartz Scheduler
To use Quartz Scheduler in your Spring Boot application, you need to add the Quartz dependency to your project’s build.gradle or pom.xml file.

<dependency>
 <groupId>org.quartz-scheduler</groupId>
 <artifactId>quartz</artifactId>
 <version>${lasted.version}</version>
</dependency>

Creating a Quartz Job
To execute Quartz jobs in a distributed environment with locking for executions, you’ll need to configure Quartz to use a database-backed job store and set up job locking. Below is a step-by-step guide on how to achieve this.

• Database Configuration

To enable distributed job execution, you need to configure Quartz to use a database-backed job store. This allows multiple instances of your Spring Boot application to share the same job scheduling data.

You can configure the database properties in your application.properties or application.yml file, depending on your preferred configuration style. For example, using H2 as an in-memory database (or other depending on your preference):

spring.datasource.url=jdbc:h2:mem:quartz
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password

• Quartz Configuration

Create a Quartz configuration class in your Spring Boot application to configure Quartz with database support:

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
import org.springframework.scheduling.quartz.SpringBeanJobFactory;

@Configuration
public class QuartzConfig {

    @Bean
    public SchedulerFactoryBean schedulerFactoryBean() {
        SchedulerFactoryBean factory = new SchedulerFactoryBean();
        factory.setJobFactory(new SpringBeanJobFactory());
        factory.setDataSource(dataSource); // Inject your data source here
        factory.setQuartzProperties(quartzProperties());
        factory.setOverwriteExistingJobs(true);
        factory.setWaitForJobsToCompleteOnShutdown(true);
        return factory;
    }

    // Configure Quartz properties (e.g., thread count, clustering, etc.)
    private Properties quartzProperties() {
        Properties properties = new Properties();
        properties.setProperty("org.quartz.scheduler.instanceName", "MyScheduler");
        properties.setProperty("org.quartz.scheduler.instanceId", "AUTO");
        properties.setProperty("org.quartz.threadPool.threadCount", "5"); // Set the number of worker threads
        properties.setProperty("org.quartz.jobStore.isClustered", "true");
        properties.setProperty("org.quartz.jobStore.clusterCheckinInterval", "2000"); // Interval for cluster node check-in
        properties.setProperty("org.quartz.jobStore.class", "org.quartz.impl.jdbcjobstore.JobStoreTX");
        properties.setProperty("org.quartz.jobStore.driverDelegateClass", "org.quartz.impl.jdbcjobstore.StdJDBCDelegate");
        properties.setProperty("org.quartz.jobStore.tablePrefix", "QRTZ_");
        return properties;
    }
}

• Locking with Quartz

Quartz provides built-in support for locking jobs to prevent concurrent executions. You can set the @DisallowConcurrentExecution annotation on your job classes to ensure that only one instance of a job runs at a time:

import org.quartz.DisallowConcurrentExecution;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

@DisallowConcurrentExecution
public class MyQuartzJob implements Job {
    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        // Your job logic here
    }
}

By adding @DisallowConcurrentExecution, Quartz will automatically ensure that a job instance is locked while it's running, preventing concurrent executions of the same job.

• Triggering Jobs

You can trigger Quartz jobs as previously described using the @Scheduled annotation on a method within a Spring component. Ensure that you inject the Scheduler bean created in your QuartzConfig class.

import org.quartz.Scheduler;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

@Component
public class JobScheduler {
    @Autowired
    private Scheduler scheduler;

    /**
     * Define your cron expression, 
     * better if it is on properties configurations
     */
    @Scheduled(cron = "0 0/5 * * * ?")
    public void scheduleJob() {
        // Schedule your Quartz job here using the Scheduler
    }
}

With these configurations in place, your Quartz jobs will run in a distributed environment with locking to ensure only one instance of each job is executed at a time, even in a Kubernetes cluster. This setup allows for high availability and prevents job duplication while ensuring efficient job execution.

Advantages of Quartz Scheduler

• Robust scheduling capabilities with cron-like expressions;
• Distributed job execution support;
• Extensive features for job management and persistence;
• Suitable for complex scheduling requirements.

2. ShedLock: Distributed Locking and Scheduling

ShedLock is a lightweight and efficient library specifically designed for distributed lock management and job scheduling within Spring Boot applications. It ensures that a scheduled job is executed exclusively by only one instance of your application at any given time, even in a distributed environment like a Kubernetes cluster.

Setting Up ShedLock
To use ShedLock in your Spring Boot application, add the ShedLock dependency to your project’s build.gradle or pom.xml file.

<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-spring</artifactId>
    <version>${lasted.version}</version>
</dependency>

Configuration

ShedLock requires minimal configuration. You’ll need to create a LockProvider bean and specify which lock providers to use (e.g., database-backed locks, ZooKeeper, Redis, etc.). Below is an example of configuring ShedLock with a database-backed lock using Spring Data JPA:

import net.javacrumbs.shedlock.core.LockProvider;
import net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import javax.sql.DataSource;

@Configuration
public class ShedLockConfig {

    @Bean
    public LockProvider lockProvider(DataSource dataSource) {
        return new JdbcTemplateLockProvider(JdbcTemplateLockProvider.Configuration
                .builder()
                .withJdbcTemplate(new JdbcTemplate(dataSource))
                .usingDbTime() // Use database time for locks
                .build());
    }
}

This configuration sets up ShedLock to use the database as a lock provider, ensuring that only one instance of your application can acquire the lock for a specific job.

Scheduling with ShedLock:

To schedule a method for execution using ShedLock, use the @SchedulerLock annotation on the method in your Spring component. Specify a unique name for the lock to ensure that other instances of your application won't run the same job concurrently (ensure to enable ShedLock — @EnableSchedulerLock).

import net.javacrumbs.shedlock.core.SchedulerLock;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

@Component
public class MyScheduledJob {

    /**
     * Define your cron expression, 
     * better if it is on properties configurations
     */
    @Scheduled(cron = "0 0/5 * * * ?")
    @SchedulerLock(name = "myScheduledJobLock") // Provide a unique lock name
    public void scheduledMethod() {
        // Your job logic here
    }
}

ShedLock will automatically manage the locking for this scheduled job. It ensures that only one instance of your application, regardless of how many are running in your Kubernetes cluster, will execute the job at any given time. In the code above, the task will be released only when the task is finished.

Advantages of ShedLock

• Simplicity: ShedLock provides an easy-to-use API and requires minimal configuration.
• Distributed Locks: Ensures exclusive job execution in distributed environments.
• Flexible Lock Providers: Supports various lock providers, including databases, Redis, ZooKeeper, and more.
• Spring Integration: Seamlessly integrates with Spring Boot and Spring Scheduler.

ShedLock simplifies distributed lock management and job scheduling within Spring Boot applications. By using ShedLock, you can confidently schedule jobs in a distributed environment like Kubernetes while preventing duplicate executions and ensuring efficient job processing. This lightweight library is a valuable addition to your toolbox for building robust and scalable applications.

Conclusion

In this article, we explored two options for job scheduling and locking in Spring Boot applications: Quartz Scheduler and ShedLock. While Quartz offers robust scheduling capabilities suitable for complex requirements, ShedLock simplifies distributed lock management and ensures job execution exclusivity. The choice between these two solutions depends on your specific use case and requirements. Select the one that best aligns with your project’s needs to maintain job execution integrity in your Kubernetes cluster.