Skip to content

Spring Batch is very slow when using two datasources (one for batch/task and another for the app) #4069

Open
@mlouis08

Description

@mlouis08

Bug description
Spring batch configured with two datasources (one for batch/task and another for the app) is much slower than using one datasource for both (boot configured or custom). Following the spring recommended approach for two datasources (here and here) does not help. Tried this on the sample file-to-jdbc example provided by spring. The datasources (in my case) point to the same MySql DB server but to different schemas.

Timing observed with the provided input file "data/name-list.csv" in the example:

  1. Default Boot configured datasource - 1 second
  2. One custom datasource (for both Batch/Task and App) - 1 second
  3. Two custom datasources (one each for Batch/Task and App) - 90 seconds

Environment
Java 17, Spring Boot 2.4.x, MySql 8 running on Ubuntu 20.04

Steps to reproduce
Run the file-to-jdbc sample app provided by spring with two data sources created per the recommendations in links above in bug description. More details below on the Minimal complete reproducible section.

Expected behavior
Expect to see the similar timing as one datasource for both batch/task and app

Minimal Complete Reproducible example
I used the sample file-to-jdbc as the baseline and changed it to use two datasources. Modified files attached in zip.
file-to-jdbc-ChangedFiles.zip

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions