Skip to content

Conversation

Jeremydupras
Copy link
Contributor

@Jeremydupras Jeremydupras commented Jun 24, 2025

Description

This Transport API gathers and returns all scheduled jobs within the Job Scheduler plugin. The resulting jobs can be displayed either by node or as a total list.

API Call -> GET /_plugins/_job_scheduler/api/jobs
-> GET /_plugins/_job_scheduler/api/jobs?by_node

{
  "jobs": [
    {
      "job_type": "reports-scheduler",
      "job_id": "Ch1gqJcBTMEkmCgKhetp",
      "index_name": ".opendistro-reports-definitions",
      "name": "FAILED_LOGIN3",
      "descheduled": false,
      "enabled": true,
      "enabled_time": "2025-06-25T18:36:27.368Z",
      "last_update_time": "2025-06-25T18:36:27.368Z",
      "last_execution_time": "none",
      "last_expected_execution_time": "none",
      "next_expected_execution_time": "2025-06-30T02:15:00Z",
      "schedule": {
        "type": "cron",
        "timezone": "Africa/Abidjan",
        "expression": "15 2 1,15 * 1"
      },
      "delay": "none",
      "jitter": "none",
      "lock_duration": "no_lock"
    },
    {
      "job_type": "reports-scheduler",
      "job_id": "fvhgqJcBfv0jncv5UMGZ",
      "index_name": ".opendistro-reports-definitions",
      "name": "index_function",
      "descheduled": false,
      "enabled": true,
      "enabled_time": "2025-06-25T18:36:13.848Z",
      "last_update_time": "2025-06-25T18:36:13.848Z",
      "last_execution_time": "none",
      "last_expected_execution_time": "none",
      "next_expected_execution_time": "2025-06-26T18:36:12.053424964Z",
      "schedule": {
        "start_time": "2025-06-25T18:36:12.053Z",
        "unit": "Days",
        "interval": 1,
        "type": "interval"
      },
      "delay": "none",
      "jitter": "none",
      "lock_duration": "no_lock"
    }
  ],
  "failures": [],
  "total_jobs": 2
}

-> BY NODE

{
  "nodes": [
    {
      "node_id": "RiTenTucTveJSPeBplLqIg",
      "node_name": "opensearch-node2",
      "scheduled_job_info": {
        "total_jobs": 1,
        "jobs": [
          {
            "job_type": "reports-scheduler",
            "job_id": "Ch1gqJcBTMEkmCgKhetp",
            "index_name": ".opendistro-reports-definitions",
            "name": "FAILED_LOGIN3",
            "descheduled": false,
            "enabled": true,
            "enabled_time": "2025-06-25T18:36:27.368Z",
            "last_update_time": "2025-06-25T18:36:27.368Z",
            "last_execution_time": "none",
            "last_expected_execution_time": "none",
            "next_expected_execution_time": "2025-06-30T02:15:00Z",
            "schedule": {
              "type": "cron",
              "timezone": "Africa/Abidjan",
              "expression": "15 2 1,15 * 1"
            },
            "delay": "none",
            "jitter": "none",
            "lock_duration": "no_lock"
          }
        ]
      }
    },
    {
      "node_id": "eH1IvTmLScyeYrhGzHnYQw",
      "node_name": "opensearch-node1",
      "scheduled_job_info": {
        "total_jobs": 1,
        "jobs": [
          {
            "job_type": "reports-scheduler",
            "job_id": "fvhgqJcBfv0jncv5UMGZ",
            "index_name": ".opendistro-reports-definitions",
            "name": "index_function",
            "descheduled": false,
            "enabled": true,
            "enabled_time": "2025-06-25T18:36:13.848Z",
            "last_update_time": "2025-06-25T18:36:13.848Z",
            "last_execution_time": "none",
            "last_expected_execution_time": "none",
            "next_expected_execution_time": "2025-06-26T18:36:12.053424964Z",
            "schedule": {
              "start_time": "2025-06-25T18:36:12.053Z",
              "unit": "Days",
              "interval": 1,
              "type": "interval"
            },
            "delay": "none",
            "jitter": "none",
            "lock_duration": "no_lock"
          }
        ]
      }
    }
  ],
  "failures": [],
  "total_jobs": 2
}

Related Issues

Resolves #775
Does not include an extensive history of jobs.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Jun 24, 2025

Codecov Report

Attention: Patch coverage is 10.30928% with 174 lines in your changes missing coverage. Please review.

Project coverage is 33.76%. Comparing base (d98cb15) to head (61530d3).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...nsport/action/TransportGetScheduledInfoAction.java 0.00% 80 Missing ⚠️
...r/transport/response/GetScheduledInfoResponse.java 0.00% 45 Missing ⚠️
...ansport/response/GetScheduledInfoNodeResponse.java 0.00% 18 Missing ⚠️
...transport/request/GetScheduledInfoNodeRequest.java 0.00% 16 Missing ⚠️
...ler/transport/request/GetScheduledInfoRequest.java 31.25% 11 Missing ⚠️
...eduler/rest/action/RestGetScheduledInfoAction.java 75.00% 2 Missing ⚠️
...rg/opensearch/jobscheduler/JobSchedulerPlugin.java 83.33% 1 Missing ⚠️
...earch/jobscheduler/scheduler/ScheduledJobInfo.java 0.00% 1 Missing ⚠️

❌ Your project status has failed because the head coverage (33.76%) is below the target coverage (75.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #786      +/-   ##
============================================
- Coverage     37.67%   33.76%   -3.92%     
- Complexity      135      143       +8     
============================================
  Files            22       29       +7     
  Lines          1189     1380     +191     
  Branches        109      132      +23     
============================================
+ Hits            448      466      +18     
- Misses          704      877     +173     
  Partials         37       37              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.


@Override
public List<Route> routes() {
return List.of(new Route(GET, JobSchedulerPlugin.JS_BASE_URI + "/info"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about renaming the route to GET /_plugins/_job_scheduler/api/jobs?

I could imagine that we would also re-use this Rest action in the future to get info for a particular job.

i.e. GET /_plugins/_job_scheduler/api/jobs/{jobID}

Using that pattern would make JS inline with how the security plugin defines rest APIs: https://docs.opensearch.org/docs/latest/security/access-control/api

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. should be /_plugins/_job_scheduler/api/jobs/{jobID}.
/_plugins/_job_scheduler/api/jobs?by_node if by_node query-param is present then list by node else all jobs. No need for by_node=true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Api call has been changed


try {
// Create a list to hold all job details
List<Map<String, Object>> jobs = new java.util.ArrayList<>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we use an import to avoid using the fully qualified class name here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imports added

String jobId = jobEntry.getKey();
JobSchedulingInfo jobInfo = jobEntry.getValue();

if (jobInfo == null) continue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this scenario actually happen where a jobID exists without jobInfo? (job deleted?) If this can happen should we consider having a warning log?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning log added

// Get job provider type if available
String jobType = "unknown";

if (indexToJobProvider.get(indexName).getJobType() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar as above. Are there instances where the index is not in the indexToJobProvider map? That sounds like an error/warn condition that we should log if it can happen

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The null check has been removed. Index to job provider is required for the initial scheduling of a job.

jobDetails.put("last_update_time", jobInfo.getJobParameter().getLastUpdateTime().toString());
// Add execution information

if (jobInfo.getActualPreviousExecutionTime() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't null valid for a value here if the job has never run before? i.e. I setup a report and schedule it to first run for tomorrow. I think we should still return this value in the response even if it is null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will now be added as "none" if the field is null.

jobDetails.put("index_name", indexName);

// Add job parameter details
if (jobInfo.getJobParameter() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

job parameter is required so we shouldn't need a null check here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Null check removed.

if (jobInfo.getActualPreviousExecutionTime() != null) {
jobDetails.put("last_execution_time", jobInfo.getActualPreviousExecutionTime());
}
if (jobInfo.getExpectedPreviousExecutionTime() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments as above, let's include these in the response even if the values are not defined

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Both info should be included even if they haven't run

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Field will be shown as "none".

jobDetails.put("next_expected_execution_time", jobInfo.getExpectedExecutionTime().toString());
}

// Add next time to execute
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this a repeat of the value above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundancy removed.

);

// Add schedule information
if (jobInfo.getJobParameter().getSchedule() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should never be null. If it is let's make sure to have a warning/error log.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error log added

@@ -0,0 +1,59 @@
<?xml version="1.0" encoding="UTF-8"?>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be removed now.

// Refresh indices to ensure all jobs are available
makeRequest(client(), "POST", "/_refresh", Collections.emptyMap(), null);

Thread.sleep(1000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the sleep required if we are explicitly calling refresh above?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would suggest using Awaitility.await if you need to wait for certain info to be available before proceeding with the tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sleep function call has been removed.

Copy link
Member

@cwperks cwperks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this PR @Jeremydupras! This is a great start and goes a long way towards making the job scheduler more transparent and useful.

Copy link
Member

@DarshitChanpura DarshitChanpura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good and are definitely in the right direction. I have left some suggestions and comments to improve testing.

I also suggest to update PR title to be more descriptive. Something like:
Adds REST API to list jobs with an option to list them per node

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file probably was an accidental push and should be deleted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed.


@Override
public List<Route> routes() {
return List.of(new Route(GET, JobSchedulerPlugin.JS_BASE_URI + "/info"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. should be /_plugins/_job_scheduler/api/jobs/{jobID}.
/_plugins/_job_scheduler/api/jobs?by_node if by_node query-param is present then list by node else all jobs. No need for by_node=true

import org.opensearch.jobscheduler.transport.response.GetScheduledInfoResponse;

public class GetScheduledInfoAction extends ActionType<GetScheduledInfoResponse> {
public static final String NAME = "cluster:admin/opensearch/_job_scheduler/info";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: cluster:admin/opensearch/job_scheduler/info

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to match the new api call

if (jobInfo.getActualPreviousExecutionTime() != null) {
jobDetails.put("last_execution_time", jobInfo.getActualPreviousExecutionTime());
}
if (jobInfo.getExpectedPreviousExecutionTime() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Both info should be included even if they haven't run


// Add next time to execute
java.time.Instant now = java.time.Instant.now();
jobDetails.put(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like a duplicate of above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate has been removed.

scheduleMap.put("interval", intervalSchedule.getInterval());
scheduleMap.put("unit", intervalSchedule.getUnit().toString());
} else if (jobInfo.getJobParameter().getSchedule() instanceof CronSchedule) {
scheduleMap.put("type", "cron");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should replace the values with constants declared in the classes, scheduleMap.put("type", "cron") -> scheduleMap.put("type", CronSchedule.CRON_FIELD);. Same for IntervalSchedule above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to implement this and it was causing the program to crash. I will continue to look into having this implemented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a work around. I needed to change the permissions to the constants.

// Refresh indices to ensure all jobs are available
makeRequest(client(), "POST", "/_refresh", Collections.emptyMap(), null);

Thread.sleep(1000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would suggest using Awaitility.await if you need to wait for certain info to be available before proceeding with the tests

@Jeremydupras Jeremydupras changed the title Lists jobs Adds REST API to list jobs with an option to list them per node Jun 25, 2025
@@ -1,6 +0,0 @@
<component name="CopyrightManager">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 files can remain. I know they are IDE specific files in a codebase that can be run in any editor, but its common to pull this repo into Intellij and these files ensure that copywrite headers are in place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced files

import org.opensearch.jobscheduler.transport.response.GetScheduledInfoResponse;

public class GetScheduledInfoAction extends ActionType<GetScheduledInfoResponse> {
public static final String NAME = "cluster:admin/opensearch/_job_scheduler/api/jobs";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public static final String NAME = "cluster:admin/opensearch/_job_scheduler/api/jobs";
public static final String NAME = "cluster:admin/opensearch/jobscheduler/jobs/all";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

Object jobs = nodeResponse.getScheduledJobInfo().get("jobs");
if (jobs instanceof List) {
for (Object job : (List<?>) jobs) {
if (uniqueJobs.add(job)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too generic here and I think it will add duplicates because its comparing generic Objects from 2 different GetScheduledInfoNodeResponse instead of checking based on job_id.

Instead of Object can this be cast to the more narrow Map<String, Object> or a separate class that contains all of the fields in the API response (like jobInfo or something similar?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the logic to look at job ID's.

jngz-es and others added 2 commits June 30, 2025 16:23
* remove guava dependency

Signed-off-by: Jing Zhang <[email protected]>

* remove more unused dependencies

Signed-off-by: Jing Zhang <[email protected]>

---------

Signed-off-by: Jing Zhang <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Jeremy Dupras and others added 22 commits June 30, 2025 16:23
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
* Add a CHANGELOG and changelog_verifier workflow

Signed-off-by: Craig Perkins <[email protected]>

* spotless

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
@@ -0,0 +1,6 @@
<component name="CopyrightManager">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be at the top level under .idea and not under src

Signed-off-by: Jeremy Dupras <[email protected]>
@cwperks cwperks merged commit 2bf1b98 into opensearch-project:main Jul 2, 2025
12 of 14 checks passed
@github-project-automation github-project-automation bot moved this from 👀 In Review to ✅ Done in Engineering Effectiveness Board Jul 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

[FEATURE] Create REST APIs to list all registered jobs and get jobs actively running in the cluster

6 participants