Adding DeScheduled Job Information to Jobs REST API

[Jeremydupras]

Description

The Jobs REST API currently only lists the scheduled jobs which have the fields "descheduled = false and enabled = true". This will allow the user to see all jobs in job scheduler whether or not they are scheduled in both cluster wide call and by_node call.

There is no difference in the API call. The differences can be seen in the Descheduled and Enabled Fields. Additionally, "last_execution_time", "last_expected_execution_time", and "next_expected_execution_time" are all set to null values when the job is removed from the scheduled job info map. See the example JSON output below.

Lastly, when a job is deleted it is removed from both the scheduled job and the deScheduled jobs and will not show in the API call.

{
  "jobs": [
    {
      "job_type": "scheduler_sample_extension",
      "job_id": "jobid1",
      "index_name": ".scheduler_sample_extension",
      "name": "sample-job-it",
      "descheduled": true,
      "enabled": false,
      "enabled_time": "1970-07-23T00:13:01.533Z",
      "last_update_time": "1970-07-23T00:13:01.533Z",
      "last_execution_time": "2025-07-31T21:43:01.53706613Z",
      "last_expected_execution_time": "2025-07-31T21:43:01.533105468Z",
      "next_expected_execution_time": "none",
      "schedule": {
        "type": "interval",
        "start_time": "1970-07-23T00:13:01.533Z",
        "interval": 1,
        "unit": "Minutes",
        "delay": "none"
      },
      "lock_duration": 10,
      "jitter": "none"
    }
  ],
  "failures": [],
  "total_jobs": 1
}

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

❌ Patch coverage is 6.83761% with 109 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.24%. Comparing base (e88586e) to head (dddc207).

Files with missing lines	Patch %	Lines
...nsport/action/TransportGetScheduledInfoAction.java	0.00%	66 Missing ⚠️
...earch/jobscheduler/scheduler/ScheduledJobInfo.java	9.52%	18 Missing and 1 partial ⚠️
...rg/opensearch/jobscheduler/sweeper/JobSweeper.java	6.66%	14 Missing ⚠️
...pensearch/jobscheduler/scheduler/JobScheduler.java	38.46%	7 Missing and 1 partial ⚠️
...arch/jobscheduler/scheduler/JobSchedulingInfo.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #803      +/-   ##
============================================
- Coverage     31.43%   30.24%   -1.20%     
+ Complexity      150      148       -2     
============================================
  Files            34       34              
  Lines          1527     1574      +47     
  Branches        137      145       +8     
============================================
- Hits            480      476       -4     
- Misses         1010     1060      +50     
- Partials         37       38       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jeremydupras]

Many of the changes in this PR are also in #802. Once #802 is merged I will update this PR.

[Jeremydupras]

LastExecutionTime and LastExpectedExecuitionTIme have been added to a disabledJobs. JSON return has been updated.

[cwperks]

Any idea why the PR is showing lines 121-130 as added? I think these are the same on the main branch so I'm confused why GH is showing these as added.

[cwperks]

oh I see now, the removeDeScheduledJob method was added.

[cwperks]

As much as possible, I would like to isolate the changes in ScheduledJobInfo to keep track of job status (enabled/disabled). If a job is fully deleted then we should delete it regardless of status, I don't think we should have multiple branches of logic to check.

[cwperks]

Let's make sure that disabled jobs are removed from the map containing active jobs and vice-versa. We should have a test for both cases. Create job -> disable -> re-enable the job and call list jobs after both disabling and re-enabling. Ensure that the same job only appears in the response once and with expected status.

[cwperks]

Oh I see, there is a bug when deleting a disabled job that it may not also delete corresponding lock?

Maybe we should combine this with the block above since the code within the block is duplicated?

[cwperks]

We should push the logic up into each respective class to put the onus on the class being serialized. The job_parameter class already implements ToXContentObject so we can use that for the parameter, but the JobSchedulingInfo would need to be updated to specify how it should be serialized.

[cwperks]

This is an example of how you can rely on toXContent when constructing a response: https://github.com/opensearch-project/security/blob/main/src/integrationTest/java/org/opensearch/security/api/InternalUsersRestApiIntegrationTest.java#L144-L146

[cwperks]

FYI I found that the code already handles disabled jobs in a couple of (seemingly) repetitive ways:

• The JobScheduler.schedule method aborts fast: https://github.com/opensearch-project/job-scheduler/blob/main/src/main/java/org/opensearch/jobscheduler/scheduler/JobScheduler.java#L74-L76
• JobScheduler.reschedule checks on each instance of a job running if its been descheduled and then doesn't schedule it for a run: https://github.com/opensearch-project/job-scheduler/blob/main/src/main/java/org/opensearch/jobscheduler/scheduler/JobScheduler.java#L183-L208

I think we can actually keep track of all jobs in the same data structure regardless of status as disabled jobs are already handled in many ways.

[Jeremydupras]

Currently working to refactor how descheduled Jobs are stored