Skip to content

Conversation

Jeremydupras
Copy link
Contributor

Description

This API provides a way to get the information within the Job History Index.

API calls
GET /_plugins/_job_scheduler/api/history

With
GET /_plugins/_job_scheduler/api/history/.scheduler_sample_extension-jobid1

GET /_plugins/_job_scheduler/api/history/

The history ID is the "index_id"-"jobid"

Example API response

{
  "total_history": 2,
  "history": {
    ".scheduler_sample_extension-jobid1-1755727454": {
      "job_index_name": ".scheduler_sample_extension",
      "job_id": "jobid1",
      "start_time": 1755727454,
      "completion_status": 0,
      "end_time": 1755727454
    },
    ".scheduler_sample_extension-jobid1-1755727514": {
      "job_index_name": ".scheduler_sample_extension",
      "job_id": "jobid1",
      "start_time": 1755727514,
      "completion_status": 0,
      "end_time": 1755727514
    }
  }
}

Related Issues

Resolves #808

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

cwperks and others added 30 commits August 20, 2025 15:13
* WIP on allowing interval schedule to operate in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Support defining IntervalSchedule in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Use seconds in bwc test

Signed-off-by: Craig Perkins <[email protected]>

* Update runThenListJobs test

Signed-off-by: Craig Perkins <[email protected]>

* Consistent create watcher job

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
…, but metadata exists (opensearch-project#797)

* WIP on allowing interval schedule to operate in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Support defining IntervalSchedule in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Use seconds in bwc test

Signed-off-by: Craig Perkins <[email protected]>

* Update runThenListJobs test

Signed-off-by: Craig Perkins <[email protected]>

* Consistent create watcher job

Signed-off-by: Craig Perkins <[email protected]>

* Add test that disables watcher job and verifies that it stops running, but metadata exists

Signed-off-by: Craig Perkins <[email protected]>

* Disable security manager for bwc test

Signed-off-by: Craig Perkins <[email protected]>

* Always test with security manager

Signed-off-by: Craig Perkins <[email protected]>

* Add java agent to SPI

Signed-off-by: Craig Perkins <[email protected]>

* Ensure that document is searchable immediately

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
…SON response (opensearch-project#801)

* adding active jobs by lock info

Signed-off-by: Jeremy Dupras <[email protected]>

* adding active jobs and tests

Signed-off-by: Jeremy Dupras <[email protected]>

* changing to ActionListener

Signed-off-by: Jeremy Dupras <[email protected]>

* fixing failing multinode test and format errors

Signed-off-by: Jeremy Dupras <[email protected]>

* correcting test

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* trying test fix

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* rerunning tests

Signed-off-by: Jeremy Dupras <[email protected]>

* removing getFirst()

Signed-off-by: Jeremy Dupras <[email protected]>

* removing lock search

Signed-off-by: Jeremy Dupras <[email protected]>

* remiving client

Signed-off-by: Jeremy Dupras <[email protected]>

* adding lock duration

Signed-off-by: Jeremy Dupras <[email protected]>

---------

Signed-off-by: Jeremy Dupras <[email protected]>
Co-authored-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
…arch-project#802)

* adding active jobs by lock info

Signed-off-by: Jeremy Dupras <[email protected]>

* adding active jobs and tests

Signed-off-by: Jeremy Dupras <[email protected]>

* changing to ActionListener

Signed-off-by: Jeremy Dupras <[email protected]>

* fixing failing multinode test and format errors

Signed-off-by: Jeremy Dupras <[email protected]>

* correcting test

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* trying test fix

Signed-off-by: Jeremy Dupras <[email protected]>

* updating test

Signed-off-by: Jeremy Dupras <[email protected]>

* rerunning tests

Signed-off-by: Jeremy Dupras <[email protected]>

* removing getFirst()

Signed-off-by: Jeremy Dupras <[email protected]>

* removing lock search

Signed-off-by: Jeremy Dupras <[email protected]>

* remiving client

Signed-off-by: Jeremy Dupras <[email protected]>

* adding lock duration

Signed-off-by: Jeremy Dupras <[email protected]>

* adding getLocks API

Signed-off-by: Jeremy Dupras <[email protected]>

* adding test for lock transport action

Signed-off-by: Jeremy Dupras <[email protected]>

* adding tests

Signed-off-by: Jeremy Dupras <[email protected]>

* commmiting changes

Signed-off-by: Jeremy Dupras <[email protected]>

* adding lockid path parameter

Signed-off-by: Jeremy Dupras <[email protected]>

* adding tests

Signed-off-by: Jeremy Dupras <[email protected]>

* Empty commit message

Signed-off-by: Jeremy Dupras <[email protected]>

* renaming, reformatting response JSON, test changes

Signed-off-by: Jeremy Dupras <[email protected]>

* changing name in variable

Signed-off-by: Jeremy Dupras <[email protected]>

* fixing test timing

Signed-off-by: Jeremy Dupras <[email protected]>

* Move ser/de logic to LockModel

Signed-off-by: Craig Perkins <[email protected]>

* adressing comments

Signed-off-by: Jeremy Dupras <[email protected]>

---------

Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Co-authored-by: Jeremy Dupras <[email protected]>
Co-authored-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: opensearch-ci <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
* adding descheduled jobs

Signed-off-by: Jeremy Dupras <[email protected]>

* changing check order

Signed-off-by: Jeremy Dupras <[email protected]>

---------

Signed-off-by: Jeremy Dupras <[email protected]>
Co-authored-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
…pensearch-project#800)

* WIP on allowing interval schedule to operate in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Support defining IntervalSchedule in seconds

Signed-off-by: Craig Perkins <[email protected]>

* Use seconds in bwc test

Signed-off-by: Craig Perkins <[email protected]>

* Update runThenListJobs test

Signed-off-by: Craig Perkins <[email protected]>

* Consistent create watcher job

Signed-off-by: Craig Perkins <[email protected]>

* Add test that disables watcher job and verifies that it stops running, but metadata exists

Signed-off-by: Craig Perkins <[email protected]>

* Disable security manager for bwc test

Signed-off-by: Craig Perkins <[email protected]>

* Always test with security manager

Signed-off-by: Craig Perkins <[email protected]>

* Add java agent to SPI

Signed-off-by: Craig Perkins <[email protected]>

* Ensure that document is searchable immediately

Signed-off-by: Craig Perkins <[email protected]>

* Attempt to aggregate codecov across root + sub projects for cumulative report

Signed-off-by: Craig Perkins <[email protected]>

* Fix error

Signed-off-by: Craig Perkins <[email protected]>

* Try this again

Signed-off-by: Craig Perkins <[email protected]>

* Use codecov-action

Signed-off-by: Craig Perkins <[email protected]>

* Fix flakiness

Signed-off-by: Craig Perkins <[email protected]>

* Fix flakiness

Signed-off-by: Craig Perkins <[email protected]>

* Make sure coverage can be written after node exit using JMX

Signed-off-by: Craig Perkins <[email protected]>

* Fix forbidden

Signed-off-by: Craig Perkins <[email protected]>

* Simplify

Signed-off-by: Craig Perkins <[email protected]>

* Skip JMX for multinode tests

Signed-off-by: Craig Perkins <[email protected]>

* Use 0.8.13

Signed-off-by: Craig Perkins <[email protected]>

* declare task dependency

Signed-off-by: Craig Perkins <[email protected]>

* Finalize test with jacoco

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Co-authored-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: opensearch-ci-bot <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Signed-off-by: Jeremy Dupras <[email protected]>
Copy link

codecov bot commented Aug 21, 2025

Codecov Report

❌ Patch coverage is 51.44928% with 67 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.57%. Comparing base (8b2c5fa) to head (f3a7a0e).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...er/transport/action/TransportGetHistoryAction.java 43.67% 48 Missing and 1 partial ⚠️
...heduler/transport/response/GetHistoryResponse.java 56.52% 10 Missing ⚠️
...scheduler/transport/request/GetHistoryRequest.java 38.46% 8 Missing ⚠️

❌ Your project check has failed because the head coverage (68.57%) is below the target coverage (75.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #823      +/-   ##
==========================================
- Coverage   69.99%   68.57%   -1.42%     
==========================================
  Files          38       43       +5     
  Lines        1733     1868     +135     
  Branches      156      160       +4     
==========================================
+ Hits         1213     1281      +68     
- Misses        430      496      +66     
- Partials       90       91       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Jeremy Dupras and others added 2 commits August 22, 2025 07:09
Signed-off-by: Jeremy Dupras <[email protected]>
@cwperks
Copy link
Member

cwperks commented Sep 23, 2025

@Jeremydupras FYI I fixed the conflicts after #833 was merged.

Sorry about the delay in feedback. Taking a pass now.

@cwperks
Copy link
Member

cwperks commented Sep 23, 2025

A few high-level comments on the API response:

{
  "total_history": 2,
  "history": {
    ".scheduler_sample_extension-jobid1-1755727454": {
      "job_index_name": ".scheduler_sample_extension",
      "job_id": "jobid1",
      "start_time": 1755727454,
      "completion_status": 0,
      "end_time": 1755727454
    },
    ".scheduler_sample_extension-jobid1-1755727514": {
      "job_index_name": ".scheduler_sample_extension",
      "job_id": "jobid1",
      "start_time": 1755727514,
      "completion_status": 0,
      "end_time": 1755727514
    }
  }
}
  • Instead of total_history, wdyt about run_count or something similar?
  • IMO history should be a list sorted from most recent to least recent. Perhaps a URL param can be added for sort in future versions?

@cwperks
Copy link
Member

cwperks commented Sep 23, 2025

@Jeremydupras I'll push a commit to resolve the compilation failures.

@cwperks
Copy link
Member

cwperks commented Sep 23, 2025

@Jeremydupras I'm not able to push to the branch, but the compilation issue is in TransportGetHistoryAction where this import:

import org.opensearch.jobscheduler.spi.utils.JobHistoryService;

needs to change to this:

import org.opensearch.jobscheduler.utils.JobHistoryService;

@cwperks cwperks closed this Sep 23, 2025
@cwperks cwperks reopened this Sep 23, 2025
@github-project-automation github-project-automation bot moved this from ✅ Done to 🏗 In progress in Engineering Effectiveness Board Sep 23, 2025
@cwperks
Copy link
Member

cwperks commented Sep 23, 2025

sorry, inadvertently hit Close with comment instead of Comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🏗 In progress

Development

Successfully merging this pull request may close these issues.

[FEATURE] Job execution History index

3 participants