Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[doc](load) add routine load job system table doc #2196

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[doc](load) add routine load job system table doc #2196

wants to merge 1 commit into from

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Mar 16, 2025 •
edited
Loading

Versions

  • dev
  • 3.0
  • 2.1
  • 2.0

Languages

  • Chinese
  • English

Docs Checklist

  • Checked by AI
  • Test Cases Built

@sollhui sollhui mentioned this pull request Mar 16, 2025
Merged
16 tasks
dataroaring pushed a commit to apache/doris that referenced this pull request Mar 17, 2025
@sollhui
[improve](information schema) introduce routine load job system table (…
e18ea53 
…#48963)

### What problem does this PR solve?

Part IV of #48511

doc apache/doris-website#2196

**Introduce routine load job statistic system table:**
```
mysql> show create table information_schema.routine_load_job\G
*************************** 1. row ***************************
       Table: routine_load_job
Create Table: CREATE TABLE `routine_load_job` (
  `JOB_ID` text NULL,
  `JOB_NAME` text NULL,
  `CREATE_TIME` text NULL,
  `PAUSE_TIME` text NULL,
  `END_TIME` text NULL,
  `DB_NAME` text NULL,
  `TABLE_NAME` text NULL,
  `STATE` text NULL,
  `CURRENT_TASK_NUM` text NULL,
  `JOB_PROPERTIES` text NULL,
  `DATA_SOURCE_PROPERTIES` text NULL,
  `CUSTOM_PROPERTIES` text NULL,
  `STATISTIC` text NULL,
  `PROGRESS` text NULL,
  `LAG` text NULL,
  `REASON_OF_STATE_CHANGED` text NULL,
  `ERROR_LOG_URLS` text NULL,
  `USER_NAME` text NULL,
  `CURRENT_ABORT_TASK_NUM` int NULL,
  `IS_ABNORMAL_PAUSE` boolean NULL
) ENGINE=SCHEMA;
1 row in set (0.00 sec)
```

**There are some benefits to empower job with SQL query capability for
statistical information:**

- It can be used in conjunction with metrics add through
#48209 to roughly locate abnormal
jobs when Grafana alarms, and the following SQL can be used:

```
SELECT JOB_NAME
FROM information_schema.routine_load_job_statistics
WHERE CURRENT_ABORT_TASK_NUM > 0
   OR IS_ABNORMAL_PAUSE = TRUE;
```

- User can use the `select * from information_schema.routine_load_job`
instead of the `show routine load`. The advantage is that the `show
routine load` can only be searched by name, but SQL can be very flexible
in locating jobs
sollhui added a commit to sollhui/doris that referenced this pull request Mar 20, 2025
@sollhui
[improve](information schema) introduce routine load job system table (…
fae82ee 
…apache#48963)

Part IV of apache#48511

doc apache/doris-website#2196

**Introduce routine load job statistic system table:**
```
mysql> show create table information_schema.routine_load_job\G
*************************** 1. row ***************************
       Table: routine_load_job
Create Table: CREATE TABLE `routine_load_job` (
  `JOB_ID` text NULL,
  `JOB_NAME` text NULL,
  `CREATE_TIME` text NULL,
  `PAUSE_TIME` text NULL,
  `END_TIME` text NULL,
  `DB_NAME` text NULL,
  `TABLE_NAME` text NULL,
  `STATE` text NULL,
  `CURRENT_TASK_NUM` text NULL,
  `JOB_PROPERTIES` text NULL,
  `DATA_SOURCE_PROPERTIES` text NULL,
  `CUSTOM_PROPERTIES` text NULL,
  `STATISTIC` text NULL,
  `PROGRESS` text NULL,
  `LAG` text NULL,
  `REASON_OF_STATE_CHANGED` text NULL,
  `ERROR_LOG_URLS` text NULL,
  `USER_NAME` text NULL,
  `CURRENT_ABORT_TASK_NUM` int NULL,
  `IS_ABNORMAL_PAUSE` boolean NULL
) ENGINE=SCHEMA;
1 row in set (0.00 sec)
```

**There are some benefits to empower job with SQL query capability for
statistical information:**

- It can be used in conjunction with metrics add through
apache#48209 to roughly locate abnormal
jobs when Grafana alarms, and the following SQL can be used:

```
SELECT JOB_NAME
FROM information_schema.routine_load_job_statistics
WHERE CURRENT_ABORT_TASK_NUM > 0
   OR IS_ABNORMAL_PAUSE = TRUE;
```

- User can use the `select * from information_schema.routine_load_job`
instead of the `show routine load`. The advantage is that the `show
routine load` can only be searched by name, but SQL can be very flexible
in locating jobs
This was referenced Mar 20, 2025
Merged
Open
sollhui added a commit to sollhui/doris that referenced this pull request Mar 20, 2025
@sollhui
[improve](information schema) introduce routine load job system table (…
d17cb53 
…apache#48963)

Part IV of apache#48511

doc apache/doris-website#2196

**Introduce routine load job statistic system table:**
```
mysql> show create table information_schema.routine_load_job\G
*************************** 1. row ***************************
       Table: routine_load_job
Create Table: CREATE TABLE `routine_load_job` (
  `JOB_ID` text NULL,
  `JOB_NAME` text NULL,
  `CREATE_TIME` text NULL,
  `PAUSE_TIME` text NULL,
  `END_TIME` text NULL,
  `DB_NAME` text NULL,
  `TABLE_NAME` text NULL,
  `STATE` text NULL,
  `CURRENT_TASK_NUM` text NULL,
  `JOB_PROPERTIES` text NULL,
  `DATA_SOURCE_PROPERTIES` text NULL,
  `CUSTOM_PROPERTIES` text NULL,
  `STATISTIC` text NULL,
  `PROGRESS` text NULL,
  `LAG` text NULL,
  `REASON_OF_STATE_CHANGED` text NULL,
  `ERROR_LOG_URLS` text NULL,
  `USER_NAME` text NULL,
  `CURRENT_ABORT_TASK_NUM` int NULL,
  `IS_ABNORMAL_PAUSE` boolean NULL
) ENGINE=SCHEMA;
1 row in set (0.00 sec)
```

**There are some benefits to empower job with SQL query capability for
statistical information:**

- It can be used in conjunction with metrics add through
apache#48209 to roughly locate abnormal
jobs when Grafana alarms, and the following SQL can be used:

```
SELECT JOB_NAME
FROM information_schema.routine_load_job_statistics
WHERE CURRENT_ABORT_TASK_NUM > 0
   OR IS_ABNORMAL_PAUSE = TRUE;
```

- User can use the `select * from information_schema.routine_load_job`
instead of the `show routine load`. The advantage is that the `show
routine load` can only be searched by name, but SQL can be very flexible
in locating jobs
liaoxin01
liaoxin01 approved these changes Mar 20, 2025
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

liaoxin01
liaoxin01 reviewed Mar 20, 2025
View reviewed changes
docs/admin-manual/system-tables/information_schema/routine_load_job.md

## Overview

Used to view information about routine load jobs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please describe from which version it is supported.

dataroaring pushed a commit to apache/doris that referenced this pull request Mar 25, 2025
@sollhui
[improve](information schema) introduce routine load job system table (…
a298b78 
…#48963) (#49284)

pick #48963

Part IV of #48511

doc apache/doris-website#2196

**Introduce routine load job statistic system table:**
```
mysql> show create table information_schema.routine_load_job\G
*************************** 1. row ***************************
       Table: routine_load_job
Create Table: CREATE TABLE `routine_load_job` (
  `JOB_ID` text NULL,
  `JOB_NAME` text NULL,
  `CREATE_TIME` text NULL,
  `PAUSE_TIME` text NULL,
  `END_TIME` text NULL,
  `DB_NAME` text NULL,
  `TABLE_NAME` text NULL,
  `STATE` text NULL,
  `CURRENT_TASK_NUM` text NULL,
  `JOB_PROPERTIES` text NULL,
  `DATA_SOURCE_PROPERTIES` text NULL,
  `CUSTOM_PROPERTIES` text NULL,
  `STATISTIC` text NULL,
  `PROGRESS` text NULL,
  `LAG` text NULL,
  `REASON_OF_STATE_CHANGED` text NULL,
  `ERROR_LOG_URLS` text NULL,
  `USER_NAME` text NULL,
  `CURRENT_ABORT_TASK_NUM` int NULL,
  `IS_ABNORMAL_PAUSE` boolean NULL
) ENGINE=SCHEMA;
1 row in set (0.00 sec)
```

**There are some benefits to empower job with SQL query capability for
statistical information:**

- It can be used in conjunction with metrics add through
#48209 to roughly locate abnormal
jobs when Grafana alarms, and the following SQL can be used:

```
SELECT JOB_NAME
FROM information_schema.routine_load_job_statistics
WHERE CURRENT_ABORT_TASK_NUM > 0
   OR IS_ABNORMAL_PAUSE = TRUE;
```

- User can use the `select * from information_schema.routine_load_job`
instead of the `show routine load`. The advantage is that the `show
routine load` can only be searched by name, but SQL can be very flexible
in locating jobs

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sollhui @liaoxin01