Skip to content

Commit 49a1270

Browse files
authored
[Feature-8222][python] move examples into the scope of source package (#8340)
1 parent 703d790 commit 49a1270

21 files changed

+41
-18
lines changed

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/condition.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A condition task type's example and dive into information of **PyDolphinSchedule
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_condition_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_condition_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/datax.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A DataX task type's example and dive into information of **PyDolphinScheduler**.
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_datax_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_datax_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/dependent.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A dependent task type's example and dive into information of **PyDolphinSchedule
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_dependent_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_dependent_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/flink.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A flink task type's example and dive into information of **PyDolphinScheduler**.
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_flink_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_flink_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/map_reduce.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ A Map Reduce task type's example and dive into information of **PyDolphinSchedul
2424
Example
2525
-------
2626

27-
.. literalinclude:: ../../../examples/task_map_reduce_example.py
27+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_map_reduce_example.py
2828
:start-after: [start workflow_declare]
2929
:end-before: [end workflow_declare]
3030

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/shell.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A shell task type's example and dive into information of **PyDolphinScheduler**.
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/tutorial.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/tutorial.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end task_relation_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/spark.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A spark task type's example and dive into information of **PyDolphinScheduler**.
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_spark_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_spark_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tasks/switch.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A switch task type's example and dive into information of **PyDolphinScheduler**
2323
Example
2424
-------
2525

26-
.. literalinclude:: ../../../examples/task_switch_example.py
26+
.. literalinclude:: ../../../src/pydolphinscheduler/examples/task_switch_example.py
2727
:start-after: [start workflow_declare]
2828
:end-before: [end workflow_declare]
2929

Diff for: dolphinscheduler-python/pydolphinscheduler/docs/source/tutorial.rst

+7-7
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Overview of Tutorial
2929
Here have an overview of our tutorial, and it look a little complex but do not
3030
worry about that because we explain this example below as detailed as possible.
3131

32-
.. literalinclude:: ../../examples/tutorial.py
32+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
3333
:start-after: [start tutorial]
3434
:end-before: [end tutorial]
3535

@@ -41,7 +41,7 @@ like other Python package. We just create a minimum demo here, so we just import
4141
:class:`pydolphinscheduler.core.process_definition` and
4242
:class:`pydolphinscheduler.tasks.shell`.
4343

44-
.. literalinclude:: ../../examples/tutorial.py
44+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
4545
:start-after: [start package_import]
4646
:end-before: [end package_import]
4747

@@ -60,7 +60,7 @@ interval and schedule start_time, and argument `tenant` which changing workflow'
6060
task running user in the worker, :ref:`section tenant <concept:tenant>` in *PyDolphinScheduler*
6161
:doc:`concept` page have more detail information.
6262

63-
.. literalinclude:: ../../examples/tutorial.py
63+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
6464
:start-after: [start workflow_declare]
6565
:end-before: [end workflow_declare]
6666

@@ -77,7 +77,7 @@ Here we declare four tasks, and bot of them are simple task of
7777
Beside the argument `command`, we also need setting argument `name` for each task *(not
7878
only shell task, `name` is required for each type of task)*.
7979

80-
.. literalinclude:: ../../examples/tutorial.py
80+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
8181
:dedent: 0
8282
:start-after: [start task_declare]
8383
:end-before: [end task_declare]
@@ -99,7 +99,7 @@ In this example, we set task `task_parent` is the upstream task of task
9999
`task_child_one` and `task_child_two`, and task `task_union` is the downstream
100100
task of both these two task.
101101

102-
.. literalinclude:: ../../examples/tutorial.py
102+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
103103
:dedent: 0
104104
:start-after: [start task_relation_declare]
105105
:end-before: [end task_relation_declare]
@@ -124,7 +124,7 @@ to the daemon, and set the schedule time we just declare in `process definition
124124
Now, we could run the Python code like other Python script, for the basic usage run
125125
:code:`python tutorial.py` to trigger and run it.
126126

127-
.. literalinclude:: ../../examples/tutorial.py
127+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
128128
:dedent: 0
129129
:start-after: [start submit_or_run]
130130
:end-before: [end submit_or_run]
@@ -142,7 +142,7 @@ After we run the tutorial code, you could login Apache DolphinScheduler web UI,
142142
go and see the `DolphinScheduler project page`_. they is a new process definition be
143143
created and named "Tutorial". It create by *PyDolphinScheduler* and the DAG graph as below
144144

145-
.. literalinclude:: ../../examples/tutorial.py
145+
.. literalinclude:: ../../src/pydolphinscheduler/examples/tutorial.py
146146
:language: text
147147
:lines: 24-28
148148

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
"""Init examples package which provides users with pydolphinscheduler examples."""

Diff for: dolphinscheduler-python/pydolphinscheduler/tests/example/test_example.py

+5-2
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,8 @@ def test_example_basic():
9696
ex.suffix == ".py"
9797
), f"We expect all examples is python script, but get {ex.name}."
9898

99-
# All except tutorial is end with keyword "_example"
100-
if ex.stem != "tutorial":
99+
# All except tutorial and __init__ is end with keyword "_example"
100+
if ex.stem != "tutorial" and ex.stem != "__init__":
101101
assert ex.stem.endswith(
102102
"_example"
103103
), f"We expect all examples script end with keyword '_example', but get {ex.stem}."
@@ -159,6 +159,9 @@ def test_file_name_in_process_definition(mock_code_version, mock_submit, mock_st
159159
"""
160160
global process_definition_name
161161
for ex in get_all_examples():
162+
# Skip __init__ file
163+
if ex.stem == "__init__":
164+
continue
162165
# Skip bulk_create_example check, cause it contain multiple workflow and
163166
# without one named bulk_create_example
164167
if ex.stem == "bulk_create_example":

Diff for: dolphinscheduler-python/pydolphinscheduler/tests/testing/path.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,9 @@
2323
path_code_tasks = Path(__file__).parent.parent.parent.joinpath(
2424
"src", "pydolphinscheduler", "tasks"
2525
)
26-
path_example = Path(__file__).parent.parent.parent.joinpath("examples")
26+
path_example = Path(__file__).parent.parent.parent.joinpath(
27+
"src", "pydolphinscheduler", "examples"
28+
)
2729
path_doc_tasks = Path(__file__).parent.parent.parent.joinpath("docs", "source", "tasks")
2830

2931

0 commit comments

Comments
 (0)