DLStreamer coding agent (draft implementation for validation purposes). by tjanczak · Pull Request #741 · open-edge-platform/dlstreamer

tjanczak · 2026-04-01T16:00:17Z

Description

Instructions for AI Coding Agents on how to generate new Python sample apps.
Work in progress - PR submitted for validation purposes on 'clean machines'.

Fixes # (issue)

Any Newly Introduced Dependencies

No new dependencies.

How Has This Been Tested?

Experimental feature, work in progress.

Checklist:

I agree to use the MIT license for my code changes.
I have not introduced any 3rd party components incompatible with MIT.
I have not included any company confidential information, trade secret, password or security token.
I have performed a self-review of my code.

… not intended for EXTERNAL use yet.

qianlongding · 2026-04-02T03:21:16Z

I’m thrilled about the new development direction for DLStreamer: creating skills for AI agents to invoke.
Instead of providing samples to user, invoking the skills through AI agents, we will significantly help user to accelerate the process of generating custom pipelines tailored to their specific business scenarios by typing nature language.

Copilot

Pull request overview

Adds a draft “DLStreamer coding agent” skill package intended to guide AI coding agents in generating new DLStreamer Python sample apps from natural-language pipeline descriptions.

Changes:

Introduces the dlstreamer-coding-agent skill definition (SKILL.md) with usage guidance, reference samples, and a generation procedure.
Adds reference documentation for pipeline construction, model preparation, design patterns, and coding conventions.
Adds reusable assets/templates for generated sample app READMEs and Python/model-export scripts.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
`.github/skills/dlstreamer-coding-agent/SKILL.md`	Defines when/how to use the skill and a recommended sample-app layout/procedure.
`.github/skills/dlstreamer-coding-agent/references/pipeline-construction.md`	Reference for elements, patterns, and rules for building pipelines/apps.
`.github/skills/dlstreamer-coding-agent/references/model-preparation.md`	Guidance for downloading/exporting models from common ecosystems to OV IR.
`.github/skills/dlstreamer-coding-agent/references/design-patterns.md`	Catalog of reusable implementation patterns mapped to existing samples.
`.github/skills/dlstreamer-coding-agent/references/coding-conventions.md`	Conventions for headers/imports/plugins/errors/metadata access.
`.github/skills/dlstreamer-coding-agent/assets/README-template.md`	README scaffold for generated samples.
`.github/skills/dlstreamer-coding-agent/assets/python-app-template.py`	Starter Python app template (pipeline + IO + event loop).
`.github/skills/dlstreamer-coding-agent/assets/export-models-template.py`	Starter model download/export script template.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/skills/dlstreamer-coding-agent/SKILL.md

.github/skills/dlstreamer-coding-agent/references/pipeline-construction.md

.github/skills/dlstreamer-coding-agent/assets/python-app-template.py

.github/skills/dlstreamer-coding-agent/assets/export-models-template.py

.github/skills/dlstreamer-coding-agent/assets/README-template.md

tjanczak · 2026-04-08T13:34:22Z

I’m thrilled about the new development direction for DLStreamer: creating skills for AI agents to invoke. Instead of providing samples to user, invoking the skills through AI agents, we will significantly help user to accelerate the process of generating custom pipelines tailored to their specific business scenarios by typing nature language.

Feel free to contribute and improve!
I think we need some 'minimum set' of samples for AI Coding Agents to learn from, but indeed we cannot generate samples endlessly - the way to scale is to use AI Coding Agents.

…e-platform/dlstreamer into dlstreamer_coding_agent

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/skills/dlstreamer-coding-agent/SKILL.md

.github/skills/dlstreamer-coding-agent/assets/python-app-template.py

.github/skills/dlstreamer-coding-agent/SKILL.md

qianlongding

AI Harness Documentation: To make this user-friendly, should we provide a brief README or guide on how to integrate these skills with popular AI harnesses (e.g., Claude Code, OpenCode, or Open Codex)? This would help users understand exactly how to 'plug' our skills into their existing AI agent workflows

tjanczak · 2026-04-09T08:13:40Z

AI Harness Documentation: To make this user-friendly, should we provide a brief README or guide on how to integrate these skills with popular AI harnesses (e.g., Claude Code, OpenCode, or Open Codex)? This would help users understand exactly how to 'plug' our skills into their existing AI agent workflows

Yes, this is by no means 'complete' - it is still an early version; the goal is to have it hardened and released to external custoemers as part of OEP 2026.1.

oonyshch · 2026-04-09T10:02:00Z

.github/skills/dlstreamer-coding-agent/SKILL.md

+
+## Procedure
+
+### Quick Recipes


the pattern ids in the table below , like 1 + 4 + 11 require the reader to cross-reference design-patterns.md to understand what they mean.

so i suggest inlining the names to make the table self-contained, e.g.:
1 (pipeline core) + 4 (ai inference) + 11 (file output)

this also reduces the risk of the agent misreading the recipe if pattern numbering ever shifts.

oonyshch · 2026-04-09T10:03:02Z

.github/skills/dlstreamer-coding-agent/SKILL.md

+
+Map the user's description to one or more of these patterns:
+
+| Pattern | When to Apply |


this table lists patterns by name only, while the quick recipes above reference them by number only. an agent reasoning top-to-bottom through this file sees numbers first, then names, with no explicit link between them. adding the pattern numbers as a column here (or as a prefix in the pattern name column) would close that gap.

oonyshch · 2026-04-09T10:04:28Z

.github/skills/dlstreamer-coding-agent/SKILL.md

+    --device /dev/dri \
+    --group-add $(stat -c "%g" /dev/dri/render*) \
+    --device /dev/accel \
+    --group-add $(stat -c "%g" /dev/accel/accel*) \


stat -c "%g" /dev/accel/accel* will fail on systems without an npu because the glob expands to nothing. please maybe wrap the --device /dev/accel and --group-add lines in a conditional, or at minimum add a note that these lines should only be included when npu devices are present on the host.

oonyshch · 2026-04-09T10:07:30Z

.github/skills/dlstreamer-coding-agent/assets/python-app-template.py

+
+    # GPU fallback
+    device = args.device
+    if device == "GPU" and not os.path.exists("/dev/dri/renderD128"):


the gpu detection relies on /dev/dri/renderD128 which is the first intel igpu render node on linux. this could manifest as a silent callback to cpu on systems where the render node is renderD129 or higher (like, discrete gpu), and it won't work at all on windows. a more portable approach would be to query openvino directly, like "GPU" in Core().available_devices

as i was typing this, i remembered @pbartosik found a way to do this in an even better way in Optimizer.

oonyshch · 2026-04-09T10:08:35Z

.github/skills/dlstreamer-coding-agent/references/coding-conventions.md

+
+```python
+det_dev = args.device
+if det_dev == "GPU" and not os.path.exists("/dev/dri/renderD128"):


same issue as in the python-app-template

same issue as in the app template: the recommended gpu check is hardcoded to /dev/dri/renderD128. since this is the canonical convention document that all generated apps will follow, fixing it here would propagate the improvement automatically

oonyshch · 2026-04-09T10:09:50Z

.github/skills/dlstreamer-coding-agent/assets/README-template.md

+
+{{APP_DESCRIPTION}}
+
+![{{APP_TITLE}}]({{APP_IMAGE}})


![{{APP_TITLE}}]({{APP_IMAGE}}) is a required-looking line with no guidance on what to do when no screenshot exists yet. a newly generated app won't have one. i suggest adding a comment like  to prevent the agent from leaving a broken image reference in the final readme

oonyshch · 2026-04-09T10:14:56Z

.github/skills/dlstreamer-coding-agent/references/model-preparation.md

+
+**When to use:** User asks for OCR (PaddleOCR), or any PaddlePaddle model from HuggingFace.
+
+**CRITICAL:** PaddlePaddle v3+ models use PIR format (`.json` + `.pdiparams`), **not** the


the note that paddleocr v3+ uses pir format (.json + .pdiparams) instead of .pdmodel is in the export template. could cause failures during model prep.

please move it to it to the surface explicitly in the paddleocr section of model-preparation.md so the agent encounters it during step 2 research, not only when copying the template.

oonyshch

added comments for potential improvement

Draft implementation of DLStreamer coding agent for testing purpose -…

eb6c579

… not intended for EXTERNAL use yet.

tjanczak and others added 7 commits April 3, 2026 15:47

Merge branch 'main' into dlstreamer_coding_agent

7b4bfd2

Update skill instructions with more accurate rules.

a284e00

Refine to use standalone model download scripts.

918a5cd

More refinements

fb1803a

More tweaks

580752d

Recommend to use latest DLStreamer docker image.

e620936

Add README-template.md.

8582d1a

tjanczak marked this pull request as ready for review April 8, 2026 08:48

tjanczak requested review from BaoHuiling, OskarFiedot, ZiningLi, dmichalo, jmotow, marcin-wadolkowski, mholowni, msmiatac, nszczygl9, oonyshch, pbartosik, qianlongding, tbujewsk, walidbarakat, yangjianfeng1208 and yunowo as code owners April 8, 2026 08:48

Merge branch 'main' into dlstreamer_coding_agent

bf477d5

nszczygl9 requested a review from Copilot April 8, 2026 11:36

Copilot started reviewing on behalf of nszczygl9 April 8, 2026 11:37 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

tbujewsk previously approved these changes Apr 8, 2026

View reviewed changes

tjanczak added 2 commits April 8, 2026 15:43

Apply fixes from code review.

3b23309

Merge branch 'dlstreamer_coding_agent' of https://github.com/open-edg…

20f34ca

…e-platform/dlstreamer into dlstreamer_coding_agent

tjanczak dismissed tbujewsk’s stale review via 20f34ca April 8, 2026 13:44

More fixes from code review.

949a8d8

Copilot AI review requested due to automatic review settings April 8, 2026 13:48

Copilot started reviewing on behalf of tjanczak April 8, 2026 13:49 View session

Merge branch 'main' into dlstreamer_coding_agent

148e417

Copilot AI reviewed Apr 8, 2026

View reviewed changes

tbujewsk approved these changes Apr 8, 2026

View reviewed changes

qianlongding reviewed Apr 9, 2026

View reviewed changes

.github/skills/dlstreamer-coding-agent/SKILL.md Show resolved Hide resolved

qianlongding reviewed Apr 9, 2026

View reviewed changes

.github/skills/dlstreamer-coding-agent/SKILL.md Show resolved Hide resolved

qianlongding reviewed Apr 9, 2026

View reviewed changes

jmotow approved these changes Apr 9, 2026

View reviewed changes

tjanczak merged commit d37cef0 into main Apr 9, 2026
39 checks passed

tjanczak deleted the dlstreamer_coding_agent branch April 9, 2026 08:18

oonyshch reviewed Apr 9, 2026

View reviewed changes

oonyshch mentioned this pull request Apr 13, 2026

Add example prompts for DLS Coding Agent #761

Open

4 tasks


		Map the user's description to one or more of these patterns:

		\| Pattern \| When to Apply \|


		When to use: User asks for OCR (PaddleOCR), or any PaddlePaddle model from HuggingFace.

		CRITICAL: PaddlePaddle v3+ models use PIR format (`.json` + `.pdiparams`), not the


		{{APP_DESCRIPTION}}

		![{{APP_TITLE}}]({{APP_IMAGE}})

Conversation

tjanczak commented Apr 1, 2026

Description

Any Newly Introduced Dependencies

How Has This Been Tested?

Checklist:

Uh oh!

qianlongding commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tjanczak commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qianlongding left a comment

Choose a reason for hiding this comment

Uh oh!

tjanczak commented Apr 9, 2026

Uh oh!

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oonyshch left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants