Track B: Make Neva strong for applied generative AI systems

[emil14]

Tracking

• Parent umbrella tracker: 2026 AI Direction Tracker: Neva for AI Codegen and GenAI Apps #1056

Goal

Make Neva a strong language for building applied generative AI systems (chatbots, tool-using assistants, agent workflows, MCP-enabled systems) with explicit, reliable dataflow semantics.

Problem statement

Python dominates applied AI mostly through ecosystem velocity and convenience. Neva can compete in a focused way: explicit static wiring, deterministic flow control, and built-in parallel data pipelines for robust production workflows.

This track is about turning those language strengths into practical APIs and patterns for real GenAI applications.

Existing context to leverage

• Stdlib usability baseline + dependency map: Stdlib usability stabilization #1033
• IO/file-handle direction: IO API - Neva vs Go (descriptions and explicit opening/closing) #790 (open PR stdlib/io: add explicit Open/Create/Close file-handle API #1040)
• std/os expansion: Implement os package for stdlib #586 (open PR Implement std/os package for env/process/fs primitives #1041)
• Stream redesign for reliable streaming semantics: Redesign of stream #908 (open PR Redesign stream as union and add streams.Enumerate (#908) #1042)
• Producer cancellation / stream interruption gap: No way to interrupt stream producing (iterator protocol/interface, "early return" for stream processing) #666
• Stream processor error-port consistency: All stream processors must have err outport #861
• Visual editor for review/debug workflows: Implement Visual Editor for Source Code level (AST) Program Representation #970

Workstreams

1) Provider/protocol foundation

• Define minimal, provider-agnostic interfaces for LLM calls (OpenAI-compatible schemas first, other providers second).
• Decide stdlib vs external-module boundaries for provider SDK wrappers.
• Keep transport-level primitives explicit (HTTP/SSE/WebSocket) and reusable.

2) Tool calling and typed contracts

• Design a Neva-native way to describe tool contracts (input/output/error semantics).
• Ensure tool invocation flows integrate with existing res/err conventions and static typing.
• Define interoperability path for MCP client/server usage.

3) Streaming and concurrency patterns for chat systems

• Specify recommended patterns for:
  • assistant-response streaming,
  • debounce/buffer/window behavior,
  • per-thread concurrency/locking semantics,
  • cancellation and backpressure.
• Reuse/extend stream primitives instead of ad-hoc runtime behavior.

4) Agent workflow building blocks

• Evaluate primitives for:
  • mailbox/task routing,
  • multi-agent orchestration,
  • deterministic handoff and retries,
  • observability/tracing hooks.
• Start with small composable components; avoid framework-like monolith design.

5) Reference implementations and docs

• Build at least one reference project (for example: tool-using chatbot with streaming output and deterministic guards).
• Publish architecture and API guidance with explicit tradeoffs.

Non-goals

• Not model training / ML research.
• Not replacing Python ecosystem breadth in one release.
• Not introducing hidden control-flow syntax that breaks Neva’s explicit graph model.

Acceptance criteria

• Clear MVP API proposal(s) for LLM provider calls and tool-calling contracts.
• At least one runnable reference app demonstrating streaming + tools + concurrency policy.
• Documented guidance on when to use Neva-native components vs external adapters.
• Follow-up implementation issues are split, labeled, and prioritized.

[emil14]

Todo update this issue

What I think

1. maybe we should start from, or stick with - not std, but third party module, yet official

2. stuff I would care about: LLM, Tools, OpenAI API, MCP, ACP, Skills, etc.

[emil14]

AgentOps re-trigger after webhook enable: please provide concise plan + acceptance criteria.

[emil14]

Implementation Plan for Track B: Applied Generative AI Systems

Based on the issue description and your comment about third-party modules, here's a concrete plan:

1. Start with External Modules Approach

• Create x/llm as an official but external module (not stdlib)
• This allows faster iteration without stdlib stability constraints
• Provides a clear migration path: successful patterns can be standardized later

2. Core Interface Design

• Provider-agnostic LLM interfaces (llm.Provider, llm.ChatCompletion, llm.Embedding)
• OpenAI-compatible schema as first-class citizen
• Tool contract system with typed inputs/outputs using Neva's type system
• Streaming primitives building on Redesign of stream #908 (stream redesign)

3. Key Components to Build

1. x/llm/core - Core interfaces and types

   • Chat completion request/response types
   • Tool definition types
   • Streaming chunk types

2. x/llm/openai - OpenAI provider implementation

   • HTTP transport with SSE support
   • Error handling with Neva's err port convention

3. x/llm/tools - Tool system

   • Tool registry and invocation patterns
   • Integration with existing res/err conventions
   • MCP compatibility layer

4. x/llm/patterns - Common AI patterns

   • Assistant response streaming with debounce
   • Tool calling with typed contracts
   • Multi-agent orchestration primitives

4. Reference Implementation

• Tool-using chatbot demonstrating:
  • Streaming LLM responses
  • Concurrent tool execution
  • Error handling and backpressure
  • Observable dataflow

5. Documentation & Guidance

• When to use Neva-native vs external adapters
• Performance/scalability considerations
• Integration patterns with existing systems
• Migration path from Python/TypeScript AI systems

Acceptance Criteria

1. MVP API proposal with clear type definitions for:

   • LLM provider calls (chat completions, embeddings)
   • Tool contracts with typed I/O
   • Streaming response handling

2. Runnable reference app showing:

   • Streaming + tool calling + concurrency
   • Error propagation through dataflow
   • Backpressure and cancellation

3. Split follow-up issues covering:

   • Core interface definitions
   • Provider implementations (OpenAI first)
   • Tool system implementation
   • Streaming enhancements
   • Example applications

4. Clear dependency map to existing work:

   • Stream redesign (Redesign of stream #908, PR Redesign stream as union and add streams.Enumerate (#908) #1042)
   • IO/file-handle API (stdlib/io: add explicit Open/Create/Close file-handle API #1040)
   • std/os expansion (Implement std/os package for env/process/fs primitives #1041)
   • Visual editor (Implement Visual Editor for Source Code level (AST) Program Representation #970)

Next Steps

1. Create initial issue for x/llm module structure and interfaces
2. Define the tool contract type system
3. Build minimal OpenAI provider with streaming support
4. Create reference chatbot example

This approach respects Neva's strengths (explicit wiring, deterministic flow control) while addressing AI system needs. Starting external allows rapid prototyping before standardizing successful patterns.

[emil14]

Perfect! The comment was successfully posted to issue #1058. The implementation plan has been shared with:

1. External modules approach - Starting with x/llm instead of stdlib for faster iteration
2. Core interface design - Provider-agnostic LLM interfaces with OpenAI compatibility
3. Key components - Core types, OpenAI provider, tool system, patterns
4. Reference implementation - Tool-using chatbot demonstrating streaming + tools + concurrency
5. Acceptance criteria - MVP API proposal, runnable app, split issues, dependency map
6. Next steps - Creating issues for module structure, tool contracts, provider implementation, and examples

The plan addresses the author's comment about using third-party modules and provides a concrete roadmap for making Neva strong for applied generative AI systems while respecting its explicit dataflow semantics.

opencode session  |  github run

[emil14]

I'll analyze this tracking issue and the comments to provide an implementation plan.

Looking at the issue and comments:

1. Issue: A comprehensive tracking issue for making Neva strong for applied generative AI systems
2. Goal: Compete with Python in explicit static wiring, deterministic flow control, and parallel data pipelines for GenAI applications
3. Workstreams: 5 areas covering provider interfaces, tool calling, streaming patterns, agent workflows, and reference implementations
4. Author comment: Suggests starting with third-party modules rather than stdlib for faster iteration
5. Author comment (recent): Confirms the need for a concise plan + acceptance criteria

Based on the author's feedback about using third-party modules and the comprehensive scope, here's my implementation plan:

Implementation Plan: External Modules for Generative AI Systems

1. External Modules Approach (x/llm)

• Start with x/llm as an official-but-external module for rapid iteration
• This allows API evolution without stdlib stability guarantees
• Once patterns solidify, consider promoting to stdlib

2. Core Interface Design

• Provider-agnostic interfaces for LLM calls (OpenAI-compatible schemas first)
• Transport primitives reusable across HTTP/SSE/WebSocket
• Type-safe contracts aligned with Neva's structural typing

3. Key Components

A. Core Types (x/llm/types)

// Provider-agnostic message types
pub type Message {
  role: string
  content: string
  // tool_calls, function_call, etc.
}

pub type ChatCompletionRequest {
  model: string
  messages: []Message
  temperature: float?
  max_tokens: int?
  tools: []Tool?
  // ... other common parameters
}

pub type ChatCompletionResponse {
  id: string
  choices: []Choice
  usage: Usage?
}

pub type Tool {
  type: string  // "function"
  function: Function
}

pub type Function {
  name: string
  description: string?
  parameters: any  // JSON schema
}

B. OpenAI Provider (x/llm/openai)

• Implements core interfaces for OpenAI API
• Handles streaming vs. non-streaming modes
• Includes authentication and error handling

C. Tool System (x/llm/tools)

• Type-safe tool definitions and invocation
• Integration with res/err conventions
• MCP interoperability path

D. Patterns (x/llm/patterns)

• Assistant-response streaming patterns
• Debounce/buffer/window components
• Mailbox/task routing primitives
• Multi-agent orchestration building blocks

4. Reference Implementation

• Tool-using chatbot demonstrating:
  • Streaming LLM responses
  • Tool invocation with typed contracts
  • Concurrent tool execution
  • Error handling and retries
• Built as a standalone example module

5. Acceptance Criteria

MVP API Proposal:

• Provider-agnostic LLM interface in x/llm/types
• OpenAI implementation in x/llm/openai
• Tool contract system in x/llm/tools
• Basic streaming patterns in x/llm/patterns

Runnable Reference App:

• Tool-using chatbot example in examples/llm_chatbot
• Demonstrates streaming + tools + concurrency
• Includes basic error handling patterns

Documentation & Guidance:

• README explaining when to use Neva-native vs external adapters
• Architecture guide for GenAI applications
• Explicit tradeoffs documented

Split Issues:

• Create separate issues for each workstream component
• Prioritize based on dependencies (types → provider → tools → patterns)
• Label appropriately (track-b, genai, enhancement)

6. Next Steps

1. Create x/ module structure with neva.yml and basic imports
2. Design tool contracts focusing on type safety and res/err conventions
3. Implement OpenAI provider with streaming support
4. Build reference chatbot to validate patterns
5. Document patterns and tradeoffs

This approach respects Neva's explicit dataflow semantics while providing practical APIs for GenAI applications. Starting with external modules (x/llm) allows rapid iteration before considering stdlib inclusion.

The plan aligns with the author's suggestion of using third-party modules and provides a clear path from foundational interfaces to complete reference implementations.

opencode session  |  github run