Skip to content

Latest commit

 

History

History
409 lines (332 loc) · 12.4 KB

SECTION_MIGRATION_PLAN.md

File metadata and controls

409 lines (332 loc) · 12.4 KB

Section Migration Plan

Summary

The validator infrastructure is complete. This plan covers migrating configuration sections to use schema validators instead of redundant known dict entries.

Completed Infrastructure

  • validator.py - 733 lines, 18 validators, all 23 ValueTypes mapped
  • Leaf.get_validator() and LeafList.get_validator() - auto-generates validators
  • Section.parse() - uses schema validators as fallback
  • schema_to_json_schema() - JSON Schema export working
  • tests/unit/test_configuration_validator.py - 611 lines, 13 test classes
  • CLI command exabgp schema export - working

Migration Scope

11 files with 117 entries where known dict entries can be removed (schema validators handle them):

File Entries Priority Complexity
process/__init__.py 3 Phase 1 Easy
announce/label.py 1 Phase 1 Easy
announce/vpn.py 1 Phase 1 Easy
neighbor/api.py 9 Phase 1 Easy (all BOOLEAN)
capability.py 10 Phase 2 Medium
operational/__init__.py 8 Phase 2 Medium
neighbor/family.py 5 Phase 2 Medium (dynamic init)
neighbor/nexthop.py 2 Phase 2 Medium (dynamic init)
announce/ip.py 18 Phase 3 Complex
l2vpn/vpls.py 21 Phase 3 Complex
neighbor/__init__.py 25 Phase 3 Complex

Migration Pattern

Before (current state):

class ParseProcess(Section):
    schema = Container(children={
        'encoder': Leaf(type=ValueType.ENUMERATION, choices=['text', 'json'], ...),
        'respawn': Leaf(type=ValueType.BOOLEAN, ...),
        'run': Leaf(type=ValueType.STRING, ...),
    })

    known = {
        'encoder': encoder,      # redundant - schema can handle
        'respawn': boolean,      # redundant - schema can handle
        'run': run,              # redundant - schema can handle
    }
    action = {
        'encoder': 'set-command',
        'respawn': 'set-command',
        'run': 'set-command',
    }

After (migrated):

class ParseProcess(Section):
    schema = Container(children={
        'encoder': Leaf(type=ValueType.ENUMERATION, choices=['text', 'json'], action='set-command', ...),
        'respawn': Leaf(type=ValueType.BOOLEAN, action='set-command', ...),
        'run': Leaf(type=ValueType.STRING, action='set-command', ...),
    })

    known = {}   # Empty - all handled by schema
    action = {}  # Empty - derived from schema

Implementation Phases

Phase 1: Quick Wins (14 entries)

Commit 1: process/init.py

  • Remove: encoder, respawn, run from known
  • Types: ENUMERATION, BOOLEAN, STRING

Commit 2: announce/label.py + announce/vpn.py

  • Remove: label, rd from known
  • Types: LABEL, RD

Commit 3: neighbor/api.py

  • Remove: parsed, packets, consolidate, open, update, notification, keepalive, refresh, operational from known
  • Types: All BOOLEAN

Phase 2: Medium Complexity (25 entries)

Commit 4: capability.py

  • Remove: nexthop, asn4, multi-session, operational, route-refresh, aigp, extended-message, software-version, add-path from known
  • Keep: graceful-restart (complex validation: max 4095, "disable" keyword)
  • Types: BOOLEAN, ENUMERATION

Commit 5: operational/init.py

  • Remove: asm, adm, rpcq, rpcp, apcq, apcp, lpcq, lpcp from known
  • Types: All STRING with append-name action

Commit 6: neighbor/family.py + neighbor/nexthop.py

  • Handle dynamic known construction in __init__
  • Types: ENUMERATION, STRING

Phase 3: Complex Sections (64 entries)

Commit 7: announce/ip.py

  • Remove: origin, med, as-path, local-preference, atomic-aggregate, aggregator, etc. from known
  • Keep entries with complex multi-token parsing
  • Types: Various BGP attributes

Commit 8: l2vpn/vpls.py

  • Remove: rd, endpoint, base, offset, size, origin, med, etc. from known
  • Types: RD, INTEGER, BGP attributes

Commit 9: neighbor/init.py

  • Remove: hold-time, rate-limit, listen, connect, passive, description, etc. from known
  • Types: INTEGER, PORT, BOOLEAN, STRING, IP_ADDRESS, ASN

Files to Modify

Phase File Action
1 src/exabgp/configuration/process/__init__.py Remove 3 known entries
1 src/exabgp/configuration/announce/label.py Remove 1 known entry
1 src/exabgp/configuration/announce/vpn.py Remove 1 known entry
1 src/exabgp/configuration/neighbor/api.py Remove 9 known entries
2 src/exabgp/configuration/capability.py Remove 9 known entries
2 src/exabgp/configuration/operational/__init__.py Remove 8 known entries
2 src/exabgp/configuration/neighbor/family.py Refactor dynamic known
2 src/exabgp/configuration/neighbor/nexthop.py Refactor dynamic known
3 src/exabgp/configuration/announce/ip.py Remove 18 known entries
3 src/exabgp/configuration/l2vpn/vpls.py Remove 21 known entries
3 src/exabgp/configuration/neighbor/__init__.py Remove 25 known entries

Validation

./qa/bin/test_everything  # Must pass after each commit

Detailed Migration Per File

Phase 1 Files

process/__init__.py

known entries to remove:
- 'encoder': encoder
- 'respawn': boolean
- 'run': run

Ensure schema has:
- encoder: Leaf(type=ValueType.ENUMERATION, choices=['text', 'json'], action='set-command')
- respawn: Leaf(type=ValueType.BOOLEAN, action='set-command')
- run: Leaf(type=ValueType.STRING, action='set-command')

announce/label.py

known entries to remove:
- 'label': label

Ensure schema has:
- label: Leaf(type=ValueType.LABEL, action='nlri-set')

announce/vpn.py

known entries to remove:
- 'rd': route_distinguisher

Ensure schema has:
- rd: Leaf(type=ValueType.RD, action='nlri-set')

neighbor/api.py

In ParseSend and ParseReceive classes:
known entries to remove:
- 'parsed': boolean
- 'packets': boolean
- 'consolidate': boolean
- 'open': boolean
- 'update': boolean
- 'notification': boolean
- 'keepalive': boolean
- 'refresh': boolean
- 'operational': boolean

All should be Leaf(type=ValueType.BOOLEAN, action='set-command')

Phase 2 Files

capability.py

known entries to remove:
- 'nexthop': boolean
- 'asn4': boolean
- 'multi-session': boolean
- 'operational': boolean
- 'route-refresh': boolean
- 'aigp': boolean
- 'extended-message': boolean
- 'software-version': boolean
- 'add-path': add_path (ENUMERATION)

KEEP in known (complex validation):
- 'graceful-restart': graceful_restart (special "disable" handling, max 4095)

operational/__init__.py

known entries to remove:
- 'asm': asm
- 'adm': adm
- 'rpcq': rpcq
- 'rpcp': rpcp
- 'apcq': apcq
- 'apcp': apcp
- 'lpcq': lpcq
- 'lpcp': lpcp

All should be Leaf(type=ValueType.STRING, action='append-name')

neighbor/family.py

Dynamic known construction - refactor to use schema:
- 'ipv4': ENUMERATION with SAFI choices
- 'ipv6': ENUMERATION with SAFI choices
- 'l2vpn': ENUMERATION with SAFI choices
- 'bgp-ls': ENUMERATION with SAFI choices
- 'all': BOOLEAN

neighbor/nexthop.py

Dynamic known construction - refactor to use schema:
- 'ipv4': STRING (SAFI with alternate next-hop)
- 'ipv6': STRING (SAFI with alternate next-hop)

Phase 3 Files

announce/ip.py

known entries to remove:
- 'origin': origin
- 'med': med
- 'local-preference': local_preference
- 'atomic-aggregate': atomic_aggregate
- 'aggregator': aggregator
- 'originator-id': originator_id
- 'cluster-list': cluster_list
- 'community': community
- 'large-community': large_community
- 'extended-community': extended_community
- 'aigp': aigp
- 'name': named
- 'split': split
- 'watchdog': watchdog
- 'withdraw': withdraw
- 'label': label
- 'attribute': attribute
- 'as-path': as_path (may need to keep - complex multi-token)

l2vpn/vpls.py

known entries to remove:
- 'rd': route_distinguisher
- 'endpoint': integer
- 'base': integer
- 'offset': integer
- 'size': integer
- Plus all BGP attribute entries inherited from announce

neighbor/__init__.py

known entries to remove:
- 'hold-time': ttl
- 'rate-limit': rate_limit
- 'incoming-ttl': incoming_ttl
- 'outgoing-ttl': outgoing_ttl
- 'passive': boolean
- 'listen': port
- 'connect': port
- 'auto-flush': boolean
- 'group-updates': boolean
- 'adj-rib-in': boolean
- 'adj-rib-out': boolean
- 'manual-eor': boolean
- 'md5-base64': boolean
- 'description': description
- 'host-name': host_name
- 'domain-name': domain_name
- 'source-interface': source_interface
- 'router-id': ip
- 'local-as': auto_asn
- 'peer-as': auto_asn
- 'local-address': ip
- 'peer-address': peer_ip

Type mappings:
- INTEGER: hold-time, rate-limit, incoming-ttl, outgoing-ttl
- BOOLEAN: passive, auto-flush, group-updates, adj-rib-in, adj-rib-out, manual-eor, md5-base64
- PORT: listen, connect
- STRING: description, host-name, domain-name, source-interface
- IP_ADDRESS: router-id, local-address
- ASN: local-as, peer-as (with auto)
- IP_RANGE: peer-address

Success Criteria

  1. All migrated entries removed from known dict
  2. Schema provides validation (via _validator_from_schema())
  3. Action derived from schema (via _action_from_schema())
  4. All existing tests pass
  5. Existing configuration files work unchanged
  6. Code is cleaner with less duplication

Notes

  • Keep complex parsers: Entries like graceful-restart, as-path with multi-token parsing stay in known
  • Dynamic known: Some sections build known in __init__ - may need refactoring
  • Actions: Ensure schema Leaf has correct action field before removing from action dict
  • Test after each commit: Run ./qa/bin/test_everything to validate

Implementation Status

Completed ✅

Phase 1:

  • process/__init__.py: encoder, respawn migrated (kept run - returns list[str])
  • neighbor/api.py: All 9 boolean entries migrated

Phase 2 (Partial):

  • capability.py: 8 boolean entries migrated (kept add-path and graceful-restart)
  • schema.py: Fixed Leaf.get_validator() to configure BooleanValidator defaults

Phase 3:

  • neighbor/__init__.py: 16 entries migrated to schema validators:
    • description, host-name, domain-name, source-interface, md5-password
    • passive, listen, connect, group-updates, auto-flush
    • adj-rib-out, adj-rib-in, manual-eor
    • peer-address, md5-ip, rate-limit

Cannot Migrate ❌

After implementation analysis, several sections in the plan cannot be migrated because:

1. Custom parsing functions bypass Section.parse():

File Reason
announce/ip.py Custom ip() function directly accesses known[command]
l2vpn/vpls.py Custom vpls() function in __init__.py directly accesses known[command]

2. Parsers return complex objects that schema validators cannot produce:

File Reason
operational/__init__.py Parsers return Advisory/Query/Response objects, NOT strings
neighbor/family.py Parsers return (AFI, SAFI) tuples with state tracking (_all, _seen)
neighbor/nexthop.py Parsers return (AFI, SAFI, AFI) tuples
capability.py add-path Returns int (0,1,2,3), but ENUMERATION validator returns strings
process/__init__.py run Returns list[str] and validates file existence

3. Parsers return optional types:

Entry Reason
neighbor/__init__.py hold-time Returns HoldTime object, not int
neighbor/__init__.py router-id Returns RouterID object, not IP
neighbor/__init__.py local-address Returns `IP
neighbor/__init__.py local-as Returns `ASN
neighbor/__init__.py peer-as Returns `ASN
neighbor/__init__.py outgoing-ttl Returns `int
neighbor/__init__.py incoming-ttl Returns `int
neighbor/__init__.py md5-base64 Returns `bool
neighbor/__init__.py inherit Returns list[str], not str

Migration Rule

Only migrate entries where:

  1. Parser returns same type as schema validator (bool for BOOLEAN, str for STRING/ENUMERATION, int for INTEGER)
  2. No complex object creation (AFI, SAFI, Advisory, etc.)
  3. No state tracking or side effects
  4. Section uses Section.parse() (not custom parsing functions)

Created: 2025-12-02 Updated: 2025-12-02 Status: Phase 1, 2, 3 complete. Migration scope reduced from 117 to ~40 entries due to architectural constraints.