Skip to content

PCIeConfigurableHierarchy.md

Latest commit

 

History

History
786 lines (761 loc) · 23.7 KB

PCIeConfigurableHierarchy.md

File metadata and controls

786 lines (761 loc) · 23.7 KB

PCIe Configurable Hierarchy

User Guide

Introduction

Configurable PCIe hierarchy allows creation of a customized PCIe hierarchy based on a JSON formatted file.

How To

Add the following parameter to the command line before running the model, with the path to the PCIe hierarchy file.

 -C pci.hierarchy_file_name=/path/to/hierarchy.json

JSON format for hierarchy

An endpoint device is represented in the following format

{
    "<device_type>/<device_name>": {
        "parameter1":<parameter_value>
        "parameter2":<parameter_value>
        "   ...    ":<       ...       >
   }
}

A root port is represented in the following format. A __downstream__ parameter denotes a device connected downstream of the port.

{
    "rootport/<rootport_name>": {
        "parameter1":<parameter value>
        "parameter2":<parameter value>
        "   ...    ":<      ...      >
        "__downstream__":{
            "<device_type>/<device_name>":{
                "parameter3":<parameter value>
                "parameter4":<parameter value>
                "   ...    ":<      ...      >

            }
        }
   }
}

A switch is represented in the following format. A __downstream__<device_number> parameter denotes the device connected downstream of the switch, with the downstream port at device number <device_number> on the switch's internal bus.

{
    "switch/<switch_name>": {
        "parameter1":<parameter value>
        "parameter2":<parameter value>
        "   ...    ":<      ...      >
        "__downstream__<device_number1>": {
            "<device_type>/<device_name>":{
                "parameter3":<parameter value>
                "parameter4":<parameter value>
                "   ...    ":<      ...      >
            }
        },
        "__downstream__<device_number2>"{
            "<device_type>/<device_name>":{
                "parameter5":<parameter value>
                "parameter6":<parameter value>
                "   ...    ":<      ...      >
            }
        }
   }
}

Device Parameters

PCIe Endpoints

PCIe endpoint is the common PCIe frontend for all endpoint type devices in the hierarchy, which are described in later sections. The parameters for PCIe related features of endpoints are described below.

Common endpoint parameters
  • device Device number of the endpoint. Type: int, Min:0, Max:31
  • function Function number for the endpoint. Type: int, Min: 0, Max: 7
  • vendor_id PCI Vendor ID. Type: int, Min:0, Max:0xFFFE
  • device_id PCI Device ID. Type: int, Min:0, Max:0xFFFE
  • base_class PCI Base Class. Type: int, Min:0, Max:255
  • sub_class PCI Sub Class. Type: int, Min:0, Max:255
  • bar0_64bit If BAR 0 is 64 bits wide, if region size is non zero. Type: bool
  • bar1_64bit If BAR 1 is 64 bits wide, if region size is non zero. Type: bool
  • bar2_64bit If BAR 2 is 64 bits wide, if region size is non zero. Type: bool
  • bar3_64bit If BAR 3 is 64 bits wide, if region size is non zero. Type: bool
  • bar4_64bit If BAR 4 is 64 bits wide, if region size is non zero. Type: bool
  • bar0_log2_size Log2 of the size of the region pointed to by BAR 0, zero is reserved which means bar is not used*. Type: int, Min:0, Max:63
  • bar1_log2_size Log2 of the size of the region pointed to by BAR 1, zero is reserved which means bar is not used. Type: int, Min:0, Max:63
  • bar2_log2_size Log2 of the size of the region pointed to by BAR 2, zero is reserved which means bar is not used. Type: int, Min:0, Max:63
  • bar3_log2_size Log2 of the size of the region pointed to by BAR 3, zero is reserved which means bar is not used. Type: int, Min:0, Max:63
  • bar4_log2_size Log2 of the size of the region pointed to by BAR 4, zero is reserved which means bar is not used. Type: int, Min:0, Max:63
  • bar5_log2_size Log2 of the size of the region pointed to by BAR 5, zero is reserved which means bar is not used. Type: int, Min:0, Max:63
  • uses_interrupt Enable support for legacy interrupts. Type: bool
  • interrupt_pin_index Interrupt pin used by this function 1:INTA, 2:INTB, 3:INTC, 4:INTD. Type: int, Min:1, Max:4
  • multi_function Set to true if this endpoint is part of a multi function device. Type: bool
  • pcie_version PCIe version , bits[3:0] in capabilities register. 1 is PCIe3.0. 2 is PCIe4.0. Type: int, Min:1, Max:15
  • prog_iface PCI Programming Interface. Type: int, Min:0, Max:15
  • rev_id PCI Revision ID. Type: int, Min:0, Max:15
  • subsys_id PCI Subsystem ID. Type: int, Min:0, Max:0xFFFE
  • subsys_vendor_id PCI Subsystem Vendor ID. Type: int, Min:0, Max:0xFFFE
  • express_capability_device_type PCI Express Capabilities Device Type, bits[7:4] in capabilities register. 0 is PCIe EndPoint, 9 is RCiEP. Type: int, Min:0, Max:15
  • msix_support Enable device support for MSI-X. Type: bool
  • msix_pba_bar BAR used by MSI-X pending bit array. Type: int, Min:0, Max:5
  • msix_table_bar BAR used by MSI-X table. Type: int, Min:0, Max:5
  • msix_table_size Size of tables for MSI-X. Type: int, Min:0, Max:2048
  • power_mgmt_capability Device supports Power-Management capabilities. Type: bool
  • pasid_supported If set then the PCIe device can emit PASID (SubstreamIDs). Type: bool
  • ext_fmt_field_supported Enable extended format field support. Type: bool
  • extended_tag_supported Extended Tag field support. Type: bool
  • link_port_number Port number for PCIe link. Type: int, Min:0, Max:255
  • pri_supported If set then the PCIe function supports Page Request Interface(requires ATS). Type: bool
  • rber_supported Enable role-based error reporting. Type: bool
  • slot_and_root_status_msi_idx MSI index for reporting slot and root status changes. Type: int, Min:0, Max:2047
  • tag_10bit_completer_supported Enable 10 bit tag completer support. Type: bool
  • tag_10bit_requester_supported Enable 10 bit tag requester support. Type: bool
  • aspm_optionality_compliant Enable ASPM optionality compliance. Type: bool
  • ats_supported If set then the PCIe function supports Address Translation Services(ATS). Type: bool
  • error_injection_supported If set then the PCIe function supports DVSEC Error Injection Capability. Type: bool
  • aer_supported If set then the PCIe function supports Advanced Error Reporting Capability(AER). Type: bool

Exerciser

Exerciser is a PCIe device used to generate various stimuli that can help test PCIe/SMMU integration in the system.

Endpoint parameters default values
  • device: 0
  • function: 0
  • vendor_id: 0x13B5
  • device_id: 0xED01
  • base_class: 0xED
  • sub_class: 0
  • bar0_64bit: false
  • bar1_64bit: false
  • bar2_64bit: false
  • bar3_64bit: false
  • bar4_64bit: false
  • bar0_log2_size: 12
  • bar1_log2_size: 14
  • bar2_log2_size: 15
  • bar3_log2_size: 0
  • bar4_log2_size: 0
  • bar5_log2_size: 12
  • uses_interrupt: true
  • interrupt_pin_index: 1
  • multi_function: false
  • pcie_version: 2
  • prog_iface: 0
  • rev_id: 0
  • subsys_id: 0
  • subsys_vendor_id: 0x13B5
  • express_capability_device_type: 0
  • msix_support: true
  • msix_pba_bar: 4
  • msix_table_bar: 2
  • msix_table_size: 2048
  • power_mgmt_capability: true
  • pasid_supported: false
  • ext_fmt_field_supported: true
  • extended_tag_supported: true
  • link_port_number: 0
  • pri_supported: true
  • rber_supported: true
  • slot_and_root_status_msi_idx: 0
  • tag_10bit_completer_supported: true
  • tag_10bit_requester_supported: true
  • aspm_optionality_compliant: true
  • ats_supported: true
  • error_injection_supported: false
  • aer_supported: false
Exerciser parameters and default values
  • memory_bar BAR index to be used for exerciser memory access. Type: int, Min:0, Max: 5 Default: 1
Example
    "exerciser/exerciser0": {
        "device": 1,
        "function": 0,
        "interrupt_pin_index": 1,
        "memory_bar": 3,
        "express_capability_device_type": 0
    }

AHCI Controller

Advanced Host Controller Interface, or AHCI, is a technical standard for an interface that enables software to communicate with Serial ATA (SATA) devices.

Endpoint parameters default values
  • device: 0
  • function: 0
  • vendor_id: 0xABC*
  • device_id: 0xACED*
  • base_class: 0x1* (Device Class for Mass Storage)
  • sub_class: 0x6* (Sub Class for SATA)
  • bar0_64bit: false
  • bar1_64bit: false
  • bar2_64bit: false
  • bar3_64bit: false
  • bar4_64bit: false
  • bar0_log2_size: 13
  • bar1_log2_size: 13
  • bar2_log2_size: 12
  • bar3_log2_size: 13
  • bar4_log2_size: 12
  • bar5_log2_size: 13
  • uses_interrupt: false*
  • interrupt_pin_index: 1
  • multi_function: false
  • pcie_version: 2
  • prog_iface: 0x1*
  • rev_id: 0x1*
  • subsys_id: 0x2*
  • subsys_vendor_id: 0x13B5*
  • express_capability_device_type: 0
  • msix_support: true*
  • msix_pba_bar: 4*
  • msix_table_bar: 2*
  • msix_table_size: 1*
  • power_mgmt_capability: true
  • pasid_supported: false
  • ext_fmt_field_supported: true
  • extended_tag_supported: true
  • link_port_number: 0
  • pri_supported: false
  • rber_supported: true
  • slot_and_root_status_msi_idx: 0
  • tag_10bit_completer_supported: true
  • tag_10bit_requester_supported: true
  • aspm_optionality_compliant: true
  • ats_supported: false
  • error_injection_supported: false
  • aer_supported: false
AHCI Parameters and Default values
  • force_mode Force disk to report support for at most PIO/DMA/NCQ mode (only for testing/bring-up purposes). PIO mode is always supported. Use NCQ for maximum performance. Default: "NCQ"
  • image_path Comma-separated list of zero or more disk images (up to 32). Each image represents one SATA disk which is connected to one port of the AHCI controller. Empty list elements are allowed and result in a SATA port which has no disk attached. An empty string (default) means one SATA port with no disk attached. Default: ""
  • run_async Do host I/O in a background thread asynchronously. Enabling this makes the simulation non-deterministic and may or may not improve performance. Default: false

* Parameter values that are fixed and cannot be overridden.

Example
"ahci/ahci0": {
        "device": 10,
        "function": 0,
        "image_path": "/path/to/image",
        "express_capability_device_type": 9
    }

Host Bridge

On real systems, Host Bridge is what connects the tree of PCI buses (which are internally connected with PCI-to-PCI Bridges) to the rest of the system. On FVPs, this is merely a placeholder representation of the concept, and does not provide any functionality.

Endpoint parameters default values
  • device: 0
  • function: 0
  • vendor_id: 0x13B5
  • device_id: 0
  • base_class: 0x6*
  • sub_class: 0
  • bar0_64bit: false
  • bar1_64bit: false
  • bar2_64bit: false
  • bar3_64bit: false
  • bar4_64bit: false
  • bar0_log2_size: 12
  • bar1_log2_size: 0
  • bar2_log2_size: 0
  • bar3_log2_size: 0
  • bar4_log2_size: 0
  • bar5_log2_size: 0
  • uses_interrupt: false
  • interrupt_pin_index: 1
  • multi_function: false
  • pcie_version: 2
  • prog_iface: 0xf
  • rev_id: 0xf
  • subsys_id: 0xf
  • subsys_vendor_id: 0x13B5
  • express_capability_device_type: 9
  • msix_support: false
  • msix_pba_bar: 6
  • msix_table_bar: 6
  • msix_table_size: 1
  • power_mgmt_capability: true
  • pasid_supported: false
  • ext_fmt_field_supported: true
  • extended_tag_supported: true
  • link_port_number: 0
  • pri_supported: true
  • rber_supported: true
  • slot_and_root_status_msi_idx: 0
  • tag_10bit_completer_supported: true
  • tag_10bit_requester_supported: true
  • aspm_optionality_compliant: true
Example
"hostbridge/hb": {
        "device": 0,
        "function": 0,
    }

SMMU Test Engine

SMMUv3TestEngine is a PCIe device used to generate various stimuli that can help test SMMU integration in the system.

Endpoint parameters default values
  • device: 0
  • function: 0
  • vendor_id: 0x13B5*
  • device_id: FF80
  • base_class: 0xFF*
  • sub_class: 0
  • bar0_64bit: true
  • bar1_64bit: false
  • bar2_64bit: true
  • bar3_64bit: false
  • bar4_64bit: true
  • bar0_log2_size: 18
  • bar1_log2_size: 0
  • bar2_log2_size: 15*
  • bar3_log2_size: 0
  • bar4_log2_size: 12*
  • bar5_log2_size: 0
  • uses_interrupt: false
  • multi_function: false
  • pcie_version: 2
  • prog_iface: 0x0
  • rev_id: 0xf
  • subsys_id: 0x0
  • subsys_vendor_id: 0x13B5*
  • express_capability_device_type: 9
  • msix_support: true
  • msix_pba_bar: 4
  • msix_table_bar: 2
  • msix_table_size: 2048
  • power_mgmt_capability: true
  • pasid_supported: false
  • ext_fmt_field_supported: true
  • extended_tag_supported: true
  • link_port_number: 0
  • pri_supported: true
  • rber_supported: true
  • slot_and_root_status_msi_idx: 0
  • tag_10bit_completer_supported: true
  • tag_10bit_requester_supported: true
  • aspm_optionality_compliant: true
  • error_injection_supported: false
  • aer_supported: false
Example
"smmuv3testengine/smmute": {
        "device": 0,
        "function": 0,
    }

RootComplex Event Collector (RCEC)

RCEC is a PCIe Type 0 device which signals Error/PME messages received from associated RCiEPs to the upstream PVBus2PCI/RootComplex.

Endpoint parameters default values

  • device: 0
  • function: 0*
  • vendor_id: 0x13B5*
  • device_id: 47B1*
  • base_class: 0x8*
  • sub_class: 0x7*
  • bar0_log2_size: 12
  • uses_interrupt: false
  • pcie_version: 2
  • prog_iface: 0x0
  • rev_id: 0x0
  • subsys_id: 0x0*
  • subsys_vendor_id: 0x13B5
  • express_capability_device_type: 9*
  • msix_support: false
  • msix_pba_bar: 6
  • msix_table_bar: 6
  • msix_table_size: 2048
  • power_mgmt_capability: true
  • pasid_supported: false
  • ext_fmt_field_supported: true
  • extended_tag_supported: true
  • link_port_number: 0
  • rber_supported: true
  • slot_and_root_status_msi_idx: 0
  • tag_10bit_completer_supported: true
  • tag_10bit_requester_supported: true
  • rcec_associated_device_function_info : ""
  • Above parameter takes a string of device-function numbers of RCiEPs associated with this RCEC. Note that an empty string("") represent that no RCiEP is associated with the RCEC. As an example, the user should set this string parameter as per the instructions below:
      params.set("rcec_associated_device_function_info", "8 16 24");

String information is interpreted as stream of associated device-function numbers each separated by a space. Note that for current PCIe RCEC-RCiEP interface implementation, the model doesn't include support for RCEC & RCiEP interfacing between different root bus devices. So all the RCiEPs and their RCECs are connected to the same RootComplex and hence have the same root bus(ecam_start_bus_number). Each device_function number is an 8 bit number with bits [7:3] as device number and bit [2:0] as function number. The input string must contain device-function number only in decimal format.

  • error_injection_supported: false
  • aer_supported: false
Example
"rcec/rcec0": {
        "device": 0,
        "function": 0,
        "msix_support" : true,
        "rciep_associated_device_function_info": "8 16"
    }

Root Port

Parameters and Default values
  • device_number Device number on this bus. Type: int, Min:0, Max:31 Default: 0
  • device_id PCI Device ID. Type: int, Min:0, Max:0xFFFE Default: 0xdef
  • vendor_id PCI Vendor ID. Type: int, Min:0, Max:0xFFFE Default: 0x13B5
  • acs_supported Access control services supported. Type: bool Default: false
  • dpc_supported Downstream Port Containment supported. Type: bool Default: false
  • aspm_optionality_compliant Enable ASPM optionality compliance. Type: bool Default: true
  • ext_fmt_field_supported Enable extended format field support. Type: bool Default: true
  • extended_tag_supported Extended Tag field support. Type: bool Default: true
  • link_port_number Port number for PCIe link. Type: int, Min:0, Max:255 Default: 0
  • pcie_version PCIe version , bits[3:0] in capabilities register. 1 is PCIe3.0. 2 is PCIe4.0. Type: int, Min:1, Max:15 Default: 2
  • rber_supported Enable role-based error reporting. Type: bool Default: true
  • slot_and_root_status_msi_idx MSI index for reporting slot and root status changes. Type: int, Min:0, Max:2047 Default: 0
  • tag_10bit_completer_supported Enable 10-bit tag completer support. Type: bool Default: true
  • tag_10bit_requester_supported Enable 10-bit tag requester support. Type: bool Default: true
  • error_injection_supported If set then the PCIe function supports DVSEC Error Injection Capability. Type: bool Default: false
  • aer_supported If set then the PCIe function supports Advanced Error Reporting Capability(AER). Type: bool Default: false
  • report_advisory_non_fatal_errors If set, then the PCIe function supports Advisory non-fatal error reporting. Type: bool Defaut: false
Example
"rootport/rootport0": {
        "device_number": 5,
        "__downstream__": {
            "ahci/ahci1": {
            "device": 0,
            "function": 0,
            "image_path": "/path/to/image"
            }
        }
    }

Switch

Parameters and Default values
  • device_number Device number on this bus. Type: int, Min:0, Max:31 Default: 0
  • acs_supported Access control services supported. Type: bool Default: false
  • report_advisory_non_fatal_errors If set, then the PCIe function supports Advisory non-fatal error reporting. Type: bool Defaut: false
Example
"switch/switch0": {
                "device_number": 0,
                "__downstream__10": {
                    "ahci/ahci4": {...}
                },
                "__downstream__20": {
                    "ahci/ahci2": {...}
                }
            }

Root Bridge

The term root bridge is taken from the PCI Firmware specification Rev 3.x. It refers to an abstraction for the platform's PCI config and memory I/O access mechanism. A root bridge can be used to create multiple ECAM and MMIO regions within the system's ECAM and MMIO limits.

Parameters and Default values
  • ecam_start ECAM base address. Type:int, Min:<System ECAM base>, Max:<System ECAM end - 1MB>
  • ecam_end_incl ECAM end address. Type:int, Min:<System ECAM base + 1MB>, Max:<System ECAM end>
  • mem32_start 32-bit memory window base address. Type:int, Min:<System MEM32 base>, Max:<System MEM32 end>
  • mem32_end_incl 32-bit memory window end address. Type:int, Min:<System MEM32 base>, Max:<System MEM32 end>
  • mem64_start 64-bit memory window base address. Type:int, Min:<System MEM64 base>, Max:<System MEM64 end>
  • mem64_end_incl 64-bit memory window base address. Type:int, Min:<System MEM64 base>, Max:<System MEM64 end>
  • ecam_start_bus_number ECAM start bus number. Type:int, Min:<System start bus>, Max:<System end bus> Note: Each bus consumes 1MB of memory, so each ecam should have enough space (ecam_start - ecam_end_incl) to accomodate the total number of buses (start_bus - end_bus) allocated to it.
Example
"rootbridge/rb0": {
        "ecam_start": 0x60000000,
        "ecam_end_incl": 0x67ffffff,
        "mem32_start": 0x70000000,
        "mem32_end_incl": 0x73ffffff,
        "mem64_start": 0x500000000,
        "mem64_end_incl": 0x67fffffff,
        "ecam_start_bus_number": 0x0,

        "__downstream__": {
            "ahci/ahci0": {
                "device": 30,
                "function": 0,
                "image_path": "",
                "express_capability_device_type": 9
            },
            "ahci/ahci1": {
                "device": 31,
                "function": 0,
                "express_capability_device_type": 9,
                "image_path": ""
            }
        }
    }

Sample hierarchies

Example 1: Single ECAM

An absence of rootbridge before the beginning of the device hierarchy implies a single ECAM configuration.

{
    "ahci/ahci0": {
        "device": 30,
        "function": 0,
        "image_path": "",
        "express_capability_device_type": 9
    },
    "smmuv3testengine/smmuv3testengine3": {
        "device": 15,
        "function": 0,
        "express_capability_device_type": 9
    },
    "rootport/rootport0": {
        "device_number": 2,
        "__downstream__": {
            "ahci/ahci1": {
            "device": 0,
            "function": 0,
            "image_path": "",
            "multi_function": true,
            },
        "ahci/ahci2": {
            "device": 0,
            "function": 1,
            "image_path": ""
            }
        }
    },
    "rootport/rootport1": {
        "device_number": 5,
        "__downstream__": {
            "ahci/ahci3": {
            "device": 0,
            "function": 0,
            "image_path": ""
            }
        }
    },
    "rootport/rootport2": {
        "device_number": 6,
        "__downstream__": {
            "switch/switch0": {
                "device_number": 0,
                "__downstream__10": {
                    "ahci/ahci4": {}
                },
                "__downstream__20": {
                    "ahci/ahci5": {}
                }
            }
        }
    }
}

Example 2: 2 ECAMs

{
    "rootbridge/rb0": {
        "ecam_start": 0x60000000,
        "ecam_end_incl": 0x67ffffff,
        "mem32_start": 0x70000000,
        "mem32_end_incl": 0x73ffffff,
        "mem64_start": 0x500000000,
        "mem64_end_incl": 0x67fffffff,
        "ecam_start_bus_number": 0x0,

        "__downstream__": {
            "ahci/ahci0": {
                "device": 30,
                "function": 0,
                "image_path": "",
                "express_capability_device_type": 9
            },
            "ahci/ahci1": {
                "device": 31,
                "function": 0,
                "express_capability_device_type": 9,
                "image_path": ""
            },
            "rcec/rcec0": {
                "device": 1,
                "msix_support": true,
                "bar0_log2_size": 16,
                "bar2_log2_size": 16,
                "bar4_log2_size": 12,
                "msix_table_bar": 2,
                "msix_pba_bar": 4,
                "msix_table_size": 2048,
                "rciep_associated_device_function_info": "240 248"
            }
        }
    },
    "rootbridge/rb1": {
        "ecam_start": 0x68000000,
        "ecam_end_incl": 0x6fffffff,
        "mem32_start": 0x74000000,
        "mem32_end_incl": 0x77ffffff,
        "mem64_start": 0x680000000,
        "mem64_end_incl": 0x7ffffffff,
        "ecam_start_bus_number": 0x1,

        "__downstream__": {
            "ahci/ahci0": {
                "device": 2,
                "function": 0,
                "image_path": "/path/to/image",
                "express_capability_device_type": 9
            }
        }
    }
}

Copyright (c) 2023-2025, Arm Limited and Contributors. All rights reserved.