Skip to content

Commit c5cb8b8

Browse files
jcps07jcps07
authored
Merge pull request #39 from cecille/pics
PICS documentation - moved from SDK
2 parents bf80b2c + 48298cb commit c5cb8b8

File tree

2 files changed

+206
-0
lines changed

2 files changed

+206
-0
lines changed

‎content/Certification/Guides/PICSandPIXIT.md

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
+++
2+
title = "PICS and PIXIT"
3+
4+
chapter = false
5+
weight = 12
6+
+++
7+
8+
## PICS
9+
10+
In many Standards Defining Organizations including the CSA, the concept of a
11+
“Protocol Implementation Conformance Statement” or “PICS” code is introduced to
12+
simplify description of protocol elements.
13+
14+
Each PICS code is a binary value that describes the presence or absence of a
15+
particular element or capability on a device. Each cluster has a defined PICS
16+
prefix string and defines a set of PICS codes to describe whether the cluster is
17+
present, and the presence of each feature, attribute, command, and event for
18+
both the server and client side.
19+
20+
For example if there was a cluster ANeatCluster with the PICS code ANC, two
21+
features (bit 0 and 1), two attributes (0x0000 and 0x0001), two accepted
22+
commands (0x00 and 0x01), a single command response (0x02) and one event (0x00),
23+
the following PICS codes would be defined
24+
25+
| CODE | Desc |
26+
| ------------- | -------------------------------------------------------------------------- |
27+
| ANC.S | Device implements the ANC cluster as a server |
28+
| ANC.S.F00 | Device implements ANC feature at bit 0 on the ANC server cluster |
29+
| ANC.S.F01 | Device implements ANC feature at bit 1 on the ANC server cluster |
30+
| ANC.S.A0000 | Device implements ANC attribute 0x0000 on the ANC server cluster |
31+
| ANC.S.A0001 | Device implements ANC attribute 0x0001 on the ANC server cluster |
32+
| ANC.S.C00.Rsp | Device accepts ANC command 0x00 on the ANC server cluster |
33+
| ANC.S.C01.Rsp | Device accepts ANC command 0x01 on the ANC server cluster |
34+
| ANC.S.C02.Tx | Device generates ANC command response 0x02 on the ANC server cluster |
35+
| ANC.S.E00 | Device generates ANC event 0x00 on the ANC server cluster |
36+
| ANC.C | Device supports an ANC client |
37+
| ANC.C.F00 | Device ANC client is capable of understanding the feature with bit 0 |
38+
| ANC.C.F01 | Device ANC client is capable of understanding the feature with bit 1 |
39+
| ANC.C.A0000 | Device ANC client is capable of reading or subscribing to attribute 0x0000 |
40+
| ANC.C.A0001 | Device ANC client is capable of reading or subscribing to attribute 0x0001 |
41+
| ANC.C.C00.Rsp | Device ANC client is capable of sending the command with id 0x00 |
42+
| ANC.C.C01.Rsp | Device ANC client is capable of sending the command with id 0x01 |
43+
| ANC.C.C02.Tx | Device ANC client understands the command response with id 0x02 |
44+
| ANC.C.E00 | Device ANC client understands the event with id 0x00 |
45+
46+
More information about the PICS code format can be found at
47+
[PICS Guidelines](https://github.com/CHIP-Specifications/chip-test-plans/blob/master/docs/PICS%20Guidelines.md). (requires access to the CSA specifications repo)
48+
49+
In addition to these standard cluster PICS, other PICS may be defined to
50+
describe capabilities that are not directly expressed via the data model. For
51+
example, whether the device responds to manual operations that affect the Matter
52+
data model.
53+
54+
Additionally, there are node-level PICS, which appear as a part of the MCORE
55+
PICS set. These PICS codes describe node level support such as the radio,
56+
whether the device is commissionable, whether the device comes with a QR code,
57+
etc.
58+
59+
PICS are used in testing in two ways. "Top level" PICS appear at the top of the
60+
test plan and indicate whether an entire test case should be run. For example, a
61+
test case for ANeatCluster would have a top-level PICS of ANC.S to indicate that
62+
the test would only be run if the ANC cluster is present on the endpoint. Test
63+
cases also use PICS to gate individual steps of the test which are not
64+
applicable if a certain element or capability is not implemented.
65+
66+
The entire collection of PICS codes for a specification release is provided as a
67+
set of PICS XML files. These files are loaded into the PICS tool, which is used
68+
to manually set all the PICS codes for a device. The PICS XML files and the PICS
69+
tool are distributed as part of the official specification release package and
70+
are available to members on
71+
[Causeway](https://groups.csa-iot.org/wg/members-all/document/folder/2269)
72+
(requires CSA member login).
73+
74+
## PIXITs
75+
76+
PIXIT stands for Protocol Implementation eXtra Information for Testing. A PIXIT
77+
value provides an implementation-defined condition or value that is required by
78+
the test.
79+
80+
PIXIT values are used to convey information that is required for testing, but
81+
not normally available to a client interacting with the device. For example, the
82+
key for a test event trigger on the device, or the network credentials for the
83+
test harness.
84+
85+
More information on PIXITs and the PIXIT format can be found in the
86+
[PICS Guidelines](https://github.com/CHIP-Specifications/chip-test-plans/blob/master/docs/PICS%20Guidelines.md).
87+
88+
## Creating PICS conformance statements for Matter devices
89+
90+
Because Matter devices may contain multiple, differing instances of the same
91+
cluster on multiple endpoints, it it not possible to unambiguously describe a
92+
Matter device with a single set of cluster-based PICS files. Instead, Matter
93+
devices use a full set of PICS XML files to describe each endpoint.
94+
95+
To create a conformance statement for a Matter device, for each endpoint, load
96+
the full set of PICS XML files into the
97+
[PICS tool](https://picstool.csa-iot.org/), and select each of the PICS elements
98+
present for the endpoint being described. Documentation on how to use the PICS
99+
tool can be found in the PICS tool readme in the tool menu.
100+
101+
Some of the full-node MCORE elements really only apply to the root node,
102+
while others apply across all endpoints. For example, the entire device is
103+
commissionable, but commissioning tests only need to be run against EP0, so
104+
MCORE PICS should be set on the EP0 PICS set. Things like MCORE.IDM apply to
105+
every endpoint.
106+
107+
108+
### Helper scripts
109+
110+
The official tooling for CSA certification is the PICS tool provided as a part
111+
of the release. PICS files need to pass validation on the PICS tool to be valid
112+
for certification.
113+
114+
However, in Matter, many of the PICS codes correspond directly to elements that
115+
are exposed directly on the device. For example, cluster presence is determined
116+
from the server list on an endpoint. Feature maps and attribute and command lists
117+
correspond directly to the PICS.S.Fxx, PICS.S.Axxxx and PICS.S.Cxx.Xx PICS
118+
codes. The Matter SDK provides a tool to pre-fill these values in the PICS XML
119+
files so they do not have to be individually, manually filled in the PICS tool.
120+
121+
- [PICS Generator](https://github.com/project-chip/connectedhomeip/tree/master/src/tools/PICS-generator)
122+
123+
PICS codes filled using this tool should still be validated by the PICS tool.
124+
Note that due to device limitations, the tool will NOT fill the following
125+
categories of PICS codes, and these will need to be filled manually in the PICS
126+
tool:
127+
128+
- Optional or conditional event PICS (PICS.S.Exx)
129+
- Client PICS
130+
- MCORE (base.xml) PICS
131+
- Manual or other non-element PICS
132+
- PICS describing whether optionally-writable attributes are writable
133+
- Any other non-element PICS
134+
135+
It is important to note that this script is NOT the official tool for PICS
136+
generation, just a helper to assist with this manual process. It is very
137+
important to go back and check that the values are as expected and to fill in
138+
the other PICS.
139+
140+
### Verifying PICS using the IDM-10.4 certification test
141+
142+
While not all PICS are verifiable on the device, we do have tests that verify
143+
the declared PICS against the device. This is one of the first tests that should
144+
be run at certification, as the PICS files are what determine which set of tests
145+
are required.
146+
147+
To run these tests locally, follow the instructions at
148+
[Running Python tests locally](./python.md#running-tests-locally). The PICS
149+
checker test is TC-IDM-10.4, implemented in
150+
[TC_pics_checker.py](https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_pics_checker.py).
151+
These tests run a single set of PICS XML files against an endpoint. The
152+
`--endpoint` and `--PICS` flags are therefore required.
153+
154+
Note that you can run tests locally against the PICS XMLs for an endpoint by
155+
supplying the name of the directory containing the set of PICS XML files for
156+
that endpoint.
157+
158+
159+
## Setting PIXITs for Matter devices
160+
161+
Matter tests do not currently have support to read PIXIT values from the PICS
162+
XML files. Instead, tests implement PIXITs as test-specific flags. When running
163+
locally, these are specified on the command line. When running in the test
164+
harness, these are specified in the test parameters section of the test
165+
configuration.
166+
167+
## PICS for test selection
168+
169+
The official source that the CSA certification team uses to determine if all the
170+
required tests have been run at certification is the submitted set of PICS XML
171+
files and the PICS tool.
172+
173+
To generate this set of tests for each endpoint, load all the filled PICS XML
174+
files for a single endpoint into the PICS tool and validate the PICS files. The
175+
PICS files should validate properly. This will also generate the list of test
176+
cases.
177+
178+
Note that the test harness will also guide test selection by pre-selecting the
179+
required tests based on the PICS file, but it is the responsibility of the testers
180+
and the ATL to ensure that all the required tests are run and the results are
181+
submitted.
182+
183+
[//]: # (## Creating PICS / PIXITs for new cluster and use in test plans)
184+
185+
[//]: # (Placeholder for more information coming in a subsequent PR. Needs to cover)
186+
187+
[//]: # (- formatting - link out to test plans doc)
188+
[//]: # (- requirement to have all the elements listed)
189+
[//]: # (- how to do conformance in a way the PICS tool can handle)
190+
[//]: # (note - special attention to choice conformance, otherwise conformance)
191+
[//]: # (- why you shouldn't use PICS values in tests other than at the top level)
192+
193+
[//]: # (## Using PICS in test scripts)
194+
195+
[//]: # (Cover top-level and step-wise pics in yaml and python and also where they should NOT be used.)
196+
197+
[//]: # (## CI PICS format)
198+
199+
[//]: # (CI PICS format, CI-only PICS and the gotchas there.)

‎content/Certification/Guides/_index.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
+++
2+
title = "Guides"
3+
chapter = false
4+
weight = 4
5+
+++
6+
7+
This section contains Guides related to General certification processes.

0 commit comments

Comments
 (0)