Skip to content

[scripts] created bootstrap script for ZAPZap scripts refinement#79

Open
markaj-nordic wants to merge 42 commits into
masterfrom
zap_scripts_refinement
Open

[scripts] created bootstrap script for ZAPZap scripts refinement#79
markaj-nordic wants to merge 42 commits into
masterfrom
zap_scripts_refinement

Conversation

@markaj-nordic
Copy link
Copy Markdown
Owner

Issue Being Resolved

Change overview

Extracted the ZAP tool bootstrap code from the run_zaptool.sh and populated it in the standalone scripts.

Testing

Tested on a pure Ubuntu 20.04 by cloning chip repo and running:

  • generate.py
  • convert.py
  • zap_regen_all.py
  • zap_convert_all.py

without running zaptool before. As a result all required nodejs packets are installed and the scripts run properly.

jmartinez-silabs and others added 26 commits September 29, 2022 10:04
* [doc] Document CommissioningStage enum.

* Update src/controller/CommissioningDelegate.h

Co-authored-by: C Freeman <cecille@google.com>

Co-authored-by: C Freeman <cecille@google.com>
* Fix memory exposure

* Apply PR comments
* Add the Software Version Info for TC-BINFO-2.1

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to save the information of the last connected AP

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to support "Manual controlled feature" for TC-OO-2.2

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Move mw320_sdk initialization to a function

* Add Identify_Timer_Handler() for TC-I-2.2

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add modules: - ConnectivityUtils.* - DeviceInfoProviderImpl.*

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add the implementation of read/write configuration key to flash

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to report information for TC_DGGEN-3.1

    - entwork interface:
		./chip-tool generaldiagnostics read network-interfaces 1 0
    - reboot count:
		./chip-tool generaldiagnostics read reboot-count 1 0

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add the missing functions for handling tc-wifidiag-3.1 Some of them are incompleted yet that hard-coded value are used

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Restyled by clang-format

* Restyled by gn

* Implement the functions for WiFiDiagnostics / Generic Diagnostics * wifi diagnostics commands: ./chip-tool wifinetworkdiagnostics read bssid 1 0 ./chip-tool wifinetworkdiagnostics read security-type 1 0 ./chip-tool wifinetworkdiagnostics read wi-fi-version 1 0 ./chip-tool wifinetworkdiagnostics read channel-number 1 0 ./chip-tool wifinetworkdiagnostics read rssi 1 0 ./chip-tool wifinetworkdiagnostics read beacon-lost-count 1 0 ./chip-tool wifinetworkdiagnostics read beacon-rx-count 1 0 ./chip-tool wifinetworkdiagnostics read packet-multicast-rx-count 1 0 ./chip-tool wifinetworkdiagnostics read packet-multicast-tx-count 1 0 ./chip-tool wifinetworkdiagnostics read packet-unicast-rx-count 1 0 ./chip-tool wifinetworkdiagnostics read packet-unicast-tx-count 1 0 ./chip-tool wifinetworkdiagnostics read current-max-rate 1 0 ./chip-tool wifinetworkdiagnostics read overrun-count 1 0

* Generic Diagnostics:
	./chip-tool generaldiagnostics read network-interfaces 1 0
	./chip-tool generaldiagnostics read reboot-count 1 0

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to support Identify_Time on multiple endpoints Verify: pass TC_I_2.2 for ep#0, ep#1

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add "reset watermark" handler

Verify: pass TC-DGSW-2.3

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to support Ethernet / WiFi network commissioning Test: pass TC-CNET-x.xx, Ethernet and WiFi cases

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* * Remove unused module: DeviceInfoProviderImpl.cpp/DeviceInfoProviderImpl.h * Change the setting to support the new tiny_crypt implementation

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to print the version string

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* * Add the Shutdown handler in PlatformMgr() * Add to register the shutdown command - Sent shutdown event - Shutdown the chip stack

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add back DeviceInfoProviderImpl.cpp / DeviceInfoProviderImpl.h to pass TC-LCFG-2.1 Update the version to: mw320-2.9.10-002

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Change to comparing SSID with characters, instead of ASCII code (previous implementation of Matter stack) ref test: TC-CNET-4.4, Scan with SSID: ./chip-tool networkcommissioning scan-networks 1 0 --Ssid hex:6E78705F6D6174746572 --Breadcrumb 2

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Update version number to: mw320-2.9.10-003

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Update README.md

* Correct the printed Rendzvous Information (QR-code, Manual code) from BLE (unsupported) => OnNetwork

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add the function to change the pincode of mw320: Usage: > pincode [pin_code] example: > pincode 20192022 [i] mflash_save_file success Done > config VendorId: 65521 (0xFFF1) ProductId: 32769 (0x8001) HardwareVersion: 0 (0x0) PinCode: 20192022 Discriminator: f01 Done

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Update version number to: mw320-2.9.10-004

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Limit the maximum supported fabric number == 5

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add a shell command to save the ssid / password of the default ap

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Put manual_control implementation into a compile option controlled

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to dump progressing message on mw320

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Update version number to: mw320-2.9.10-005

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to support reading the manufacture data from a partition.

    - Creat the project from gn: chip_with_factory_data=1
    example:
        gn gen out/debug --args='treat_warnings_as_errors=false mbedtls_repo="//third_party/connectedhomeip/third_party/nxp/libs/mbedtls" chip_crypto="" chip_with_factory_data=1'
        gn gen out/debug --args='treat_warnings_as_errors=false mbedtls_repo="//third_party/connectedhomeip/third_party/nxp/libs/mbedtls" chip_crypto="tinycrypt" chip_with_factory_data=1'

    pairing command:
        ./chip-tool pairing ethernet 1 14014 1000 [ip_of_dut] 5540 --paa-trust-store-path /home/ubuntu/certs/paa/

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add to support save/get BootReason Verify: step#5 of TC-DGGEN-2.2

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add 2 console commands: * wlan-stat: Show the current dut status > wlan-state * wlan-abort: Abort the current connecting and accept the new request > wlan-abort

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Restyled by whitespace

* Restyled by clang-format

* Restyled by gn

* Disable the chip_detail_logging

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Add a building option, wifi_conn_abort_support, to make it configurable

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>

* Restyled by gn

* update submodule

Signed-off-by: Chin-Ran Lo <chin-ran.lo@nxp.com>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: nxptest <68574485+nxptest@users.noreply.github.com>
…hip#22951)

* Report exchanged MTU size to core for BLE transaction
* Add mdns refresh after IP address update
* Migrate MatterPostAttributeChangeCallback() to current prototype
* Discriminator as gatts device name
* Get wifi mac address by filogic wifi opmode mode
* Update submodule genio-matter-bsp, lwip

Co-authored-by: Anthony Liu <anthony.liu@mediatek.com>
…hip#22946)

* Move android examples from src/android to examples/android

* Update docs

Co-authored-by: Andrei Litvin <andy314@gmail.com>
…hip#22963)

22932 shuts down the fabric table as part of shutting down the stack.
This entails interactions with the persistent storage layer
to update some keys. In Python however, the storage layer has already
been shutdown by the time we get to stack shutdown, causing a
use-after-free.

Flip the order in which we shutdown the storage adapter relative to
shutting down the rest of the stack.
…ect-chip#22877)

* Bluetooth refactor to use SLC generation

* Update Submodule to get generated files

* Fix conflict error

* move gatt_db files out of csa repo

* update sdk build gn

* Update Submodule

* Move fault logging to better location
…ready provisioned nodes (project-chip#22967)

* [ESP32]: Remove device name from the ble log as we do not add that in
the advertising packet

* Remove the sdkconfig.old

* [ESP32]: Platform, doc, example change to support ESP32C2

* [ESP32]: Do not initialize BLE if wifi/thread is already provisioned.

If ble is only used for commissioning then there is no use to initialize
the ble.
Also, renamed the config option for the purpose to make it more aligned.

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>
)

* remove groups cluster from lock endpoint on lock-app

* Restyled by whitespace

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>
…L layout (project-chip#22882)

* Initial checkin of a parser - has a main and can load XML files but does not parse them at all

* Starting a bit of a stack processing logic. This for now includes the cluster definition - name and code

* Restyle

* Add locator support

* Updated parser to handle more data (have attributes) and report tags that are unknown

* Refactored the context and paths a bit, have more concise reporting of features not handled by the api

* Ensure locator is initialized in the processing context

* More simplification

* Enum handling done

* Restyle

* More cleanup in terms of what was and was not handled

* some type safety

* Parsing structs, fix some acl parsing

* Slight refactor, preparing to have command roles as well

* Add support for command handling

* Support bitmap parsing in IDLs

* Full parsing support available - added global attribute support

* Handle FABRIC_SCOPED and this ensures every single XML in our repo CAN be parsed

* Do not make bitmap missing code fatal. Also note that events are not yet covered - need to do that

* Support parsing of events

* Skip a lot of not useful types in parsing, with some docs as to why

* Restyle

* More parsing fixes, support for multi filename entries

* Restyle

* Allow multi-parse

* Looking to split out logic for processing for zapxml

* Even more splitting of modules

* Restyle

* Move parsing into separate submodule location

* Restyle

* split out the handle base

* Restyle

* Added some comments in parsing.py

* Another small refactor, move the main handler in __init__.py

* Add comments on all handlers for their corresponding paths

* Doccomment updates

* Rename configuration to configurator to match XML naming

* more type safety

* Restyle

* Add support for fabric scoped and timed invoke attributes

* Restyle

* Add the zapxml files to the BUILD.gn package

* Support multiple cluster codes for bitmaps as well. IasZone has that

* Restyle

* Started adding tests for xml parsing

* Restyle

* Added a more complex test for unit testing xml parsing

* Skip meta data for unit tests- exact line or filename does not matter

* Restyle and cleanup some more empty data

* Ensure both hex and dec encode/decode works

* Bitmap tests

* added unit test for struct parsing from xml

* Added parsing CI step

* Add noprint support for xml parser test, added CI test

* Reorder xml validation

* Move the test to linux rather than repl

* Allow test_suites_linux to run under ACT

* Fix typo in act invocation comment

* use python3 for xml_parsing runner - CI wants this

* Use run_in_build_environment to get all packages available for python

* FIX todos: some added issues, some removed

* Undo unintended repo updates

* Add some doc comments

* More doc comments

* Restyle

* Add real ZAP XML test

* Restyle
…p#22959)

Function description didn't provide details on what happens if you pass
in 0.

Co-authored-by: Andrei Litvin <andy314@gmail.com>
* [K32W0] Add platform changes and features

* Add SVE2 fixes and other changes and features

* Add support for custom certificate policy.
* Certificate declaration: set certificate_type to 1
* Add new demo certificates
* Add demo certificates allowing testing with two different DUTs

* Add custom lighting app ZAP file - e.g.: removed generic and not needed clusters
  like the ones required for Ethernet/WiFi; removed the optional clusters for an
  On/Off Device Type;

* Add ksdk_mbedlts.c such that we redirect all the AES calls through SecLib.c

* Add TestEventTrigger Support

* Add functionality for ResetWatermarks()

* Add callback for PDM logs

* Fix NotifyUpdateApplied - message that is sent when booting for the first time
  with an OTA image was not being send due to a bug in the PDM read function:
  ReadConfigValue(...);
* Add delay when rebooting after OTA finishes
* Improve OTA mechanism

* Remove third parameter of ReadConfigValue() and use a TValue& parameter which can
  be used for extracting the value length

* Consume faster the RX buffer
* RX buffer should be processed with high priority because:
  * 15.4 packets are consumed on Thread task;
  * Thread task has a higher priority than the Matter one (3 > 2)

* Fix Low Power
  * fix 15.4 warm-up time: before this commit calibration time also
    included the time required for attaching to the Thread network
    This led to a large warm-up value that blocked some 15.4 operations
    Fix this by including in the warm-up time only the 15.4 radio
    initialization time
  * fix the warm-up initial value to avoid any conflicts
  * fix possible overflow in reading timestamp values
  * Make sure that all the logs are disabled when an application is compiled with low-power
	support; logging can interference with LP timings and break functionality;

* Fix time support for Matter
  * using FreeRTOS timers (based on FreeRTOS tickets) is inaccurate and leads to time drifting
  * use OT Timers API  (Wtimer-based) for getting us/ms timestamps

* Fix ExtractPubkeyFromX509Cert in CHIPCryptoPALTinyCrypt.cpp

* Fix BLE advertising interval + issue warning if the peer doesn't support 2Mbps

* Fix Diagnostic Data Provider

* Fix OnOff attribute persistence over hw reset

* Fix compilation issue for lock-app

* Disable by default on lighting-app BLE Set Phy Req to 2M to be extra cautious during interops

* Update readme files

Signed-off-by: Doru Gucea <doru-cristian.gucea@nxp.com>
Signed-off-by: Marius Tache <marius.tache@nxp.com>
Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Fix gn includes, spelling, wordlist

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Update ot-nxp submodule

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Fix gn includes, exclude certs from restyle, fix messaging readme

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Restyled by whitespace

* Restyled by clang-format

* Restyled by gn

* Restyled by prettier-markdown

* Restyled by shellharden

* Restyled by shfmt

* Restyled by autopep8

* Fix low-power build

Signed-off-by: Marius Tache <marius.tache@nxp.com>
Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Restyled by clang-format

* Fix SecLib compile issue

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Restyled by clang-format

* Update worflows K32W0 examples yaml

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Update docker version, expected gn gen

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

* Hotfix nxp shell build

Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>

Signed-off-by: Doru Gucea <doru-cristian.gucea@nxp.com>
Signed-off-by: Marius Tache <marius.tache@nxp.com>
Signed-off-by: Andrei Menzopol <andrei.menzopol@nxp.com>
Co-authored-by: Restyled.io <commits@restyled.io>
…project-chip#22971)

* [Telink] use zephyr in include path

* [Telink] use telink zephyr crypto

* [Telink] rename IEEE802154 functions and isr table offset fix

* [Telink] restyle

* [Telink]: Update every image to 0.6.01

* [Telink]: Update every image to 0.6.02
* Update docs to point the SMG docs

* Address comment -remove extra )

* Update path to generic Releases

* restyle

* Remove SMG acronym for the pyspelling ci to pass
…-chip#22749)

* Make it possible to use TestEventTrigger command of GeneralDiagnostics
cluster to trigger OTA query on demand.
* Use OTATestEventTriggerDelegate in CYW30739 example.
* Fixed TLV decoder issue

Fixed issue of the TLV decoder:

When giving tlv_bytes = b"\0256\001\0255\001&\000\031\346x\2077\001$\002\001$\003\006$\004\000\030(\002\030\030\030\030", the following errors occur:

Traceback (most recent call last):
  File "/home/chingtzuchen/exp/tlv2.py", line 743, in <module>
    out = reader.get()
  File "/home/chingtzuchen/exp/tlv2.py", line 459, in get
    self._get(self._tlv, self._decodings, out)
  File "/home/chingtzuchen/exp/tlv2.py", line 674, in _get
    self._decodeVal(tlv, decoding)
  File "/home/chingtzuchen/exp/tlv2.py", line 574, in _decodeVal
    self._get(tlv, decoding["Structure"], decoding["value"])
  File "/home/chingtzuchen/exp/tlv2.py", line 674, in _get
    self._decodeVal(tlv, decoding)
  File "/home/chingtzuchen/exp/tlv2.py", line 578, in _decodeVal
    self._get(tlv, decoding["Array"], decoding["value"])
  File "/home/chingtzuchen/exp/tlv2.py", line 674, in _get
    self._decodeVal(tlv, decoding)
  File "/home/chingtzuchen/exp/tlv2.py", line 574, in _decodeVal
    self._get(tlv, decoding["Structure"], decoding["value"])
  File "/home/chingtzuchen/exp/tlv2.py", line 674, in _get
    self._decodeVal(tlv, decoding)
  File "/home/chingtzuchen/exp/tlv2.py", line 574, in _decodeVal
    self._get(tlv, decoding["Structure"], decoding["value"])
  File "/home/chingtzuchen/exp/tlv2.py", line 674, in _get
    self._decodeVal(tlv, decoding)
  File "/home/chingtzuchen/exp/tlv2.py", line 582, in _decodeVal
    self._get(tlv, decoding["Path"], decoding["value"])
  File "/home/chingtzuchen/exp/tlv2.py", line 685, in _get
    out[decoding["tag"]] = decoding["value"]
IndexError: list assignment index out of range

Added condition and made it more clean to fix the issue, after this fix the bytes can be decoded:

{'Any': {1: [{1: {'Any': 2272847385, 1: [1, 6, 0], 2: False}}]}}

* Update __init__.py

* Add unit test case

Add unit test case for structure TLV bytes input

Test result:

```
$ python test_tlv.py 
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK
```

* Update test_tlv.py

Update style
Network scan is performed in the background by wpa_supplicant. When
we are trying to get BSS properties, the BSS may be already gone. This
patch prevents passing NULL to g_variant_get_fixed_array().
…ip#22841)

* [Linux] Mark GLib source as removed when exiting callback

Fixes project-chip#22840

* Add TODO for investigating GLib watch tag ID some more
achaulk-goog and others added 5 commits October 3, 2022 17:39
* Changes target board for silabs CI

* fix CI

* fix CI
…#22990)

* EFR32: SDK support code cleanup CSA supporting changes

* EFR2 : Movement of Wifi Implementation on CSA repo

* EFR32 : Restyler changes

* EFR32 : Moved rs911x and WF200 folders to example/platform/efr32

* EFR32 : Updated license attribution/blurb in newly added files

* Review comment addressed and matter support pointer changed

* EFR32 : Thread build error solved

* EFR32 : Changed the build for wifi

* Restyler Changes
Extracted bootstrap code from run_zaptool.sh and reuse it
in other Python scripts. This allows to run ZAP convert/generate
tools out of the box without necessity to run zaptool beforehand.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Platform] ZAP tool standalone scripts cannot be used out of the box