[VSC-1539] Fix scenario for partial encryption

[radurentea]

Description

Issue:
When using the --encrypted-files flag, the flash command generates duplicate arguments, causing the bootloader at address 0x0 to be written twice. This duplication can corrupt the encrypted bootloader during the flashing process.

Root Cause:
The current implementation appends encrypted files after the --encrypt-files flag, then appends all files (both encrypted and unencrypted) again, resulting in duplicated entries for encrypted files.

Fixes #1366

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

Steps to test this pull request

Note: I have manually modified the flasher_args.json file that was generated during the project build. Under normal circumstances, the ESP-IDF SDK should be configured to automatically generate a similar flasher_args.json with mixed encryption settings, but I'm unsure which specific SDK configurations are required to reproduce this scenario properly. The current setup has some files marked as "encrypted": true and others as "encrypted": false to simulate the user's environment. These manual changes will be overwritten if the project is rebuilt without the proper SDK configuration.

Prerequisites

I've used esp32-h2 devkitm-1 board with flash encryption already enabled in development mode, but any board should do.
ESP-IDF "flash_encryption" example project

Test Procedure

1. Setup the test environment:

   • Use the ESP-IDF flash_encryption example
   • Build the project to generate initial flasher_args.json
   • Ensure your board has flash encryption enabled in development mode

2. Create test configuration:

   • Replace the generated flasher_args.json with the following test configuration:

   {
       "write_flash_args": [
           "--flash_mode", "dio",
           "--flash_size", "4MB",
           "--flash_freq", "48m"
       ],
       "flash_settings": {
           "flash_mode": "dio",
           "flash_size": "4MB",
           "flash_freq": "48m"
       },
       "flash_files": {
           "0x0": "bootloader/bootloader.bin",
           "0x20000": "flash_encryption.bin",
           "0xd000": "partition_table/partition-table.bin",
           "0x3b2000": "storage.bin",
           "0x16000": "ota_data_initial.bin"
       },
       "bootloader": {
           "offset": "0x0",
           "file": "bootloader/bootloader.bin",
           "encrypted": "true"
       },
       "app": {
           "offset": "0x20000",
           "file": "flash_encryption.bin",
           "encrypted": "true"
       },
       "partition-table": {
           "offset": "0xd000",
           "file": "partition_table/partition-table.bin",
           "encrypted": "true"
       },
       "storage": {
           "offset": "0x3b2000",
           "file": "storage.bin",
           "encrypted": "false"
       },
       "ota_data": {
           "offset": "0x16000",
           "file": "ota_data_initial.bin",
           "encrypted": "false"
       },
       "extra_esptool_args": {
           "after": "no_reset",
           "before": "default_reset",
           "stub": true,
           "chip": "esp32h2"
       }
   }

3. Create placeholder files:

   • Create empty storage.bin and ota_data_initial.bin files in the build folder

4. Flash the device:

   • Use the "Flash Device" button in the VS Code status bar
   • Verify the flashing completes successfully

Expected Results

✅ With this fix:

esptool.py -p COM4 -b 460800 --before default_reset --after no_reset --chip esp32h2 write_flash --flash_mode dio --flash_freq 48m --flash_size 4MB 0x3b2000 storage.bin 0x16000 ota_data_initial.bin --encrypt-files 0x0 bootloader/bootloader.bin 0x20000 flash_encryption.bin 0xd000 partition_table/partition-table.bin

❌ Without this fix (master branch):

esptool.py -p COM4 -b 460800 --before default_reset --after no_reset --chip esp32h2 write_flash --flash_mode dio --flash_freq 48m --flash_size 4MB --encrypt-files 0x0 bootloader/bootloader.bin 0x20000 flash_encryption.bin 0xd000 partition_table/partition-table.bin 0x0 bootloader/bootloader.bin 0x20000 flash_encryption.bin 0xd000 partition_table/partition-table.bin 0x3b2000 storage.bin 0x16000 ota_data_initial.bin

Notice the duplicate entries for encrypted files in the broken version.

How has this been tested?

As described above

Test Configuration:

• ESP-IDF Version: 5.4
• OS (Windows,Linux and macOS): Windows

Checklist

• PR Self Reviewed
• Applied Code formatting
• Added Documentation
• Added Unit Test
• Verified on all platforms - Windows,Linux and macOS

[github-actions]

Download the artifacts for this pull request:
You can test these changes by installing this VSIX by click menu View -> Command Palette..., type Install from VSIX and then select downloaded esp-idf-extension.vsix file to install the extension.

• esp-idf-extension.vsix.zip
  Espressif documentation preview

[github-actions]

Pull request has been marked as stale since there are no activities, and this will be closed in 5 days if there are no further activities

[github-actions]

Pull request has been marked as stale since there are no activities, and this will be closed in 5 days if there are no further activities

[brianignacio5]

LGTM

[Fabricio-ESP]

Confirmed partially encrypted files are now handled properly when generating flash command for esptool.