Skip to content

Thunderbolt Bridge has no Loop Protection  #69

@GandalfTea

Description

@GandalfTea

The Thunderbolt Bridge driver on macOS does not have loop protection. It only functions as a flat L2 fabric and if a ring is naively routed all devices IPs end up on the same subnet resulting in a broadcast storm where MDNS, ARP and other packets are forwarded indefinitely in the loop, growing in size until the entire throughput is filled and all future requests are lost, meaning full loss of connectivity to both LAN and WLAN, DNS, ARP, high CPU utilization and temperature, etc. until one of the TBs is disconnected.

Below is the CPU under normal load but with a fully connected TB ring:

Image

A break can be seen whenever any of the TB cables are disconnected and the ring broken:

Image

On macOS 15.3.1 -- full network crash

On older version the storm escalates and results in having no connection to LAN or WAN. I only captured a smaller tcpdump of the bridge0 interface taken during the early stage of the storm. It shows about 80k packets/s and rising:

|------------------------------|
| Interval | Frames |   Bytes  |
|------------------------------|
|  0 <> 1  |  76249 | 30006253 |
|  1 <> 2  |  95122 | 31746752 |
|  2 <> 3  |  99798 | 32255080 |
|  3 <> 4  |  99558 | 32177093 |
|  4 <> 5  | 100161 | 32371754 |
Number of packets:   524 k
Capture duration:    6.451817 seconds
Data byte rate:      27 MBps
Average packet size: 335.34 bytes
Average packet rate: 81 kpackets/s

On macOS Beta 26.2

The storm stabilizes around 100k packets/s, probably due to a rate-limit update in the driver, resulting only in connections delays but not a complete loss:

Number of packets:   6,917 k
Capture duration:    69.722017 seconds
Data byte rate:      33 MBps
Average packet size: 338.75 bytes
Average packet rate: 99 kpackets/s
|------------------------------|
| Interval | Frames |   Bytes  |
|------------------------------|
|  0 <> 1  |      9 |     4493 |   <== Baseline
|  1 <> 2  |      1 |      213 |
|  2 <> 3  |      1 |      213 |
|  3 <> 4  |      2 |     2990 |
|  4 <> 5  |      6 |     3230 |
|  5 <> 6  |      0 |        0 |
|  6 <> 7  |      0 |        0 |
|  7 <> 8  |  34060 | 16080754 |   <== Last cable connected
|  8 <> 9  | 103319 | 45295802 |
|  9 <> 10 | 115833 | 48190858 |
| 10 <> 11 | 109716 | 44041404 |
| 11 <> 12 | 105955 | 42650075 |
| 12 <> 13 | 107182 | 43143676 |
| 13 <> 14 | 111384 | 44835336 |
| 14 <> 15 | 108720 | 43762165 |
| 15 <> 16 | 108989 | 43871459 |
| 16 <> 17 | 108004 | 43475207 |
| 17 <> 18 | 113107 | 44604846 |
| 18 <> 19 | 104005 | 38541778 |
| 19 <> 20 | 111194 | 40068920 |

Reproduction:

  • Connect 3+ Macs with TB4/5 in a loop (A -> B -> C -> A) but keep one cable disconnected
  • Allow for automatic IP assignment or manually assign all devices on same subnet (10.10.0.0/24)
  • Trace all packets with tcpdump -i bridge0 or do netstat -I bridge0 1 for full-interface bytes/s stats
  • Connect the last TB cable and see traffic increase

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingcriticalCritical issueperformancePerformance optimizations and resource efficiency

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions