Set multicast TTL on UDP sockets for shred broadcast and retransmit

[armcconnell]

Problem

The kernel defaults the IP TTL on multicast sockets to 1 which prevents multicast packets from being routed beyond the first hop router. For routed multicast the TTL should be increased.

Summary of Changes

Hardcode a multicast TTL of 64 and set it on UDP sockets.

Proof of testing

Unit tests

andrew@host:~/jito-solana$ cargo test --package solana-turbine test_multicast_ttl -- --nocapture
   Compiling solana-turbine v4.0.0-alpha.0 (/home/andrew/jito-solana/turbine)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 9.07s
     Running unittests src/lib.rs (target/debug/deps/solana_turbine-205ae9db9d2946b5)

running 2 tests
test retransmit_stage::tests::test_multicast_ttl_set_on_retransmit_sockets ... ok
test broadcast_stage::test::test_multicast_ttl_set_on_broadcast_sockets ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 33 filtered out; finished in 0.00s

Compiled from HEAD

sol@ams-tn-bm1:~$ /tmp/agave-validator --version
agave-validator 4.0.0 (src:8ba448e0; feat:1718981463, client:JitoLabs)

Running in a private test cluster

sol@host:~$ ps auxw | grep agave
sol      51182  759  1.1 10286860 3098000 pts/2 Sl+ 17:28  13:21 /tmp/agave-validator --identity /home/sol/validator-keypair.json --vote-account /home/sol/vote-account-keypair.json --accounts /home/sol/accounts --ledger /home/sol/ledger --allow-private-addr --block-production-method central-scheduler --dynamic-port-range 8002-8027 --entrypoint 67.213.120.9:8001 --expected-genesis-hash C97Q7ygjHJZU8AonPcgjYiBHhkfFXPVmTEH4bEwzTcX2 --tip-payment-program-pubkey GJHtFqM9agxPmkeKjHny6qiRKrXZALvvFGiKf11QE7hy --tip-distribution-program-pubkey DzvGET57TAgEDxvm3ERUM4GNcsAJdqjDLCne9sdfY4wf --merkle-root-upload-authority 7T4inmPmtNBX3MhLwJ9hFsSMnGJYYkKioVABSNTWVRuS --commission-bps 800 --block-engine-url https://127.0.0.1/jito --shred-receiver-address 127.0.0.1:1002 --shred-receiver-address 233.84.178.8:1002 --full-rpc-api --full-snapshot-interval-slots 1000 --gossip-port 8001 --known-validator 2yCbbFnG1z8iJotPb1YsjG6gUKGgu3UW5oeqDXC32wUq --known-validator 5njVpT6ek2Dgf2GLApAfJFiSnREDvrTafvokECByURUN --known-validator 7gBgs3dgyGSiNyp2wMNr5JbK1nFFGAV1z6bzQGezJ44r --limit-ledger-size 50000000 --no-wait-for-vote-to-start-leader --rpc-bind-address 0.0.0.0 --rpc-faucet-address 67.213.120.9:8899 --rpc-port 8899 --use-snapshot-archives-at-startup when-newest --wal-recovery-mode skip_any_corrupted_record

Confirm TTL is now 64.

sol@host:~$ sudo tcpdump -c5 -vvv -ni doublezero1 port 1002
tcpdump: listening on doublezero1, link-type LINUX_SLL (Linux cooked v1), snapshot length 262144 bytes
17:30:10.803025 IP (tos 0x0, ttl 64, id 63863, offset 0, flags [DF], proto UDP (17), length 1231)
    148.51.120.3.8015 > 233.84.178.8.1002: [bad udp cksum 0xac60 -> 0x1594!] UDP, length 1203
17:30:10.803043 IP (tos 0x0, ttl 64, id 63864, offset 0, flags [DF], proto UDP (17), length 1231)
    148.51.120.3.8015 > 233.84.178.8.1002: [bad udp cksum 0xac60 -> 0xc066!] UDP, length 1203
17:30:10.803052 IP (tos 0x0, ttl 64, id 63865, offset 0, flags [DF], proto UDP (17), length 1231)
    148.51.120.3.8015 > 233.84.178.8.1002: [bad udp cksum 0xac60 -> 0xbc5a!] UDP, length 1203
17:30:10.803060 IP (tos 0x0, ttl 64, id 63866, offset 0, flags [DF], proto UDP (17), length 1231)
    148.51.120.3.8015 > 233.84.178.8.1002: [bad udp cksum 0xac60 -> 0x613d!] UDP, length 1203
17:30:10.803068 IP (tos 0x0, ttl 64, id 63867, offset 0, flags [DF], proto UDP (17), length 1231)
    148.51.120.3.8015 > 233.84.178.8.1002: [bad udp cksum 0xac60 -> 0x0604!] UDP, length 1203
5 packets captured
246 packets received by filter
0 packets dropped by kernel