Project 32: Open-Source 10GbE with UART Debug

Part of FPGA Trading Systems Portfolio

This project is part of a complete end-to-end trading system:

• Main Repository: fpga-trading-systems
• Project Number: 32 of 35
• Category: FPGA Core
• Dependencies: None - project will not be updated beyond this point

---

Overview

This project implements a 10 Gigabit Ethernet (10GbE) interface using the Kintex-7 GTX transceivers on the ALINX AX7325B board. It uses the open-source verilog-ethernet library for the MAC/PHY layers.

Hardware

• Board: ALINX AX7325B
• FPGA: Kintex-7 XC7K325T-2FFG900I
• GTX: Quad 117, Lane 0 (X0Y8)
• Reference Clock: 156.25 MHz on pins G7/G8
• SFP+ TX/RX: Pins K1/K2 (TX), K5/K6 (RX)

Current Status

Working

• GTX QPLL locks successfully
• TX and RX reset done signals assert
• GTX produces TXOUTCLK (322 MHz for 32-bit width)
• Clock generation (MMCM divides to 161 MHz for 64-bit interface)

Known Issues

• Byte Sync Issues: UDP packets not properly synchronized
• Tested with both 32-bit and 64-bit GTX data widths - neither achieved full UDP lock

Architecture

+----------------+     +------------------+     +-------------+
| verilog-       | --> | gtx_10gbase_r    | --> | GTX         |
| ethernet       |     | _wrapper         |     | Transceiver |
| (eth_phy_10g)  | <-- | (32->64 gearbox) | <-- | (X0Y8)      |
+----------------+     +------------------+     +-------------+
       ^                      |
       |              +-------+-------+
       |              | 322MHz->161MHz |
       |              | (MMCM /2)      |
       +------------- +---------------+

GTX IP Configuration

The GTX IP is generated using Vivado's 7 Series FPGAs Transceivers Wizard:

• Location: /work/projects/fpga-trading-systems/gtxip/gtx_10gbase_r/
• Protocol: 10GBASE-R (64B/66B encoding)
• Line Rate: 10.3125 Gbps
• Reference Clock: Q2_CLK1 (GTREFCLK1) at 156.25 MHz
• Data Width: Currently 32-bit (testing 64-bit)

Key Files

• gtx_10gbase_r_support.vhd - Top-level wrapper with clock/reset management
• gtx_10gbase_r_common.v - QPLL configuration
• gtx_10gbase_r_gt.v - GTXE2_CHANNEL instantiation

Reference Clock Note

The IP uses Q2_CLK1_GTREFCLK naming (GTREFCLK1), but physically connects to pins G7/G8 which are labeled MGTREFCLK0P/N_117 in the package. Despite this naming discrepancy, the QPLL locks successfully.

Build Commands

cd /work/projects/fpga-trading-systems/32-10gbe-open

# Create Vivado project
/tools/Xilinx/MLE/2025.2/Vivado/bin/vivado -mode batch -source scripts/create_project.tcl

# Build bitstream
/tools/Xilinx/MLE/2025.2/Vivado/bin/vivado -mode batch -source scripts/build.tcl

# Program FPGA
/tools/Xilinx/MLE/2025.2/Vivado/bin/vivado -mode batch -source scripts/program.tcl

LED Mapping

LED	Signal	Description
LED0	pll_locked	QPLL lock status
LED1	link_up	Ethernet link status
LED2	activity	TX/RX activity blink
LED3	heartbeat	System clock running

Debug

UART Output

• 115200 baud, 8N1
• Outputs TX/RX status, packet counts, error codes

ILA Integration

The build includes optional ILA cores for real-time signal inspection:

• GTX RX data path
• 64B/66B block alignment
• Gearbox synchronization

Troubleshooting

QPLL Not Locking

1. Check reference clock present (156.25 MHz on G7/G8)
2. Verify QPLLREFCLKSEL matches actual clock input
3. Check SFP+ module installed and enabled (TX_DISABLE = low)

UDP Not Working

1. Check IP/MAC address configuration
2. Verify 64B/66B block alignment (sync headers)
3. Check gearbox phase alignment between clock domains
4. Consider testing with loopback first

Test Setup Hardware

Most developers will not have 10GbE networking at home (2.5GbE is common at best). This project was verified using a dedicated 10GbE fiber-optic test setup:

┌──────────────┐         ┌─────────────────────┐         ┌──────────────┐
│ PC           │  RJ45   │ 10GbE Managed       │  SFP+   │ AX7325B      │
│ (AQC107 NIC)│◄───────►│ Switch (Binardat)   │◄───────►│ FPGA Board   │
│ 10G RJ45    │  10Gb   │ 4xRJ45 + 4xSFP+    │  Fiber  │ (SFP+ Cage)  │
└──────────────┘         └─────────────────────┘         └──────────────┘
                                                    │
                                              OM3 LC-LC Fiber
                                              + 10G SFP+ Modules

Hardware used:

Component	Product	Specs
SFP+ Modules	10G SFP+ Fiber Transceiver	SR MM850nm, 300m range, Duplex LC
Fiber Cable	Tunghey OM3 LC to LC Patch Cable	Multimode Duplex 50/125um, 15M, LS-ZH
10GbE Switch	Binardat 8-Port 10G Managed Switch	4x10G RJ45 + 4x10G SFP+, 160Gbps, L3
PC NIC	Binardat 10G PCIe Network Adapter	Aquantia AQC107 chip, RJ45, PXE support

Important notes:

• DAC (Direct Attach Copper) cables did not work with the AX7325B SFP+ cage -- fiber optics required
• The switch bridges 10G RJ45 (PC side) to 10G SFP+ (FPGA side)
• SFP+ modules must be inserted into both the switch SFP+ port and the FPGA board SFP+ cage
• PC sends test packets via raw sockets or packet generator at 10Gbps line rate

Related Projects

• Project 31: Xilinx vendor 10G MAC IP (works reliably)
• Project 33: Custom 10GBASE-R PHY in VHDL (works reliably, 10Gbe full duplex)

References

• UG476: 7 Series Transceivers User Guide
• verilog-ethernet Library
• IEEE 802.3 10GBASE-R Specification

---

Status: Abandoned Created: January 2026 Author: Adilson Dias

Target Board: ALINX AX7325B (Kintex-7 XC7K325T-2FFG900I)