From 852c4ed21029f6b9e92056d9e6c2235dd5f977cb Mon Sep 17 00:00:00 2001 From: Santosh Bhavani Date: Tue, 10 Mar 2026 16:06:17 -0700 Subject: [PATCH] docs: add MoE technical report and Falcon-H1 blog to Latest News --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index d719da4a4f6..639897a8be4 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,8 @@ For NGC container setup and all installation options, see the **[Installation Gu # Latest News +- **[2026/03]** **[Scalable Training of Mixture-of-Experts Models with Megatron Core](https://arxiv.org/abs/2603.07685)** - Technical report on scaling MoE training with integrated optimizations for memory, communication, and computation. +- **[2026/03]** **[Implementing Falcon-H1 Hybrid Architecture in Megatron Core](https://developer.nvidia.com/blog/implementing-falcon-h1-hybrid-architecture-in-nvidia-megatron-core/)** - Technology Innovation Institute (TII) contributes Falcon-H1 hybrid transformer-Mamba architecture and BitNet ternary quantization support to Megatron Core. - **[2026/01]** **[Dynamic Context Parallelism](https://developer.nvidia.com/blog/speeding-up-variable-length-training-with-dynamic-context-parallelism-and-nvidia-megatron-core/)** - Up to 1.48x speedup for variable-length sequence training with adaptive CP sizing. - **[2025/12]** **Megatron Core development has moved to GitHub!** All development and CI now happens in the open. We welcome community contributions. - **[2025/10]** **[Megatron Dev Branch](https://github.com/NVIDIA/Megatron-LM/tree/dev)** - early access branch with experimental features.