Loss curves

## Dependency pytorch verison: `torch-2.11.0.dev20251226+cu126` torchao version: `torchao-0.16.0.dev20251226+cu126` ## What's Changed ### Features * Use new DeviceMesh unflatten to rewrite parallel_dims by @fegin in https://github.com/pytorch/torchtitan/pull/1660 * Re:Run Torchtitan ROCm workflow on cron schedule & push to Main branch only by @akashveramd in https://github.com/pytorch/torchtitan/pull/2018 * adding variable length attention to llama3 8b by @liangel-02 in https://github.com/pytorch/torchtitan/pull/2000 * [Local Tensor] Replace dry_run.py with fake mode implementation by @fegin in https://github.com/pytorch/torchtitan/pull/2057 ### Model * Enable PP and EP overlap for MoE by @H-Huang in https://github.com/pytorch/torchtitan/pull/1721 * Integrate DeepEP to torchtitan by @elfiegg in https://github.com/pytorch/torchtitan/pull/2107 * [MoE] Add node limited routing support by @shuhuayu in https://github.com/pytorch/torchtitan/pull/2111 * Add Context Parallelism to Flux model training by @limou102 in https://github.com/pytorch/torchtitan/pull/1851 * gpt-oss model enablement by @wwwjn in https://github.com/pytorch/torchtitan/pull/1754 * [GPT-OSS] Add HF state dict adapter to support loading from HF checkpoints by @shuhuayu in https://github.com/pytorch/torchtitan/pull/2021 ### Bug Fix * [FLOPs] Fix attention FLOPs estimate by @shuhuayu in https://github.com/pytorch/torchtitan/pull/1923 * Fix apply_compile called multiple times in PP initialization by @xmfan in https://github.com/pytorch/torchtitan/pull/2135 * Fix qwen3 attention scaling calculation by @wwwjn in https://github.com/pytorch/torchtitan/pull/2173 ### Experiments * **Exploring toolkit-style use of the compiler stack** @SherlockNoMad @yiming0416 : [Compiler Toolkit] JointGraph-based Training Prototype for llama3 by @SherlockNoMad in https://github.com/pytorch/torchtitan/pull/1794 * **Bit-wise identity RL between torchtitan Trainer and vLLM sampler**: Add deterministic RL training experiment with vLLM by @bwasti in https://github.com/pytorch/torchtitan/pull/1975 * **Train models from`transformers` with torchtitan**: 3outeille/transformers backend (Dense model only) by @3outeille in https://github.com/pytorch/torchtitan/pull/2048 * **Auto Parallel Examples** @wconstab @xmfan: Autoparallel as an experiment in main by @xmfan in https://github.com/pytorch/torchtitan/pull/2054 * **Unified Model definition in RL loop** @wwwjn @acisseJZhong @zhxchen17 : Run vLLM inference using torchtitan model definition (single GPU) by @wwwjn in https://github.com/pytorch/torchtitan/pull/2119 **Full Changelog**: https://github.com/pytorch/torchtitan/compare/v0.2.0...v0.2.1

# Dependency pytorch verison: `torch-2.10.0.dev20251017+cu126` torchao version: `torchao-0.15.0.dev20251015+cu126` **Full Changelog**: https://github.com/pytorch/torchtitan/compare/v0.1.0...v0.2.0

This is the first pre-release of torchtitan, following the release practice outlined in https://github.com/pytorch/torchtitan/issues/688. torch version: torch-2.8.0.dev20250617+cu126 torchao version: `torchao-0.12.0.dev20250617+cu126`