## What's Changed * edits to docs guide by @lrobledo in https://github.com/oumi-ai/oumi/pull/2105 * Fix issue with synthesis rounding errors by @jgreer013 in https://github.com/oumi-ai/oumi/pull/2104 * Add conversation and schema for outputs by @ryan-arman in https://github.com/oumi-ai/oumi/pull/2098 * Drop support for python 3.9 by @oelachqar in https://github.com/oumi-ai/oumi/pull/2107 * Deprecate alpaca_eval integration by @oelachqar in https://github.com/oumi-ai/oumi/pull/2108 * [deps] upgrade pytorch 2.9 & python 3.14 by @oelachqar in https://github.com/oumi-ai/oumi/pull/2109 * [perf] Add lazy imports in the cli by @oelachqar in https://github.com/oumi-ai/oumi/pull/2110 * Dec news updates by @stefanwebb in https://github.com/oumi-ai/oumi/pull/2111 * Bump actions/cache from 4 to 5 in the all-actions group by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2112 * Update peft requirement from <0.18,>=0.17 to >=0.17,<0.19 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2114 * Update uvicorn requirement from <0.36.0 to <0.41.0 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2115 * Update bitsandbytes requirement from <0.49,>=0.47 to >=0.47,<0.50 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2116 * Update click requirement from <8.3.0 to <8.4.0 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2117 * [bug] Pin typer version causing build docs failures by @oelachqar in https://github.com/oumi-ai/oumi/pull/2118 * [tiny] update deprecated torch_dtype by @oelachqar in https://github.com/oumi-ai/oumi/pull/2123 * add skypilot k8s dependency by @min-oumi in https://github.com/oumi-ai/oumi/pull/2124 * Add ty config by @oelachqar in https://github.com/oumi-ai/oumi/pull/2120 * [logs] cleanup errant print statements by @oelachqar in https://github.com/oumi-ai/oumi/pull/2121 * deprecate protobuf conversation definitions by @oelachqar in https://github.com/oumi-ai/oumi/pull/2127 * Update cli: list aliases, auto-complete, help, common args by @oelachqar in https://github.com/oumi-ai/oumi/pull/2122 * [docs] add analyze, tune, and quantize CLI reference sections by @oelachqar in https://github.com/oumi-ai/oumi/pull/2126 * [tiny] fix typo by @oelachqar in https://github.com/oumi-ai/oumi/pull/2128 * [tiny] minor judge command UX improvements by @oelachqar in https://github.com/oumi-ai/oumi/pull/2129 * [docs] update CLI command tables with complete command list by @oelachqar in https://github.com/oumi-ai/oumi/pull/2130 * [docs] fix inference recipe paths to match directory structure by @oelachqar in https://github.com/oumi-ai/oumi/pull/2125 * [docs] misc updates & cleanup by @oelachqar in https://github.com/oumi-ai/oumi/pull/2131 * [cli] minor cli updates by @oelachqar in https://github.com/oumi-ai/oumi/pull/2132 * [docs] Add custom theme by @oelachqar in https://github.com/oumi-ai/oumi/pull/2133 * [docs] add docs build step by @oelachqar in https://github.com/oumi-ai/oumi/pull/2134 * Update pillow requirement from <11.4,>=11.3 to >=11.3,<12.1 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2136 * Update typer requirement from <0.20.1 to <0.21.1 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2135 * Rule Based Judge for Deterministic Evals by @aniruddh-alt in https://github.com/oumi-ai/oumi/pull/2119 * Update torchao requirement from <0.15,>=0.12 to >=0.12,<0.16 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2137 * [train] Add callback to log metrics to disk by @oelachqar in https://github.com/oumi-ai/oumi/pull/2140 * feat: Add loading spinner during inference (fixes #1519) by @RajdeepKushwaha5 in https://github.com/oumi-ai/oumi/pull/2085 * Add support for loading pre-trained custom models by @ritankarsaha in https://github.com/oumi-ai/oumi/pull/2044 * [docker] add useful utils, enable arm by @oelachqar in https://github.com/oumi-ai/oumi/pull/2141 * Add version and system info utilities by @oelachqar in https://github.com/oumi-ai/oumi/pull/2142 * [datasets] add RaR datasets and base rubric dataset classes by @oelachqar in https://github.com/oumi-ai/oumi/pull/2144 * Add posthog telemetry manager by @oelachqar in https://github.com/oumi-ai/oumi/pull/2145 * [ops] disable telemetry during CI, update variables by @oelachqar in https://github.com/oumi-ai/oumi/pull/2146 * [train] add reward_function_kwargs support for per-reward function configuration by @oelachqar in https://github.com/oumi-ai/oumi/pull/2143 * Update pillow requirement from <12.1,>=11.3 to >=11.3,<12.2 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2147 * Add support for XLSX and DOCX formats for synthesis and datasets by @jgreer013 in https://github.com/oumi-ai/oumi/pull/2148 * [docker] Add tmux to docker image by @shanghongsim in https://github.com/oumi-ai/oumi/pull/2149 * Add support in synthesis for few-shot sampling from sources by @jgreer013 in https://github.com/oumi-ai/oumi/pull/2151 * Update typer requirement from <0.21.1 to <0.21.2 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2153 * [docs] cleanup css style for python reference page, remove reading progress indicator by @oelachqar in https://github.com/oumi-ai/oumi/pull/2154 * [ops] Add install.sh script by @oelachqar in https://github.com/oumi-ai/oumi/pull/2155 * Fix generic print output in the Oumi Tour notebook by @brian-nguyen in https://github.com/oumi-ai/oumi/pull/2157 * Update gemma3-4b-it SFT training config after offline tuning by @lefft in https://github.com/oumi-ai/oumi/pull/2156 * [infer] Add fireworks inference engine by @oelachqar in https://github.com/oumi-ai/oumi/pull/2158 * Update pycares requirement from <5.0.0 to <6.0.0 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2139 * [configs] add gemma-2-it chat template and example config by @oelachqar in https://github.com/oumi-ai/oumi/pull/2159 * [tiny] tweaks to main cli by @oelachqar in https://github.com/oumi-ai/oumi/pull/2160 * [tiny] explicitly set wandb as 3rd party to avoid isort issues by @oelachqar in https://github.com/oumi-ai/oumi/pull/2161 * Update wandb requirement from <0.24,>=0.21 to >=0.21,<0.25 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2162 * Update trl requirement from <0.27,>=0.24 to >=0.24,<0.28 by @dependabot[bot] in https://github.com/oumi-ai/oumi/pull/2163 * [fsdp] Update transformer_wrap_class parsing to handle both class names and fully qualified names by @oelachqar in https://github.com/oumi-ai/oumi/pull/2164 * [bug] fix logging of distributed training cli commands by @oelachqar in https://github.com/oumi-ai/oumi/pull/2165 * [bug] revert trl version update & make sure tests run on pyproject.toml changes by @oelachqar in https://github.com/oumi-ai/oumi/pull/2166 * [docs] finish docs clean-up & refresh by @oelachqar in https://github.com/oumi-ai/oumi/pull/2167 * [infer] Add openrouter inference engine by @oelachqar in https://github.com/oumi-ai/oumi/pull/2168 * [docs] update installation instructions by @oelachqar in https://github.com/oumi-ai/oumi/pull/2169 * [infer] Add docs for engines support infer_batch, + raise explicit error if not supported by @oelachqar in https://github.com/oumi-ai/oumi/pull/2170 * [evals] Add rule-based judge examples, cli integration, tests by @oelachqar in https://github.com/oumi-ai/oumi/pull/2171 * Return INIT skypilot clusters in get clusters by class by @oumiandy in https://github.com/oumi-ai/oumi/pull/2178 * add nebius as cloud provider by @min-oumi in https://github.com/oumi-ai/oumi/pull/2179 * [Evaluations] Fixing Typecheker errors by @kaisopos in https://github.com/oumi-ai/oumi/pull/2180 * Adding Batch Support For AttributeSynthesizer by @rlehman221 in https://github.com/oumi-ai/oumi/pull/2181 ## New Contributors * @lrobledo made their first contribution in https://github.com/oumi-ai/oumi/pull/2105 * @RajdeepKushwaha5 made their first contribution in https://github.com/oumi-ai/oumi/pull/2085 * @ritankarsaha made their first contribution in https://github.com/oumi-ai/oumi/pull/2044 * @brian-nguyen made their first contribution in https://github.com/oumi-ai/oumi/pull/2157 * @lefft made their first contribution in https://github.com/oumi-ai/oumi/pull/2156 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.6.0...v0.7

# **Oumi v0.5 Release Notes** We're excited to announce Oumi v0.5, featuring hyperparameter tuning capabilities, expanded inference options, and enhanced launcher functionality. ## **🚀 Major Features** ### **Data Synthesis Module** * Introducing `oumi synth` - a powerful data synthesis module for automatically generating high-quality training datasets using LLMs (#1965) * **Template-based Generation**: Control attributes like difficulty, style, and domain for diverse dataset creation * **Domain-specific Datasets**: Generate data for specialized fields (legal, medical, technical, etc.) * **Data Augmentation**: Expand existing small datasets by generating variations * **Multiple Formats**: Support for instruction-following, QA, and conversational datasets ### **Hyperparameter Tuning Module** * Introducing `oumi tune` - a new hyperparameter search and optimization module for efficient model tuning (#1998, #1991) ### **Inference & Training Enhancements** * **Bedrock Integration**: Added AWS Bedrock Inference Engine support for scalable model deployment (#1983) * **GKD Trainer Support**: New Generalized Knowledge Distillation trainer for model compression workflows (#2000) * **OpenEnv RL Training**: Demo notebook showcasing reinforcement learning training with reward visualization (#1996, #2012) ### **HPC & Launcher Improvements** * **NERSC Perlmutter Support**: Oumi launcher now supports the NERSC Perlmutter HPC cluster (#1959) * **Enhanced Logging**: Added job log trailing and dedicated logs command for better debugging (#1951, #1964) * **Lazy Cloud Initialization**: Improved launcher startup performance (#1985) ## **✨ Improvements** **Model Configuration** * Added Qwen3 VL 4B model configurations (#1992, #1993) * Exposed `chat_template_kwargs` parameter in ModelParams for fine-grained control (#1997) **Developer Experience** * Updated BaseConfig to support non-primitive field types (#1684) * Optional stdout_file parameter in SLURM client (#1974) ## **🐛 Bug Fixes** * Fixed NaN values in dataset analyzer for single-conversation datasets (#1961) * Resolved SLURM environment variable issues (PMI_RANK → SLURM_PROCID) (#2010) * Fixed non-primitive field saving in base config (#2005) * Updated uv pip install commands to include --system flag (#1979) * Unique inference scratch filenames via hashing (#1986) ## **📦 Dependency Updates** * Upgraded transformers: 4.56 → 4.57 (#1966, #1990) * Upgraded TRL: 0.24.0 → 0.25 (#1995, #2011) * Pinned uvicorn version for SkyPilot compatibility (#1978) ## **🎉 New Contributors** Welcome to our new contributors! * @gbladislau * @oumiandy * @AliliRayane ## **📖 Full Changelog** For a complete list of changes, see the [full changelog](https://github.com/oumi-ai/oumi/compare/v0.4.0...0.5)

# Release Notes - v0.4.2 ## 🚀 New Features - **Model Support**: Added support Qwen3-VL ([[#1992](https://github.com/oumi-ai/oumi/pull/1992)](https://github.com/oumi-ai/oumi/pull/1992)) - **HPC Cluster Support**: Added support for NERSC Perlmutter HPC cluster in Oumi launcher ([[#1959](https://github.com/oumi-ai/oumi/pull/1959)](https://github.com/oumi-ai/oumi/pull/1959)) - **Enhanced Logging**: - Added ability to trail logs for launcher jobs ([[#1951](https://github.com/oumi-ai/oumi/pull/1951)](https://github.com/oumi-ai/oumi/pull/1951)) - Added launcher logs command for easier log access ([[#1964](https://github.com/oumi-ai/oumi/pull/1964)](https://github.com/oumi-ai/oumi/pull/1964)) ## 🐛 Bug Fixes - Fixed Sky Pilot unit tests ([[#1967](https://github.com/oumi-ai/oumi/pull/1967)](https://github.com/oumi-ai/oumi/pull/1967)) - Fixed GPU test issues ([[#1970](https://github.com/oumi-ai/oumi/pull/1970)](https://github.com/oumi-ai/oumi/pull/1970)) - Pinned uvicorn version to resolve SkyPilot compatibility ([[#1978](https://github.com/oumi-ai/oumi/pull/1978)](https://github.com/oumi-ai/oumi/pull/1978)) - Updated inference to always hash for unique scratch filenames ([[#1986](https://github.com/oumi-ai/oumi/pull/1986)](https://github.com/oumi-ai/oumi/pull/1986)) - Improved error handling for document processing issues ([[#1989](https://github.com/oumi-ai/oumi/pull/1989)](https://github.com/oumi-ai/oumi/pull/1989)) ## 🔧 Improvements - **Performance**: Lazy initialization of clouds in Oumi launcher for faster startup ([[#1985](https://github.com/oumi-ai/oumi/pull/1985)](https://github.com/oumi-ai/oumi/pull/1985)) - **Code Quality**: - Refactored dataset analysis utilities ([[#1962](https://github.com/oumi-ai/oumi/pull/1962)](https://github.com/oumi-ai/oumi/pull/1962), [[#1982](https://github.com/oumi-ai/oumi/pull/1982)](https://github.com/oumi-ai/oumi/pull/1982)) - Extracted conversation_turns to top-level for better data structure ([[#1969](https://github.com/oumi-ai/oumi/pull/1969)](https://github.com/oumi-ai/oumi/pull/1969)) - Made stdout_file optional in Slurm client ([[#1974](https://github.com/oumi-ai/oumi/pull/1974)](https://github.com/oumi-ai/oumi/pull/1974)) ## 📚 Documentation - Updated README with latest information ([[#1968](https://github.com/oumi-ai/oumi/pull/1968)](https://github.com/oumi-ai/oumi/pull/1968)) - Added synthesis documentation and example configurations ([[#1965](https://github.com/oumi-ai/oumi/pull/1965)](https://github.com/oumi-ai/oumi/pull/1965)) --- **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.4.0...v0.4.2

## What's Changed * Update README.md with v0.3 release by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1904 * Rename BaseExperimentalDpoDataset to BaseDpoDataset by @oelachqar in https://github.com/oumi-ai/oumi/pull/1903 * [tiny] Pin exact omegaconf version by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1905 * Add gpt-oss-120b Together AI inference config by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1907 * feat: Add basic hugging face model management functionality in CLI by @aniruddh-alt in https://github.com/oumi-ai/oumi/pull/1897 * [Breaking change] Change default model dtype from fp32 to "auto" by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1910 * Add inference config for gpt-oss by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1909 * Update dataset planner to use dependency injection by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1913 * Modify document ingestion to read bytes by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1917 * Update dependencies by @oelachqar in https://github.com/oumi-ai/oumi/pull/1908 * Update dependencies [part 2] by @oelachqar in https://github.com/oumi-ai/oumi/pull/1912 * Add processor support to AnalyzeConfig for vision-language datasets by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1916 * Unpin `flash-attn` version by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1918 * Add august webinar to the readme. by @taenin in https://github.com/oumi-ai/oumi/pull/1919 * Add Conversation-Level Analysis by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1914 * Deprecate experimental ring attention support by @oelachqar in https://github.com/oumi-ai/oumi/pull/1921 * [tiny] Cleanup redundant specifications of lora_dropout 0.0 by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1926 * Update README.md to highlight `gpt-oss` support by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1927 * Add lora `target_parameters` field and upgrade `peft` library by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1925 * Improve e2e eval tests by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1924 * Update README.md - updated after Aug webinar by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1930 * refactor: Build tokenizer inside analysis_utils based on input config by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1928 * Replace Luma link with YouTube recording for August webinar by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1931 * Ryan arman add analysis summary by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1929 * Add KTO support for preference tuning by @efsiatras in https://github.com/oumi-ai/oumi/pull/1538 * [tiny] Rename mac to macos in config names by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1932 * GGUF configs, MacOS LlamaCPP configs by @penfever in https://github.com/oumi-ai/oumi/pull/1923 * Enable gpt-oss training and vLLM inference by @penfever in https://github.com/oumi-ai/oumi/pull/1920 * Judge CLI | Display Overall Score by @kaisopos in https://github.com/oumi-ai/oumi/pull/1934 * Replace mentions of deprecated `huggingface-cli` tool with `hf` by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1933 * Add support for custom datasets in Analyze by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1935 * Judge | Validate Input Dataset by @kaisopos in https://github.com/oumi-ai/oumi/pull/1938 * Add Pre-Populated GitHub Issue Link On Failures by @rlehman221 in https://github.com/oumi-ai/oumi/pull/1936 * Adding Verbose Flag by @rlehman221 in https://github.com/oumi-ai/oumi/pull/1940 * [tiny] Update .gitignore by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1944 * Upgrade verl to 0.5.0 by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1941 * Add shell script for launching jobs on NERSC Perlmutter HPC cluster by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1945 * Update llama_cpp_inference_engine.py by @penfever in https://github.com/oumi-ai/oumi/pull/1946 * Simplify github issue creation in cli by @rlehman221 in https://github.com/oumi-ai/oumi/pull/1949 * Add a parsable enum state field to the launcher to convey job status by @taenin in https://github.com/oumi-ai/oumi/pull/1950 * Penfever/api and macos configs by @penfever in https://github.com/oumi-ai/oumi/pull/1947 * Penfever/deepspeed by @penfever in https://github.com/oumi-ai/oumi/pull/1886 * Run GitHub workflows in merge queue instead of on PR events by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1952 * Dependency Upgrade For V4 by @rlehman221 in https://github.com/oumi-ai/oumi/pull/1942 * Allow initializing DatasetAnalyzer directly with a dataset object by @ryan-arman in https://github.com/oumi-ai/oumi/pull/1953 * Follow-up updates to DeepSpeed PR by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1954 * Add vision dpo support, refactor dpo by @oelachqar in https://github.com/oumi-ai/oumi/pull/1922 * Enable users to log data samples during training for debugging by @shanghongsim in https://github.com/oumi-ai/oumi/pull/1943 * [tiny] Remove `add_special_tokens` kwarg from vision DPO dataset processor by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1957 * Fix CPU tests by @taenin in https://github.com/oumi-ai/oumi/pull/1958 * Penfever/llamacpp role fix by @penfever in https://github.com/oumi-ai/oumi/pull/1960 * Update synthesis with usability feedback by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1939 ## New Contributors * @efsiatras made their first contribution in https://github.com/oumi-ai/oumi/pull/1538 * @rlehman221 made their first contribution in https://github.com/oumi-ai/oumi/pull/1936 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.3.0...v0.4.0

# Oumi v0.3 Changelog ## 🔧 Model Quantization (NEW) Quantization is a crucially important family of methods for reducing model size, for example, prior to deployment. Oumi now supports applying Activation-aware Weight Quantization (AWQ) to all models. See how in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Quantization%20Tutorial.ipynb). **Usage Example:** ``` # Quick start - quantize TinyLlama to 4-bit oumi quantize --method awq_q4_0 --model "TinyLlama/TinyLlama-1.1B-Chat-v1.0" --output quantized_model # With configuration file oumi quantize --config quantization_config.yaml ``` ## ⚖️ Judge API V2 (MAJOR UPDATE) LLM-as-a-Judge is a method for using foundation models to reliably evaluate other foundation models. We’ve overhauled Oumi’s LLM-as-Judge interface for ease-of-use and flexibility. Check out our notebook [here](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Simple%20Judge.ipynb). **Usage Example:** ``` from oumi.judges.simple_judge import SimpleJudge # Built-in truthfulness judge simple_judge = SimpleJudge(judge_config="oumi://configs/projects/judges/generic/truthfulness.yaml") dataset = [{"request": "What is the capital of France?", "response": "Rome"}] outputs = simple_judge.judge(dataset) ``` ## 🎯 Adaptive Inference (NEW) 💪 Adaptive Inference, as we term it, refers to new features in Oumi for resuming training (or any task) when a job has crashed, as well as optimizing inference parallelization to maximize bandwidth. Learn more in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Bulk%20Inference%20of%20LLM%20APIs.ipynb). ## 🛠️ Developer Experience - Updated contributing guidelines - Enhanced documentation - Tutorial notebook fixes - Improved error handling and testing - MLflow integration improvements - Multi-node verl Slurm job support - Rich logging handler option ## New Contributors * @amarpal made their first contribution in https://github.com/oumi-ai/oumi/pull/1831 * @42Shawn made their first contribution in https://github.com/oumi-ai/oumi/pull/1837 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.2.1...v0.3.0

# Oumi v0.3 Changelog ## 🔧 Model Quantization (NEW) Quantization is a crucially important family of methods for reducing model size, for example, prior to deployment. Oumi now supports applying Activation-aware Weight Quantization (AWQ) to all models. See how in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Quantization%20Tutorial.ipynb). **Usage Example:** ``` # Quick start - quantize TinyLlama to 4-bit oumi quantize --method awq_q4_0 --model "TinyLlama/TinyLlama-1.1B-Chat-v1.0" --output quantized_model # With configuration file oumi quantize --config quantization_config.yaml ``` ## ⚖️ Judge API V2 (MAJOR UPDATE) LLM-as-a-Judge is a method for using foundation models to reliably evaluate other foundation models. We’ve overhauled Oumi’s LLM-as-Judge interface for ease-of-use and flexibility. Check out our notebook [here](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Simple%20Judge.ipynb). **Usage Example:** ``` from oumi.judges.simple_judge import SimpleJudge # Built-in truthfulness judge simple_judge = SimpleJudge(judge_config="oumi://configs/projects/judges/generic/truthfulness.yaml") dataset = [{"request": "What is the capital of France?", "response": "Rome"}] outputs = simple_judge.judge(dataset) ``` ## 🎯 Adaptive Inference (NEW) 💪 Adaptive Inference, as we term it, refers to new features in Oumi for resuming training (or any task) when a job has crashed, as well as optimizing inference parallelization to maximize bandwidth. Learn more in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Bulk%20Inference%20of%20LLM%20APIs.ipynb). ## 🛠️ Developer Experience - Updated contributing guidelines - Enhanced documentation - Tutorial notebook fixes - Improved error handling and testing - MLflow integration improvements - Multi-node verl Slurm job support - Rich logging handler option ## New Contributors * @amarpal made their first contribution in https://github.com/oumi-ai/oumi/pull/1831 * @42Shawn made their first contribution in https://github.com/oumi-ai/oumi/pull/1837 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.2.1...v0.3.0

# Oumi v0.3 Changelog ## 🔧 Model Quantization (NEW) Quantization is a crucially important family of methods for reducing model size, for example, prior to deployment. Oumi now supports applying Activation-aware Weight Quantization (AWQ) to all models. See how in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Quantization%20Tutorial.ipynb). **Usage Example:** ``` # Quick start - quantize TinyLlama to 4-bit oumi quantize --method awq_q4_0 --model "TinyLlama/TinyLlama-1.1B-Chat-v1.0" --output quantized_model # With configuration file oumi quantize --config quantization_config.yaml ``` ## ⚖️ Judge API V2 (MAJOR UPDATE) LLM-as-a-Judge is a method for using foundation models to reliably evaluate other foundation models. We’ve overhauled Oumi’s LLM-as-Judge interface for ease-of-use and flexibility. Check out our notebook [here](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Simple%20Judge.ipynb). **Usage Example:** ``` from oumi.judges.simple_judge import SimpleJudge # Built-in truthfulness judge simple_judge = SimpleJudge(judge_config="oumi://configs/projects/judges/generic/truthfulness.yaml") dataset = [{"request": "What is the capital of France?", "response": "Rome"}] outputs = simple_judge.judge(dataset) ``` ## 🎯 Adaptive Inference (NEW) 💪 Adaptive Inference, as we term it, refers to new features in Oumi for resuming training (or any task) when a job has crashed, as well as optimizing inference parallelization to maximize bandwidth. Learn more in our [notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Bulk%20Inference%20of%20LLM%20APIs.ipynb). ## 🛠️ Developer Experience - Updated contributing guidelines - Enhanced documentation - Tutorial notebook fixes - Improved error handling and testing - MLflow integration improvements - Multi-node verl Slurm job support - Rich logging handler option ## New Contributors * @amarpal made their first contribution in https://github.com/oumi-ai/oumi/pull/1831 * @42Shawn made their first contribution in https://github.com/oumi-ai/oumi/pull/1837 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.2.1...v0.3.0

## What's Changed * Set infer_online and infer_from_file to private by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1745 * Update launch.md by @shanghongsim in https://github.com/oumi-ai/oumi/pull/1781 * Add adaptive semaphore to enable future adaptive throughput scenarios by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1780 * Fix a pyright regression by @taenin in https://github.com/oumi-ai/oumi/pull/1783 * Judge API V2 | Fix judge config from repo path by @kaisopos in https://github.com/oumi-ai/oumi/pull/1782 * Add permutable attributes and combination sampling for data synthesis by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1773 * Removed collator in finetuning tutorial notebook by @shanghongsim in https://github.com/oumi-ai/oumi/pull/1788 * Update our contributing guidelines. by @taenin in https://github.com/oumi-ai/oumi/pull/1789 * Add adaptive concurrency controller in preparation for adaptive inference by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1784 * Fixed issue with final conversations not consistently being saved by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1795 * Add support for ingesting datasets for synthesis by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1790 * Add support for adaptive inference by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1791 * Add support for Example Sources in Synthesis by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1797 * Webinar announcement and other news by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1800 * Added utm_source parameters by @stefanwebb in https://github.com/oumi-ai/oumi/pull/1802 * Add code to handle document ingestion by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1796 * Add code for handling basic document segmentation by @jgreer013 in https://github.com/oumi-ai/oumi/pull/1803 * Update mflow support in oumi trainer by @oelachqar in https://github.com/oumi-ai/oumi/pull/1804 * Add multi-node verl SLURM job by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1798 * Fixed various tutorial notebooks by @shanghongsim in https://github.com/oumi-ai/oumi/pull/1792 * Add parameter logging to oumi trainer by @oelachqar in https://github.com/oumi-ai/oumi/pull/1807 * Judge API V2 | Enable prompt variable replacement by YAML by @kaisopos in https://github.com/oumi-ai/oumi/pull/1805 * [tiny] Update train config comment header by @wizeng23 in https://github.com/oumi-ai/oumi/pull/1809 * Add experimental option to use the rich logging handler by @oelachqar in https://github.com/oumi-ai/oumi/pull/1810 ## New Contributors * @shanghongsim made their first contribution in https://github.com/oumi-ai/oumi/pull/1781 **Full Changelog**: https://github.com/oumi-ai/oumi/compare/v0.2.0...v0.2.1