## New Models ### DPT  The DPT model adapts the Vision Transformer (ViT) architecture for dense prediction tasks like semantic segmentation. It uses a ViT as a powerful backbone, processing image information with a global receptive field at each stage. The key innovation lies in its decoder, which reassembles token representations from various transformer stages into image-like feature maps at different resolutions. These are progressively combined using convolutional PSP and FPN blocks to produce full-resolution, high-detail predictions. The model in `smp` can be used with a wide variety of transformer-based encoders ```python import segmentation_models_pytorch as smp # initialize with your own pretrained encoder model = smp.DPT("tu-mobilevitv2_175.cvnets_in1k", classes=2) # load fully-pretrained on ADE20K model = smp.from_pretrained("smp-hub/dpt-large-ade20k") # load the same checkpoint for finetuning model = smp.from_pretrained("smp-hub/dpt-large-ade20k", classes=1, strict=False) ``` The full table of DPT's supported `timm` encoders can be found [here](https://smp.readthedocs.io/en/latest/encoders_dpt.html#dpt-encoders). * Adding DPT by @vedantdalimkar in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1079 ## Models export A lot of work was done to add support for `torch.jit.script`, `torch.compile` (without graph breaks: `fullgraph=True`) and `torch.export` features in all encoders and models. This provides several advantages: - **`torch.jit.script`**: Enables serialization of models into a static graph format, enabling deployment in environments without a Python interpreter and allowing for graph-based optimizations. - **`torch.compile` (with `fullgraph=True`)**: Leverages Just-In-Time (JIT) compilation (e.g., via Triton or Inductor backends) to generate optimized kernels, reducing Python overhead and enabling significant performance improvements through techniques like operator fusion, especially on GPU hardware. `fullgraph=True` minimizes graph breaks, maximizing the scope of these optimizations. - **`torch.export`**: Produces a standardized Ahead-Of-Time (AOT) graph representation, simplifying the process of exporting models to various inference backends and edge devices (e.g., through ExecuTorch) while preserving model dynamism where possible. PRs: * Fix torch compile, script, export by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1031 * Fix Efficientnet encoder for torchscript by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1037 ## Core All encoders from third-party libraries such as `efficientnet-pytorch` and `pretrainedmodels.pytorch` are now vendored by SMP. This means we have copied and refactored the underlying code and moved all checkpoints to the [smp-hub](https://huggingface.co/smp-hub). As a result, you will have **fewer** additional dependencies when installing `smp` and get much faster weights downloads. * Move encoders weights to HF-Hub by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1035 * Vendor pretrainedmodels by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1039 * Vendor efficientnet-pytorch by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1036 ## 🚨🚨🚨 Breaking changes 1. UperNet model was significantly changed to reflect the original implementation and to bring pretrained checkpoints into SMP. Unfortunately, UperNet model weights trained with v0.4.0 will be not compatible with SMP v0.5.0. * Fix UperNet model and add pretrained checkpoints by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1124 2. While the high-level API for modeling should be backward compatible with v0.4.0, internal modules (such as encoders, decoders, blocks) might have changed initialization and forward interfaces. 3. `timm-` prefixed encoders are deprecated, `tu-` variants are now the recommended way to use encoders from the `timm` library. Most of the `timm-` encoders are internally switched to their `tu-` equivalent with state_dict re-mapping (backward-compatible), but this support will be dropped in upcoming versions. ## Other changes * Enable any resolution for Unet by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1029 * Update README.md by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1046 * Add binary segmentation example using cpu by @omidvarnia in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1057 * Load model with mismatched sizes by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1107 * Deprecate use_batchnorm in favor of generalized use_norm parameter by @GuillaumeErhard in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1095 * Extend usage of interpolation_mode to MAnet / UnetPlusPlus / FPN and align PAN by @GuillaumeErhard in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1108 * Fix cls token slicing for DPT by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1121 * add upsampling parameter #1106 by @DCalhas in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1123 * Fix #1125 by @Fede1995 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1126 ## New Contributors * @omidvarnia made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1057 * @GuillaumeErhard made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1095 * @kocabiyik made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1113 * @vedantdalimkar made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1079 * @DCalhas made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1123 * @Fede1995 made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1126 **Full Changelog**: https://github.com/qubvel-org/segmentation_models.pytorch/compare/v0.4.0...v0.5.0

## New models ### Segformer contributed by @brianhou0208  Originally, SegFormer is a transformer-based semantic segmentation model known for its simplicity and efficiency. It uses a lightweight hierarchical encoder to capture multi-scale features and a minimal decoder for fast inference. With `segmentation-models-pytorch` you can utilize the model with a native Mix Vision Transformer encoder as long as with 800+ other encoders supported by the library. Original weights are also supported and can be loaded as follows: ```python import segmentation_models_pytorch as smp model = smp.from_pretrained("smp-hub/segformer-b5-640x640-ade-160k") ``` or with any other encoder: ```python import segmentation_models_pytorch as smp model = smp.Segformer("resnet34") ``` See more checkpoints on the [HF Hub](https://huggingface.co/collections/smp-hub/segformer-6749eb4923dea2c355f29a1f). ### UperNet contributed by @brianhou0208  UPerNet (Unified Perceptual Parsing Network) is a versatile semantic segmentation model designed to handle diverse scene parsing tasks. It combines a Feature Pyramid Network (FPN) with a Pyramid Pooling Module (PPM) to effectively capture multi-scale context. ```python import segmentation_models_pytorch as smp model = smp.UPerNet("resnet34") ``` ## New Encoders Thanks to @brianhou0208 contribution 800+ `timm` encoders are now supported in `segmentation_models.pytorch`. New modern encoders like `convnext`, `efficientvit`, `efficientformerv2`, `hiera`, `mambaout` and more can be used as easy as: ```python import segmentation_models_pytorch as smp model = smp.create_model("upernet", encoder_name="tu-mambaout_small") # or model = smp.UPerNet("tu-mambaout_small") ``` ## New examples - Added [example](https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/camvid_segmentation_multiclass.ipynb) for multi-class segmentation by @TimbusCalin - Added [example](https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/convert_to_onnx.ipynb) for onnx export by @qubvel ## Other changes - Project migrated to `pyproject.toml` by @adamjstewart - Better dependency managing and testing (minimal and latest dependencies, linux/windows/mac platforms) by @adamjstewart - Better type annotations - Tests are refactored for faster CI and local testing by @qubvel ## All changes * Updating the tutorial file by @ytzfhqs in https://github.com/qubvel-org/segmentation_models.pytorch/pull/907 * Example on how to save and load model along with Albumentations preprocessing by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/914 * Add open-in-colab badge for all example notebooks by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/915 * Switch to pyproject.toml by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/917 * Remove dep on mock by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/919 * [feat] Adding camvid segmentation multiclass as an example by @TimbusCalin in https://github.com/qubvel-org/segmentation_models.pytorch/pull/922 * Ruff: format Jupyter notebooks too by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/923 * Remove docker files by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/925 * Test minimum and maximum supported dependencies by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/918 * Test on Linux/macOS/Windows for all supported Python versions by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/930 * Modify Jaccard, Dice and Tversky losses by @zifuwanggg in https://github.com/qubvel-org/segmentation_models.pytorch/pull/927 * [feat] Adding UPerNet by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/926 * Fix dims=None in loss by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/937 * Test PR docs build and update models.rst by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/943 * Update test_models.py by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/940 * Fix UPerNet decoder typo by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/945 * Fix Metric typo by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/966 * Expose timm constructor arguments by @DimitrisMantas in https://github.com/qubvel-org/segmentation_models.pytorch/pull/960 * fix(examples): correct Colab links by @EDM115 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/965 * Update DeepLab models by @DimitrisMantas in https://github.com/qubvel-org/segmentation_models.pytorch/pull/959 * [feat] Adding SegFormer by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/944 * Update MixVisionTransformer by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/975 * silance `"is" with 'str' literal` syntax warning from `pretrainedmodels` in python >= 3.12 by @YoniChechik in https://github.com/qubvel-org/segmentation_models.pytorch/pull/987 * Fix DeepLabV3Plus encoder depth by @munehiro-k in https://github.com/qubvel-org/segmentation_models.pytorch/pull/986 * Fix style by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/989 * Add onnx tutorial by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/990 * Fix Segformer decoder performance by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/998 * Add description for non-MIT licensed codes by @junkoda in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1000 * Fix encoder depth & output stride on DeeplabV3 & DeeplabV3+ by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/991 * Update PAN Decoder support encoder depth by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/999 * Update timm universal (support transformer-style model) by @brianhou0208 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1004 * Refactor tests by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1011 * Dependencies: packaging required for testing by @adamjstewart in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1013 * chore (ci): adopt astral-sh actions by @johnnv1 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1014 * chore (segformer): move decoder converter scripts by @johnnv1 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1017 ## New Contributors * @adamjstewart made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/917 * @TimbusCalin made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/922 * @zifuwanggg made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/927 * @brianhou0208 made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/926 * @DimitrisMantas made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/960 * @EDM115 made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/965 * @YoniChechik made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/987 * @munehiro-k made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/986 * @junkoda made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1000 * @johnnv1 made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/1014 **Full Changelog**: https://github.com/qubvel-org/segmentation_models.pytorch/compare/v0.3.4...v0.4.0

## Updates - 🤗 Hugging Face integration: you can save, load, and share models with HF [Hub](https://huggingface.co/models), see [example notebook](https://github.com/qubvel-org/segmentation_models.pytorch/blob/main/examples/save_load_model_and_share_with_hf_hub.ipynb). ### Full log * To support albumentations >= 1.4.0 some functions need to be renamed by @CallShaul in https://github.com/qubvel-org/segmentation_models.pytorch/pull/870 * Updated false positve and false negative rate functions in functional.py by @vermavinay982 in https://github.com/qubvel-org/segmentation_models.pytorch/pull/855 * Add HF hub mixin by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/876 * use precommit for code linting by @Borda in https://github.com/qubvel-org/segmentation_models.pytorch/pull/829 * Add Ruff for formatting and linting by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/877 * Add docs config by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/878 * Update docs by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/879 * Add `create_model` to docs by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/883 * Update ruff to version 0.5.2 and workflows update by @Smartappli in https://github.com/qubvel-org/segmentation_models.pytorch/pull/892 * Fix hub_mixin.py pop error by @ytzfhqs in https://github.com/qubvel-org/segmentation_models.pytorch/pull/909 * Update HF mixin by @qubvel in https://github.com/qubvel-org/segmentation_models.pytorch/pull/910 ## New Contributors * @CallShaul made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/870 * @vermavinay982 made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/855 * @Borda made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/829 * @Smartappli made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/892 * @ytzfhqs made their first contribution in https://github.com/qubvel-org/segmentation_models.pytorch/pull/909 **Full Changelog**: https://github.com/qubvel-org/segmentation_models.pytorch/compare/v0.3.3...v0.3.4

## Updates - Pytorch image models (timm) version upgrade to 0.9.2

## Updates - Added Apple's Mobile One encoder from [repo](https://github.com/apple/ml-mobileone) (use `encoder_name="mobileone_s{0..4}"`). - Pytorch image models (timm) version upgrade to 0.6.12 (500+ encoders available) - Minor typo fixes and docs updates ## Breaking changes - Minimum Python version 3.6 -> 3.7 Thanks @VadimLevin, @kevinpl07, @Abd-elr4hman