# OPT

## 1. Model Introduction

[OPT: Open Pre-trained Transformer Language Models](https://arxiv.org/abs/2205.01068) is a general-purpose language model trained with autoregressive blank-filling objectives, suitable for various understanding and generation tasks.

**Supported Model Weights:**
| Model                 |
|-----------------------|
| facebook/opt-125m     |
| facebook/opt-350m     |
| facebook/opt-1.3b     |
| facebook/opt-2.7b     |
| facebook/opt-6.7b     |
| facebook/opt-13b      |
| facebook/opt-30b      |
| facebook/opt-66b      |
| facebook/opt-iml-1.3b |
| opt-iml-max-1.3b      |

Note: The model implementation is available at [https://github.com/facebookresearch/metaseq](https://github.com/facebookresearch/metaseq). The OPT-IML (Instruction Meta-Learning) series models demonstrate enhanced instruction-following capabilities through meta-learning techniques.
