.. FlashInfer documentation master file, created by
   sphinx-quickstart on Sat Jan 20 12:31:26 2024.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to FlashInfer's documentation!
======================================

`Blog <https://flashinfer.ai/>`_ | `Discussion Forum <https://github.com/orgs/flashinfer-ai/discussions>`_ | `GitHub <https://github.com/flashinfer-ai/flashinfer/>`_

FlashInfer is a library and kernel generator for Large Language Models that provides high-performance implementation of LLM GPU kernels such as FlashAttention, PageAttention and LoRA. FlashInfer focus on LLM serving and inference, and delivers state-of-the-art performance across diverse scenarios.

.. toctree::
   :maxdepth: 2
   :caption: Get Started

   installation
   cli
   logging
   fi_trace
   trace_apply
   autotuning

.. toctree::
   :maxdepth: 2
   :caption: Tutorials

   tutorials/recursive_attention
   tutorials/kv_layout
   tutorials/generated/jax_tvm_ffi/index

.. toctree::
   :maxdepth: 2
   :caption: PyTorch API Reference

   api/attention
   api/gemm
   api/grouped_mm
   api/fused_moe
   api/cascade
   api/comm
   api/sparse
   api/pod
   api/cudnn
   api/cute_dsl
   api/concat_ops
   api/page
   api/sampling
   api/topk
   api/logits_processor
   api/norm
   api/rope
   api/activation
   api/gdn_decode
   api/gdn_prefill
   api/kda
   api/kda_decode
   api/kda_prefill
   api/mamba
   api/mhc
   api/quantization
   api/green_ctx
   api/fp4_quantization
   api/testing
