# SGLang

## Docs

- [LoRA Serving](https://mintlify.wiki/sgl-project/sglang/advanced/lora.md): Efficiently serve multiple LoRA adapters with a single base model
- [Quantization](https://mintlify.wiki/sgl-project/sglang/advanced/quantization.md): Reduce memory usage and increase throughput with model quantization
- [Separate Reasoning](https://mintlify.wiki/sgl-project/sglang/advanced/separate-reasoning.md): Parse and separate reasoning content from model outputs for reasoning models
- [Speculative Decoding](https://mintlify.wiki/sgl-project/sglang/advanced/speculative-decoding.md): Accelerate inference with EAGLE, MTP, and draft model speculation
- [Structured Outputs](https://mintlify.wiki/sgl-project/sglang/advanced/structured-outputs.md): Constrain model outputs using JSON schemas, regex, and EBNF grammars
- [Tool Calling](https://mintlify.wiki/sgl-project/sglang/advanced/tool-calling.md): Enable function calling with structured tool definitions
- [sglang generate](https://mintlify.wiki/sgl-project/sglang/api/cli/generate.md): Run inference on a multimodal diffusion model
- [sglang serve](https://mintlify.wiki/sgl-project/sglang/api/cli/serve.md): Launch the SGLang server for language models or diffusion models
- [sglang version](https://mintlify.wiki/sgl-project/sglang/api/cli/version.md): Display SGLang version and git revision information
- [Engine](https://mintlify.wiki/sgl-project/sglang/api/engine.md): Python API for SGLang inference engine
- [@sglang.function](https://mintlify.wiki/sgl-project/sglang/api/frontend/function.md): Decorator for defining SGLang prompt programs
- [gen()](https://mintlify.wiki/sgl-project/sglang/api/frontend/gen.md): Generate text from the language model
- [Role Markers](https://mintlify.wiki/sgl-project/sglang/api/frontend/roles.md): Structure conversations with system, user, and assistant roles
- [select()](https://mintlify.wiki/sgl-project/sglang/api/frontend/select.md): Choose from predefined options using the language model
- [Chat Completions](https://mintlify.wiki/sgl-project/sglang/api/http/chat-completions.md): OpenAI-compatible chat completions API
- [Completions](https://mintlify.wiki/sgl-project/sglang/api/http/completions.md): OpenAI-compatible text completions API
- [Embeddings](https://mintlify.wiki/sgl-project/sglang/api/http/embeddings.md): OpenAI-compatible embeddings API
- [Models](https://mintlify.wiki/sgl-project/sglang/api/http/models.md): List and retrieve model information
- [Runtime](https://mintlify.wiki/sgl-project/sglang/api/runtime.md): HTTP server wrapper for SGLang
- [SamplingParams](https://mintlify.wiki/sgl-project/sglang/api/sampling-params.md): Sampling parameters for text generation
- [ServerArgs](https://mintlify.wiki/sgl-project/sglang/api/server-args.md): Configuration options for SGLang server
- [Launch Server](https://mintlify.wiki/sgl-project/sglang/backend/launch-server.md): Learn how to launch and configure the SGLang inference server
- [Native API](https://mintlify.wiki/sgl-project/sglang/backend/native-api.md): Use SGLang's native Python API for inference
- [Offline Engine](https://mintlify.wiki/sgl-project/sglang/backend/offline-engine.md): Use SGLang's lightweight offline engine for batch inference
- [OpenAI Compatible API](https://mintlify.wiki/sgl-project/sglang/backend/openai-compatible-api.md): OpenAI-compatible HTTP API endpoints for chat, completions, and embeddings
- [Sampling Parameters](https://mintlify.wiki/sgl-project/sglang/backend/sampling-parameters.md): Control text generation with sampling parameters
- [Server Arguments](https://mintlify.wiki/sgl-project/sglang/backend/server-arguments.md): Complete reference for SGLang server configuration options
- [System Architecture](https://mintlify.wiki/sgl-project/sglang/concepts/architecture.md): Understanding SGLang's high-performance serving architecture
- [Continuous Batching](https://mintlify.wiki/sgl-project/sglang/concepts/continuous-batching.md): How SGLang dynamically manages request batches for optimal throughput
- [Prefix Caching](https://mintlify.wiki/sgl-project/sglang/concepts/prefix-caching.md): How SGLang's prefix caching system improves efficiency through intelligent KV cache reuse
- [RadixAttention](https://mintlify.wiki/sgl-project/sglang/concepts/radix-attention.md): Understanding SGLang's RadixAttention algorithm for efficient prefix sharing
- [Cloud Platform Deployment](https://mintlify.wiki/sgl-project/sglang/deployment/cloud-platforms.md): Deploy SGLang on AWS, Google Cloud, Azure, and other cloud platforms
- [Docker Deployment](https://mintlify.wiki/sgl-project/sglang/deployment/docker.md): Deploy SGLang using Docker containers for production and development environments
- [Kubernetes Deployment](https://mintlify.wiki/sgl-project/sglang/deployment/kubernetes.md): Deploy SGLang on Kubernetes for production-grade scalability and orchestration
- [Multi-Node Deployment](https://mintlify.wiki/sgl-project/sglang/deployment/multi-node.md): Deploy SGLang across multiple nodes for large-scale distributed inference
- [Adding Models](https://mintlify.wiki/sgl-project/sglang/developer/adding-models.md): Guide to adding support for new models in SGLang
- [Architecture Overview](https://mintlify.wiki/sgl-project/sglang/developer/architecture-overview.md): Understanding SGLang's architecture and design
- [Contribution Guide](https://mintlify.wiki/sgl-project/sglang/developer/contribution-guide.md): How to contribute to SGLang
- [Development Setup](https://mintlify.wiki/sgl-project/sglang/developer/development-setup.md): Setting up your development environment for SGLang
- [Kernel Development](https://mintlify.wiki/sgl-project/sglang/developer/kernel-development.md): Writing and optimizing custom CUDA kernels for SGLang
- [Memory Management](https://mintlify.wiki/sgl-project/sglang/developer/memory-management.md): Understanding SGLang's memory management and KV cache system
- [Scheduler](https://mintlify.wiki/sgl-project/sglang/developer/scheduler.md): Deep dive into SGLang's request scheduler
- [Testing](https://mintlify.wiki/sgl-project/sglang/developer/testing.md): Testing guidelines and best practices for SGLang
- [Data Parallelism](https://mintlify.wiki/sgl-project/sglang/distributed/data-parallelism.md): Replicate models across GPUs for maximum throughput
- [Expert Parallelism](https://mintlify.wiki/sgl-project/sglang/distributed/expert-parallelism.md): Distribute MoE expert weights across GPUs for efficient inference
- [Pipeline Parallelism](https://mintlify.wiki/sgl-project/sglang/distributed/pipeline-parallelism.md): Split model layers across nodes for long-context inference
- [Prefill-Decode Disaggregation](https://mintlify.wiki/sgl-project/sglang/distributed/prefill-decode-disaggregation.md): Separate prefill and decode phases for optimized resource utilization
- [Tensor Parallelism](https://mintlify.wiki/sgl-project/sglang/distributed/tensor-parallelism.md): Split model layers across multiple GPUs for efficient inference
- [Backend Configuration](https://mintlify.wiki/sgl-project/sglang/frontend/backends.md): Configure SGLang to work with different LLM backends and providers
- [Generation Primitives](https://mintlify.wiki/sgl-project/sglang/frontend/generation-primitives.md): Core primitives for text generation, selection, and multimodal inputs
- [Frontend Language Overview](https://mintlify.wiki/sgl-project/sglang/frontend/overview.md): Learn about SGLang's frontend language for high-level LLM programming
- [SGLang Functions](https://mintlify.wiki/sgl-project/sglang/frontend/sglang-functions.md): Learn how to define and use SGLang functions with the @sgl.function decorator
- [Installation](https://mintlify.wiki/sgl-project/sglang/installation.md): Install SGLang using pip, from source, Docker, or other deployment methods
- [SGLang documentation](https://mintlify.wiki/sgl-project/sglang/introduction.md): A high-performance serving framework for large language models and multimodal models
- [DeepSeek Models](https://mintlify.wiki/sgl-project/sglang/models/deepseek.md): Usage guide for DeepSeek's advanced reasoning models in SGLang
- [Llama Models](https://mintlify.wiki/sgl-project/sglang/models/llama.md): Usage guide for Meta's Llama model series in SGLang
- [Multimodal Models](https://mintlify.wiki/sgl-project/sglang/models/multimodal.md): Guide for vision, audio, and video models in SGLang
- [Qwen Models](https://mintlify.wiki/sgl-project/sglang/models/qwen.md): Usage guide for Alibaba's Qwen model series in SGLang
- [Supported Models](https://mintlify.wiki/sgl-project/sglang/models/supported-models.md): Comprehensive list of models supported by SGLang
- [Benchmarking](https://mintlify.wiki/sgl-project/sglang/observability/benchmarking.md): Benchmark SGLang server performance and throughput
- [Prometheus Metrics](https://mintlify.wiki/sgl-project/sglang/observability/metrics.md): Available Prometheus metrics in SGLang
- [Monitoring SGLang](https://mintlify.wiki/sgl-project/sglang/observability/monitoring.md): Set up and configure monitoring for SGLang servers
- [Request Tracing](https://mintlify.wiki/sgl-project/sglang/observability/tracing.md): Enable distributed tracing with OpenTelemetry in SGLang
- [Attention Backends](https://mintlify.wiki/sgl-project/sglang/optimization/attention-backends.md): Comprehensive guide to SGLang's attention backend options
- [CUDA Graph](https://mintlify.wiki/sgl-project/sglang/optimization/cuda-graph.md): CUDA graph optimizations for improved inference performance
- [HiCache](https://mintlify.wiki/sgl-project/sglang/optimization/hicache.md): Hierarchical KV caching for long-context optimization
- [Performance Tuning](https://mintlify.wiki/sgl-project/sglang/optimization/performance-tuning.md): Guidelines for optimizing SGLang performance and throughput
- [Quantized KV Cache](https://mintlify.wiki/sgl-project/sglang/optimization/quantized-kv-cache.md): Reduce memory footprint with FP8 and FP4 KV cache quantization
- [Quick Start](https://mintlify.wiki/sgl-project/sglang/quickstart.md): Launch an SGLang server and make your first API request in minutes
- [Environment Variables](https://mintlify.wiki/sgl-project/sglang/resources/environment-variables.md): Complete reference of SGLang environment variables
- [FAQ](https://mintlify.wiki/sgl-project/sglang/resources/faq.md): Frequently asked questions about SGLang
- [Learn More](https://mintlify.wiki/sgl-project/sglang/resources/learn-more.md): Resources, community links, and learning materials for SGLang
- [Migration Guide](https://mintlify.wiki/sgl-project/sglang/resources/migration-guide.md): Guide for migrating between SGLang versions
- [Troubleshooting](https://mintlify.wiki/sgl-project/sglang/resources/troubleshooting.md): Common errors and tips for resolving them