@sglang.function

Decorator for defining SGLang prompt programs

Overview

The @sglang.function decorator converts a Python function into an SGLang prompt program that can be executed with language models.

Syntax

@sglang.function
def my_program(s, arg1, arg2, ...):
    # Program body
    pass

Or with parameters:

@sglang.function(num_api_spec_tokens=100)
def my_program(s, arg1, arg2, ...):
    # Program body
    pass

Parameters

num_api_spec_tokens · int

Number of tokens to reserve for API specifications. Used for advanced features.

Function Requirements

• The first parameter must be named s (the SGLang state object)
• Additional parameters define the program’s inputs
• The function body contains prompt logic using SGLang primitives

Usage

Basic Definition

import sglang as sgl

@sgl.function
def simple_qa(s, question):
    s += sgl.user(question)
    s += sgl.assistant(sgl.gen("answer", max_tokens=100))

Running the Program

state = simple_qa.run(question="What is the capital of France?")
print(state["answer"])

Batch Execution

questions = [
    {"question": "What is 2+2?"},
    {"question": "What is the sky blue?"},
]

states = simple_qa.run_batch(questions)
for state in states:
    print(state["answer"])

Methods

.run()

Executes the program with a single input.

Parameters:

• All function arguments as keyword arguments
• Sampling parameters: max_new_tokens, temperature, top_p, top_k, etc.
• backend: Backend to use (defaults to global backend)
• stream: Whether to stream results

Returns: State object with generated outputs

.run_batch()

Executes the program with multiple inputs in parallel.

Parameters:

• batch_kwargs: List of dictionaries, each containing function arguments
• Sampling parameters (applied to all executions)
• num_threads: Number of parallel threads
• progress_bar: Show progress bar

Returns: List of state objects

.bind()

Partially binds arguments to create a new function.

@sgl.function
def greet(s, name, greeting="Hello"):
    s += f"{greeting}, {name}!"

friendly_greet = greet.bind(greeting="Hi")

.trace()

Traces the program execution for debugging.

trace = simple_qa.trace(question="What is Python?")
print(trace)

.cache()

Caches the program’s prefix for faster repeated execution.

simple_qa.cache()

Example: Multi-turn Conversation

@sgl.function
def chatbot(s, user_msg, context):
    s += sgl.system(context)
    s += sgl.user(user_msg)
    s += sgl.assistant(sgl.gen("response", max_tokens=200))

state = chatbot.run(
    user_msg="Tell me a joke",
    context="You are a friendly comedian",
    temperature=0.9
)
print(state["response"])

See Also

• gen() - Generate text from the model
• select() - Choose from predefined options
• Role markers - Structure conversations