Deploy functions

• Prompts: LLM prompts with model configuration and templating (see Deploy prompts)
• Tools: General-purpose code that LLMs can invoke to perform operations or access external data
• Scorers: Functions for evaluating LLM output quality (returning a number from 0 to 1)
• Workflows: Chains of two or more prompts for multi-step workflows
• Parameters: Configuration schemas for evaluations with runtime values (see Write parameters)

​

Composability

• A vector search tool that embeds a query, searches for relevant documents, and returns them
• A system prompt with instructions for how to retrieve content and synthesize answers using the tool

​

Deploy tools

import * as braintrust from "braintrust";
import { z } from "zod";

const project = braintrust.projects.create({ name: "calculator" });

project.tools.create({
  handler: ({ op, a, b }) => {
    switch (op) {
      case "add":
        return a + b;
      case "subtract":
        return a - b;
      case "multiply":
        return a * b;
      case "divide":
        return a / b;
    }
  },
  name: "Calculator",
  slug: "calculator",
  description: "A simple calculator that can add, subtract, multiply, and divide.",
  parameters: z.object({
    op: z.enum(["add", "subtract", "multiply", "divide"]),
    a: z.number(),
    b: z.number(),
  }),
  returns: z.number(),
  ifExists: "replace",
});

npx braintrust push calculator.ts

from typing import Literal

import braintrust
from pydantic import BaseModel, RootModel

project = braintrust.projects.create(name="calculator")

class CalculatorInput(BaseModel):
    op: Literal["add", "subtract", "multiply", "divide"]
    a: float
    b: float

class CalculatorOutput(RootModel[float]):
    pass

def calculator(op, a, b):
    if op == "add":
        return a + b
    elif op == "subtract":
        return a - b
    elif op == "multiply":
        return a * b
    elif op == "divide":
        return a / b
    else:
        raise ValueError(f"Unsupported operation: {op}")

project.tools.create(
    handler=calculator,
    name="Calculator",
    slug="calculator",
    description="A simple calculator that can add, subtract, multiply, and divide.",
    parameters=CalculatorInput,
    returns=CalculatorOutput,
)

braintrust push calculator.py

​

View and manage tools

​

Add tools to prompts

​

Call tools directly

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  input: {
    op: "add",
    a: 5,
    b: 3,
  },
});

console.log(result); // 8

​

Deploy scorers

​

Deploy workflows

1. Navigate to Playgrounds
2. Click + Workflow
3. Add prompt nodes by selecting + in the comparison pane
4. Use template variables to pass data between prompts:
   • {{dataset.input}} - Access dataset inputs
   • {{input}} - Access previous prompt’s output
   • {{input.field}} - Access structured output fields
5. Save the agent

​

Invoke functions

• Slug: The unique identifier within a project for any function type (e.g., slug: "calculator")
• Global function name: Built-in Braintrust scorers only - globally unique functions like Factuality from autoevals

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "my-project",
  slug: "my-function",
  input: { query: "hello" },
});

import { invoke } from "braintrust";

const score = await invoke({
  globalFunction: "Factuality",
  functionType: "scorer",
  input: {
    output: "The capital of France is Paris",
    expected: "Paris is the capital of France",
  },
});

​

Version functions

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  version: "a1b2c3d4", // Pin to specific version
  input: { op: "add", a: 5, b: 3 },
});

​

Sync with SDK

• Push to Braintrust: Push changes from your codebase to the UI using braintrust push. You can push one or more files or directories. If you specify a directory, all files under that directory are pushed.
  Report incorrect code

  Copy

  Ask AI

  npx braintrust push calculator.ts
  
  Report incorrect code

  Copy

  Ask AI

  braintrust push calculator.py
  
  Python tools and prompts can be pushed to Braintrust using braintrust push, but pulling from Braintrust is not yet available for Python.
• Pull from Braintrust: Pull changes from the UI to your codebase using braintrust pull:
  Report incorrect code

  Copy

  Ask AI

  npx braintrust pull
  
  Use pull to:
  • Download functions from public projects so others can use them
  • Pin your production environment to a specific prompt version without routing requests through Braintrust
  • Review changes to functions in pull requests
  Currently, braintrust pull only works for TypeScript prompts and tools.

​

Handle dependencies

• TypeScript: Braintrust uses esbuild to bundle your code. Bundling creates a single JavaScript file containing all necessary code, reducing the risk of version mismatches and dependency errors when deploying functions. Since esbuild statically analyzes your code, it cannot handle:
  • Dynamic imports
  • Runtime code modifications
  • Native libraries like SQLite
• Python: Braintrust uses uv to cross-bundle dependencies to the target platform (Linux). This works for most binary dependencies, except for libraries that require on-demand compilation. If you encounter bundling issues, file an issue on GitHub for our JavaScript/TypeScript or Python SDKs.

​

Use the REST API

curl https://api.braintrust.dev/v1/function \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BRAINTRUST_API_KEY" \
  -d '{
    "project_name": "calculator",
    "slug": "calculator",
    "input": {
      "op": "add",
      "a": 5,
      "b": 3
    }
  }'

​

Function features

• Well-defined parameters and return types: Type-safe interfaces using Zod (TypeScript) or Pydantic (Python)
• Streaming and non-streaming invocation: Handle real-time and batch operations
• Automatic tracing and logging: All function calls are traced in Braintrust
• OpenAI argument format: Prompts can be loaded directly in OpenAI-compatible format
• Version control: Functions are automatically versioned with each deployment

​

Organize functions

import * as braintrust from "braintrust";

// Get a handle to the project (creates if it doesn't exist)
const project = braintrust.projects.create({ name: "my-project" });

// Use the project to create functions
project.tools.create({...});
project.prompts.create({...});
project.scorers.create({...});

​

Create custom table views

1. Apply the filters and display settings you want.
2. Open the menu and select Save view… or Save view as….

​

Set default table views

• Organization default: Visible to all members when they open the page. This applies per page — for example, you can set separate organization defaults for Logs, Experiments, and Review. To set an organization default, you need the Manage settings organization permission (included by default in the Owner role). See Access control for details.
• Personal default: Overrides the organization default for you only. Personal defaults are stored in your browser, so they do not carry over across devices or browsers.

1. Switch to the view you want by selecting it from the menu.
2. Open the menu again and hover over the currently selected view to reveal its submenu.
3. Choose Set as personal default view or Set as organization default view.

1. Open the menu and hover over the currently selected view to reveal its submenu.
2. Choose Clear personal default view or Clear organization default view.

​

Next steps

• Write scorers to create custom evaluation functions
• Deploy prompts that use your tools
• Monitor deployments to track function performance
• Manage environments to version functions across environments