Modal Code Sandbox

An ephemeral, cost-optimized Modal application for running code sandboxes (opencode/hermes) with integrated LLM inference (vLLM). Designed for minimal costs, maximum performance, and complete data portability.

Features

• Ephemeral Sessions: Volumes and prompt history completely wiped on Modal side, all artifacts saved locally
• Code Sandboxes: Run opencode or hermes in Modal Sandboxes with full terminal/web UI access
• LLM Inference: Integrated vLLM server with OpenAI-compatible API, speculative decoding, reasoning support
• Cost Optimization: Auto-scaledown, fast boot mode, GPU memory snapshots support
• Dual Deployment: Run on Modal directly or via Docker/Docker Compose locally
• Extensible: Plugin architecture following Open/Closed principle
• Verbose Logging: Comprehensive logging at every layer
• Artifact Sync: Automatic bidirectional sync of session artifacts to local machine

Quick Start

Install

pip install -e .

Configure Modal Credentials

# Option 1: Environment variables
export MODAL_TOKEN_ID=your_token_id
export MODAL_TOKEN_SECRET=your_token_secret

# Option 2: Modal config file
modal setup

Create Password Secret

modal secret create opencode-secret OPENCODE_SERVER_PASSWORD=your_password

Start a Session

# Basic usage
modal-code-sandbox start

# With custom config
modal-code-sandbox start --config config.yaml

# With specific model and sandbox
modal-code-sandbox start --model google/gemma-4-26B-A4B-it --sandbox-type opencode

# Run in Docker mode
modal-code-sandbox start --docker

# Custom timeout
modal-code-sandbox start --timeout 2h --idle-timeout 600

Access the Sandbox

After starting, you'll see access information:

Sandbox: abc12345 (opencode)
  Web UI: https://abc12345.modal.direct
  TUI: OPENCODE_SERVER_PASSWORD=xxx opencode attach https://abc12345.modal.direct
  Direct: modal shell abc12345

Model Server: https://abc12345--model-server.modal.direct

Configuration

YAML Config (config.yaml)

modal:
  app_name: "code-sandbox"
  region: "us-east"
  gpu_type: "H200"
  gpu_count: 1
  scaledown_window_seconds: 300

sandbox:
  type: "opencode"
  timeout_seconds: 43200
  github_repo: "modal-labs/modal-examples"
  allow_modal_access: true

model_server:
  model_name: "google/gemma-4-26B-A4B-it"
  fast_boot: false
  enable_reasoning: true

artifacts:
  local_artifacts_dir: "./artifacts"
  sync_on_session_end: true
  wipe_remote_volumes: true

logging:
  level: "DEBUG"
  verbose: true

docker:
  enabled: false

session:
  idle_timeout_seconds: 300
  plugins:
    - "artifact_persistence"
    - "cost_monitoring"
    - "audit_logging"

Environment Variables

Variable	Description	Default
MODAL_TOKEN_ID	Modal token ID	-
MODAL_TOKEN_SECRET	Modal token secret	-
MODAL_APP_NAME	Modal app name	code-sandbox
MODAL_GPU_TYPE	GPU type	H200
MODAL_GPU_COUNT	Number of GPUs	1
SANDBOX_TYPE	Sandbox type (opencode/hermes)	opencode
MODEL_NAME	vLLM model name	google/gemma-4-26B-A4B-it
FAST_BOOT	Enable fast boot	false
ARTIFACTS_DIR	Local artifacts directory	./artifacts
LOG_LEVEL	Log level	DEBUG
DOCKER_ENABLED	Enable Docker mode	false

Docker Deployment

Build Images

modal-code-sandbox docker build

Start Stack

# Start all services
modal-code-sandbox docker up

# Start specific services
modal-code-sandbox docker up --service vllm-model-server opencode-server

Stop Stack

modal-code-sandbox docker down --volumes

View Logs

modal-code-sandbox docker logs opencode-server --follow

Plugin Architecture

The application follows the Open/Closed principle - extensible without modification.

Built-in Plugins

• artifact_persistence: Auto-saves session artifacts locally
• cost_monitoring: Tracks estimated GPU compute costs
• audit_logging: Logs all model requests/responses
• auto_shutdown: Automatically shuts down idle resources

Creating Custom Plugins

from modal_code_sandbox.core.plugin_base import Plugin, PluginMetadata

class MyPlugin(Plugin):
    @property
    def metadata(self) -> PluginMetadata:
        return PluginMetadata(
            name="my_plugin",
            version="1.0.0",
            description="My custom plugin",
            author="Me",
        )

    async def initialize(self, config: Dict[str, Any]) -> None:
        pass

    async def shutdown(self) -> None:
        pass

    async def on_session_start(self, context: SessionContext) -> None:
        pass

    async def on_session_end(self, context: SessionContext) -> None:
        pass

    async def on_sandbox_create(self, context: SandboxContext) -> None:
        pass

    async def on_sandbox_destroy(self, context: SandboxContext) -> None:
        pass

    async def on_model_request(self, context: ModelRequestContext) -> None:
        pass

    async def on_model_response(self, context: ModelResponseContext) -> None:
        pass

# Register
from modal_code_sandbox.core.plugin_base import PluginRegistry
PluginRegistry().register(MyPlugin())

Cost Optimization

• Scaledown Window: 5 minutes (configurable) - shuts down GPU when idle
• Fast Boot Mode: --enforce-eager disables compilation for faster cold starts
• Volume Caching: HuggingFace and vLLM caches persist across sessions
• Local Artifacts: No persistent Modal storage costs
• Auto-shutdown: Idle timeout terminates session automatically

Security

• Sandbox authentication via Modal Secrets
• Encrypted tunnels for sandbox access
• No persistent storage of prompts on Modal
• All artifacts transferred to local machine on session end

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      CodeSandboxApp                          │
├──────────────┬──────────────┬──────────────┬────────────────┤
│ SessionMgr   │ SandboxMgr   │ ModelServer  │ ArtifactMgr    │
│              │              │              │                │
│ • Lifecycle  │ • opencode   │ • vLLM       │ • Sync         │
│ • Idle detect│ • hermes     │ • Speculative│ • Volume wipe  │
│ • Callbacks  │ • Git clone  │ • Reasoning  │ • Verification │
└──────┬───────┴──────┬───────┴──────┬───────┴───────┬────────┘
       │              │              │               │
       ▼              ▼              ▼               ▼
┌─────────────────────────────────────────────────────────────┐
│                    Plugin Registry                           │
│  artifact_persistence │ cost_monitoring │ audit_logging     │
└─────────────────────────────────────────────────────────────┘

License

MIT