Tech quickstart

This article guides you through deploying a model on Deeploy and then layering the governance controls that matter most for (agentic) AI applications: guardrails, tracing, and agent registration through the Model Context Protocol (MCP) server.

The quickstart is split into two parts:

• Beginner — deploy one of our sample models, make your first prediction, and view an explanation. You'll also deploy a small LLM with custom resource configuration and see how model registries such as AzureML, MLflow, and Databricks Unity Catalog plug in. Start here if this is your first time using Deeploy.
• Advanced — attach a guardrail to a Deployment (using either a built-in regex guardrail or a custom model-based guardrail from the guardrail models sample repository), instrument an application with tracing, and connect an MCP-compatible AI agent so it registers and audits itself automatically.

tip

Already comfortable deploying models on Deeploy? Skip straight to the Advanced part.

Beginner: deploy your first model

This part first deploys the Scikit-learn census example and makes a first prediction, then deploys a small LLM to show off resource configuration, and finally covers connecting a model registry. You can also follow the same steps to deploy a model from a repository of your own.

1. Deploy a scikit-learn model

tip

Using Deeploy Cloud? The Scikit-learn census example repository is included in the 'Example' Workspace by default, so you can start at step 3.

1. Navigate to an existing Workspace or create a new one.
2. Navigate to Repositories and click Add. Copy the HTTPS path from the Scikit-learn census example Git repository into the designated field and click Save.
   • If you're trying to add a different repository, consult add a Repository for more details.
3. Navigate to Deployments and click Create → Managed.
4. Choose the existing Repository option. Select the Repository, the master branch, and the latest commit.
5. Choose a name for your Deployment and optionally add a description. Other metadata can be adjusted in the 'metadata.json' file at the root of the folder you are deploying from (learn more about metadata). Take a look at the guardrails selector on the Deployment details step, leave it empty for now, you'll attach one in the Advanced part.
6. Select the scikit-learn model framework.
   • You can leave the options untouched.
7. Choose the trained explainer option and select the SHAP kernel explainer framework.
   • You can leave the options untouched.
8. Skip the transformer framework
9. Click Deploy.
10. Wait until your Deployment has finished deploying. Click on the create deployment event to view more details.
11. Navigate to the Test page (located at the Manage dropdown) to test your Deployment. If you added example input during step 5, click Use example input. Otherwise, copy and paste the example input from the example-request.json file in the Scikit-learn census example Git repository into the request field. Click Predict.
12. View the details of your prediction by navigating to the Predictions page. Click Explain to test an explanation of the prediction. Note that you can also test two prediction enrichment scenarios from the test page: submitting an actual value or an evaluation.

2. Deploy a small LLM and configure its resources

Tabular models like the census example use modest resources by default. LLMs are a different story, even a small one benefits from explicit CPU, memory, and replica configuration. The Hugging Face SmolLM2-135M-Instruct example is a good, lightweight model to practice this on.

1. Navigate to Repositories and click Add. Copy the HTTPS path from the example repository into the designated field and click Save.
2. Navigate to Deployments and click Create. Choose the existing Repository option and select the repository, the main branch, and the latest commit.
3. Choose a name for your Deployment. The repository's metadata.json is picked up automatically.
4. Select the Hugging Face model framework.

   • The repository's model/reference.json already references the model by its Hugging Face model ID:

     {
         "schemaVersion": "2.0.0",
         "reference": {
             "huggingface": {
                 "model": "HuggingFaceTB/SmolLM2-135M-Instruct"
             }
         }
     }

   • note

     Only open, non-gated model IDs can be referenced directly like this, private or gated models need a blob reference instead. See Creating Hugging Face Deployments.

5. Choose the trained explainer option and select the attention standard explainer, one of the two explainer types available for generative Hugging Face models.
6. Open the advanced configuration options on the model step to size the Deployment:
   • Set the CPU and memory request and limit, LLMs typically need more memory headroom than classic ML models. See Advanced deployment options.
   • Set minimum replicas to 0 for a cost-effective, scale-to-zero setup (accepting a cold-start delay on the first request), or to 1+ to keep the Deployment always warm for low-latency responses. Set maximum replicas above the minimum to allow autoscaling under load.
   • On Enterprise/Private Cloud plans with GPU nodes available, select a GPU-enabled node to accelerate inference, see GPU support.
7. Skip the transformer framework and click Deploy.
8. Once the Deployment is Ready, open the Test page and select the chat completions endpoint. Use the example request from the repository's chat_completion.json, or use the completions endpoint with "explain": true in the request body to also retrieve an attention-based explanation.

3. Connect a model registry without a repository

Instead of linking a Git repository, you can generate a reference.json directly from a connected model registry, so Deeploy picks up a specific, versioned model artifact for you. This is available for:

• Azure Machine Learning — after configuring the integration, choose Generate a reference from Azure Machine Learning when creating a Deployment without a Repository. This lists the models and versions registered in your Azure Machine Learning workspace and ingests the one you select into the reference field.
• MLflow — after configuring the integration, reference a registered model by alias or version so Deeploy resolves the artifact from your MLflow model registry.
• Databricks Unity Catalog — after configuring the integration, reference a registered model by alias from Unity Catalog.

Each of these integrations resolves to a specific model version at deploy time, so promoting a new model version in the registry (for example, moving an MLflow alias or Unity Catalog alias to a new version) is enough to update what Deeploy serves, no repository commit required.

You've now deployed a scikit-learn model and an LLM, both with full request/prediction logging and explainability, and seen how to connect a model registry. The rest of this article builds on top of these Deployments to add the controls that agentic, LLM-based applications need.

Advanced: agentic AI governance controls

Agentic applications, LLMs that call tools, chain prompts, or make decisions with limited human review, need governance controls beyond logging and explainability. Deeploy provides three controls purpose-built for this:

• Guardrails flag, block, or mask unsafe input and output at inference time, using either a regex pattern or a dedicated model Deployment.
• Tracing captures the full, multi-step execution of an agentic flow using the OpenTelemetry protocol, so you can inspect exactly what happened inside a request.
• The MCP server connects MCP-compatible agent platforms (Claude, GitHub Copilot, Copilot Studio, ChatGPT, Perplexity, and others) directly to Deeploy, so agents register themselves and send an audit trail of every interaction without any code changes.

1. Add a guardrail to a Deployment

Guardrails apply to input, output, or both, and can either flag (log only) or block (reject) matching requests. There are two ways to add one:

Option A — Regex guardrail (quickest)

Use a built-in regular expression when you want to catch a well-defined pattern, such as email addresses or <script> tags, without deploying any additional model.

1. In your Workspace, go to the Guardrails page and click + Create.
2. Enter a unique name and select guardrail type regex.
3. Choose one of the default patterns (email, inline event handler, prompt injection, script tags) or provide your own.
4. Set a replacement string (default ****) and choose whether it applies to input, output, or both.
5. Click Save.

See Guardrails for the full list of default patterns.

Option B — Model-based guardrail (advanced)

For detection logic that a regular expression can't express, such as classifying prompt-injection attempts or grading an agent's reply against a rubric, deploy a dedicated guardrail model and reference it from another Deployment. The guardrail models sample repository has two ready-to-use examples:

Example	Type	Approach
input-prompt-injection-classifier	Input	Classifies user prompts with the leolee99/PIGuard model, baked into the image for offline, low-latency inference.
output-llm-judge	Output	Uses Claude as an "LLM judge" to grade an agent's reply against a rubric, calling the Anthropic API at request time.

Both expose a small FastAPI service that returns the JSON contract Deeploy expects from a deployment-type guardrail: { "guardrail_triggered": <bool>, ... }.

1. Fork or clone the repository and pick one of the two example folders as your starting point.

2. Build the container image with the folder's build.sh / build-all.sh script, then push it to a registry your Workspace can pull from. Add Docker credentials if the registry is private.

3. Write a reference.json that points at the image and its /chat/completions endpoint, for example:

   {
     "schemaVersion": "2.0.0",
     "reference": {
       "docker": {
         "image": "<your-registry>/guardrail-piguard:1.0",
         "uri": "/chat/completions",
         "port": 8000
       }
     }
   }

4. Create a Deployment for the guardrail using the Custom Docker model framework and the reference.json above. On the Deployment details step, toggle Available as guardrail — this is what makes it selectable as a deployment-type guardrail elsewhere.

5. Once the Deployment is ready, open its Test page and send a sample /chat/completions request to confirm it returns guardrail_triggered: true for unsafe input and false for benign input.

6. Go to the Guardrails page, click + Create, select guardrail type deployment, and pick the Deployment you just created.

7. Attach the guardrail to the Deployment you want to protect (for example, an LLM or agent Deployment) as an input, output, or input + output guardrail, in flag or block mode.

tip

Start every model-based guardrail in flag mode and review the prediction logs before switching to block mode, so you can validate match quality against real traffic first.

See Using a deployment as a guardrail for the full response schema, including the masking variant.

2. Add tracing to observe agent behavior

Guardrails decide what to let through; tracing shows you what actually happened across a multi-step agent execution. Deeploy accepts traces over the OpenTelemetry protocol (OTLP) via HTTP.

1. Generate a Deployment API token for the Deployment you want to trace.

2. Instrument your application with an OpenTelemetry SDK and point the OTLP HTTP exporter at your Deployment's traces endpoint:

   from opentelemetry import trace
   from opentelemetry.sdk.resources import Resource
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import SimpleSpanProcessor
   from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
   
   resource = Resource(attributes={"service.name": "your_service_name"})
   provider = TracerProvider(resource=resource)
   
   workspace_id = "<workspaceId>"
   deployment_id = "<deploymentId>"
   
   otlp_exporter = OTLPSpanExporter(
       headers={"Authorization": "Bearer <DPTexamppletoken>"},
       endpoint=f"https://api.app.deeploy.ml/workspaces/{workspace_id}/deployments/{deployment_id}/traces",
   )
   provider.add_span_processor(SimpleSpanProcessor(otlp_exporter))
   trace.set_tracer_provider(provider)
   
   # ... your agentic application

3. Run your application and open the Deployment's Monitoring → Tracing tab to inspect the resulting traces, spans, and attributes.

info

The tracing endpoint in this example refers to the tracing endpoint for Deeploy Cloud. If you're running the enterprise version of Deeploy, contact support to set up the correct endpoint for your installation.

info

Only HTTP OTLP exporters are supported, gRPC endpoints aren't. See Tracing for sampling strategies and auto-instrumentation of common libraries.

3. Connect an agent through the MCP server

Instead of instrumenting code by hand, you can connect an MCP-compatible AI agent platform directly to Deeploy. Once connected, the agent registers itself as a Deployment and automatically sends an audit trail of every interaction, with no code changes required.

1. In your MCP-compatible platform (Claude, Claude Code, GitHub Copilot, Microsoft Copilot Studio, ChatGPT, Perplexity, ...), add https://mcp.deeploy.ai/mcp as the MCP endpoint.
2. Complete the login form using your Deeploy personal key pair (access_key and secret_key).
3. Add a system instruction telling the agent to use the Deeploy MCP server at the start of every conversation, so the flow below runs automatically rather than requiring a manual prompt each time. See Introduction for the exact wording.
4. Start a new session. The agent automatically:
   • Lists the agents already registered in your Workspace (list_agents).
   • Asks you to choose a name and registers the session under it (register_agent) — using an existing name resumes tracing under that registration instead of creating a duplicate.
   • Silently sends an audit trail entry after every response for the rest of the session (send_trace).
5. Open the resulting registration Deployment's Monitoring → Tracing tab in Deeploy to inspect each traced interaction: input, output, interaction type, timestamp, and session context (environment, model, user).

See What is possible with the MCP server for the full tool reference and an example skill file that makes the registration flow trigger automatically at the start of every session.

Next steps

You've deployed a model, attached a guardrail, added tracing, and connected an agent through MCP. To build these controls into your organization's governance workflows, continue with the Governance quickstart, or dive deeper into:

• Guardrails
• Tracing
• MCP server
• Deployment types