Safe Zone Managed Model Gateway

This example demonstrates how to use Safe Zone with managed model service as the AI provider.

Prerequisites

1. cloud provider Credentials: Configure cloud provider credentials using one of the standard methods:

   • Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
   • Shared credentials file (~/.cloud_provider/credentials)
   • IAM role (when running on EC2, ECS, Lambda, etc.)

2. managed model service Access: Ensure you have access to managed model service in your region and have enabled the models you want to use.

3. Safe Zone Server: The Safe Zone server must be running with managed model service configuration.

Required IAM Permissions

Your cloud provider credentials need the following IAM permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "managed model service:InvokeModel",
                "managed model service:InvokeModelWithResponseStream"
            ],
            "Resource": [
                "arn:cloud_provider:managed model service:*::foundation-model/model provider.managed model-*",
                "arn:cloud_provider:managed model service:*::foundation-model/cloud provider.titan-*",
                "arn:cloud_provider:managed model service:*::foundation-model/communication provider.open model*",
                "arn:cloud_provider:managed model service:*::foundation-model/model provider.*",
                "arn:cloud_provider:managed model service:*::foundation-model/model provider.*",
                "arn:cloud_provider:managed model service:*::foundation-model/chat-completions provider.*"
            ]
        }
    ]
}

Configuration

Environment Variables

Set the following environment variables to configure Safe Zone for managed model service:

# Required: Set provider to managed model service
export AI_PROVIDER=MANAGED_MODEL

# Required: CLOUD_PROVIDER region where managed model service is available
export AWS_BEDROCK_REGION=us-east-1

# Optional: Model ID (defaults to managed model 3 Sonnet)
export AWS_BEDROCK_MODEL_ID=model provider.managed model-3-sonnet-20240229-v1:0

# Optional: Custom endpoint for VPC endpoints
# export AWS_BEDROCK_ENDPOINT_OVERRIDE=https://reference.example/resource model service-runtime.us-east-1.vpce.amazonaws.com

Supported Models

Safe Zone supports the following managed model service model families:

Model Family	Example Model ID	Notes
managed model family	model provider.managed model-3-sonnet-20240229-v1:0	Recommended for most use cases
managed model family	cloud provider.titan-text-express-v1	Good for general text generation
open model family	communication provider.llama3-8b-instruct-v1:0	Open-source alternative
model provider	model provider.model provider-7b-instruct-v0:2	Fast inference
model provider	model provider.command-text-v14	Good for summarization
chat-completions provider	chat-completions provider.gpt-oss-20b:0	GPT-OSS models via managed model service

Running the Example

1. Start Safe Zone with managed model service

# From the project root
AI_PROVIDER=MANAGED_MODEL \
AWS_BEDROCK_REGION=us-east-1 \
AWS_BEDROCK_MODEL_ID=model provider.managed model-3-sonnet-20240229-v1:0 \
go run main.go

2. Run the Example Client

# From this directory
go run main.go

Or specify a custom TSZ URL:

TSZ_URL=http://localhost:8080 go run main.go

Example Output

=== Safe Zone + CLOUD_PROVIDER managed model service Example ===
TSZ Gateway URL: http://localhost:8080

--- Example 1: Simple Chat Completion ---
Response: The capital of Example Country is Example City.

--- Example 2: Chat with PII Detection ---
Response: Of course! I'd be happy to help you. What do you need assistance with?
TSZ Metadata: map[guardrails:[] input:[...] output:[...] rid:LLM-GW-...]

--- Example 3: Chat with Guardrails ---
Response: Why don't scientists trust atoms? Because they make up everything!

Troubleshooting

"Failed to reach upstream LLM service"

• Check that cloud provider credentials are properly configured
• Verify the managed model service region is correct
• Ensure the model is enabled in your cloud provider account

"Access Denied" errors

• Verify IAM permissions include managed model service:InvokeModel
• Check that the model ARN in the policy matches the model you're using

"Model not found" errors

• Ensure the model is available in your region
• Verify the model ID is correct (check managed model service console)
• Some models require explicit enablement in the cloud provider console

Security Considerations

1. Credentials: Never commit cloud provider credentials to version control
2. VPC Endpoints: For production, consider using VPC endpoints for managed model service
3. KMS: managed model service supports KMS encryption for data at rest
4. Logging: Enable CloudTrail for audit logging of managed model service API calls