Skip to main content

Local Model Firewall

Secure your local LLMs using TSZ firewall.

This example protects local model 3 running on local model runtime using TSZ guardrails.

Architecture​

TSZ inspects:

  • PII
  • Sensitive patterns
  • Prompt injection

Features​

  • Local model security
  • chat-completions-compatible API
  • Explainable blocking
  • Request IDs
  • Enterprise guardrails

Prerequisites​

Install local model runtime​

brew install local model runtime
local model runtime serve

Pull local model 3 or Model of your choice​

local model runtime pull local model:3b

Start TSZ​

docker compose -f deployment/docker/docker-compose.yml up -d

Setup Python​

cd examples/local model runtime-local-firewall

python -m venv .venv
source .venv/bin/activate

pip install \
  chat-completions provider \
  "tszclient-py @ git+https://source.example/thyris/repository@main"

Run​

python main.py

Example Output​

No REQUEST BLOCKED BY TSZ
Content blocked by security policy: EMAIL

Why this matters​

  • Secure local LLMs
  • No cloud dependency
  • Production security controls
  • Prevents data leakage
  • Full observability

Security Principles​

PrincipleWhy
Fail-closedBlock immediately if any validator fails
ExplainableAlways return reasons and confidence scores
TraceableEvery request has a unique Request ID (RID)
Zero-trustInspect every prompt, no implicit trust

Use cases​

  • Offline AI security
  • On-prem LLMs
  • Privacy-first deployments
  • Enterprise compliance