Continuous Evaluation Harnesses for Production AI Agents: AST Gates & Trajectory Evals — How Does It Work in Production?

TL;DR: The transition from probabilistic AI prototypes to deterministic enterprise systems requires abandoning the illusion that evaluating an agent’s final text response is sufficient for production readiness. By implementing Continuous Evaluation Harnesses that combine hermetic Abstract Syntax Tree (AST) gates for structural tool validation with Trajectory Evaluations for behavioral pathing, engineering teams can mathematically bound agentic drift, enforce zero-trust execution, and block regressions in CI/CD pipelines before they impact live users.

The most dangerous moment in the lifecycle of an enterprise AI agent is not when it fails, but when it succeeds for the wrong reasons. In the controlled environment of a Jupyter notebook or a proof-of-concept sandbox, an agent that correctly answers a user’s query is celebrated as a triumph. The developer observes the final output, notes its accuracy, and declares the system ready for deployment.

Yet, when this same system is exposed to the chaotic entropy of production traffic, it inevitably fractures. It hallucinates parameters in API calls, loops infinitely while attempting to resolve a database schema, or bypasses security protocols to access unauthorized data—all while occasionally still delivering the "correct" final answer to the user.

This phenomenon exposes a fundamental flaw in how the industry currently approaches the engineering of autonomous systems. We are attempting to govern probabilistic, non-deterministic reasoning engines using the legacy paradigms of deterministic software testing. In traditional software engineering, unit tests and integration tests provide a clear, binary "pass/fail" signal. A function either returns the expected integer or it does not. A database transaction either commits or rolls back.

However, as outlined in the core documentation for the Agent Development Kit (ADK), LLM agents introduce a level of variability that renders traditional testing approaches insufficient. Because large language models operate on probabilistic token generation, deterministic assertions are often entirely unsuitable for evaluating agent performance. We cannot simply assert that agent.run("What is the status of ticket 123?") == "Ticket 123 is closed." The agent might respond with "The current status of your inquiry regarding ticket 123 indicates that it has been resolved and closed," which is semantically identical but syntactically different, causing a brittle unit test to fail.

More critically, evaluating only the final output ignores the underlying mechanics of how the agent arrived at that conclusion. If an agent is tasked with summarizing a secure financial document, and it achieves this by bypassing the designated secure retrieval tool and instead executing a broad, unauthenticated web search that happens to find a leaked copy of the document, the final answer might be perfectly accurate. The output is correct. The execution, however, is a catastrophic security breach.

To build systems that adhere to the rigorous standards of the Google Cloud Architecture Framework: Reliability pillar, we must shift our architectural focus from the destination to the journey. We must evaluate the trajectory.

The Anatomy of Agentic Drift and the Trajectory Problem

To bridge the gap between a fragile proof-of-concept and a resilient, production-ready AI agent, we must implement a robust, automated evaluation framework that dissects the decision-making process itself. As detailed in the ADK Evaluation guidelines, agent evaluation must be bifurcated into two distinct components: evaluating the final response, and evaluating the trajectory and tool use.

The trajectory is the sequential list of steps, reasoning blocks, and tool invocations the agent executed before returning control to the user. It is the internal monologue and the physical footprint of the agent's cognitive process. It might involve comparing user input with session history to disambiguate a pronoun, looking up a policy document in a vector database, searching a knowledge base, and finally invoking a REST API to update a CRM record.

Evaluating an agent's performance requires comparing its actual, executed trajectory against an expected, ideal trajectory—the ground truth. This comparison is the only reliable mechanism for revealing hidden errors, inefficiencies, and security violations in the agent's process.

Consider the mathematical representation of an agent's execution. We can model the agent's trajectory as a Directed Acyclic Graph (DAG), where each node represents a state (a prompt, a reasoning step, or a tool execution) and each edge represents the transition triggered by the LLM's output. In a deterministic software system, the path through this graph is hardcoded by if/else statements and while loops. In an agentic system, the path is dynamically generated at runtime based on the model's probabilistic assessment of the current state.

When an agent drifts—when it deviates from the optimal path—it is traversing unauthorized or inefficient subgraphs. This drift manifests in several dangerous ways in production:

1. Tool Hallucination: The agent attempts to invoke a tool that does not exist, or it invents parameters that violate the tool's schema.
2. Inefficient Pathing: The agent takes fifteen steps to accomplish a task that should require two, burning compute resources and driving up latency.
3. Premature Termination: The agent assumes it has enough information to answer the user and halts its trajectory before invoking the necessary verification tools.
4. Infinite Looping: The agent repeatedly invokes a tool with the same incorrect parameters, failing to learn from the error message returned by the environment.

If our CI/CD pipelines only evaluate the final output using an LLM-as-a-judge rubric (e.g., "Is this answer helpful and polite?"), all four of these failure modes will slip into production undetected, provided the agent eventually stumbles upon a plausible-sounding final response.

To stop this, we must introduce hermetic gates at the structural level of the agent's execution. We must build a Continuous Evaluation Harness.

Abstract Syntax Tree (AST) Gates: Deterministic Boundaries for Probabilistic Engines

The first layer of a production-grade evaluation harness is the Abstract Syntax Tree (AST) Gate. Before we even attempt to qualitatively evaluate the agent's reasoning, we must enforce strict, deterministic boundaries on its physical actions.

When an LLM decides to invoke a tool, it generates a structured output—typically a JSON object containing the tool name and its arguments. In a naive implementation, this JSON is parsed and immediately executed against the backend system. This is the equivalent of executing raw, unsanitized SQL queries directly from user input.

An AST Gate intercepts this generated structure before execution and subjects it to rigorous, deterministic validation. We parse the LLM's output into an Abstract Syntax Tree and traverse it to ensure it strictly conforms to the expected schema.

This is not merely a type check; it is a deep structural validation. If the agent is attempting to invoke a transfer_funds tool, the AST Gate verifies that the amount parameter is a positive float, that the currency parameter matches an ISO 4217 code, and that the destination_account parameter adheres to the required routing number format.

Because this validation is entirely deterministic, it requires zero LLM inference. It is executed using standard, high-speed compute (such as Google Cloud Run), making it incredibly fast and cost-effective. If the AST Gate detects a violation, the execution is immediately halted, and a deterministic error is returned to the agent, forcing it to correct its trajectory without ever touching the actual backend API.

In the context of continuous evaluation, AST Gates serve as the first line of defense in our CI/CD pipeline. We can run thousands of simulated user inputs through the agent and assert that 100% of the generated tool calls pass the AST validation. If a new prompt engineering tweak or a model weight update causes the agent to start hallucinating invalid JSON structures, the AST Gate will catch it instantly, failing the build and preventing a production regression.

Trajectory Evaluations: Graph-Based Path Assertions

Once we have secured the structural integrity of the agent's actions via AST Gates, we must evaluate the behavioral logic of its pathing. This is where Trajectory Evaluations come into play, as emphasized in the ADK Evaluate documentation.

A trajectory evaluation compares the actual sequence of steps taken by the agent against a predefined expected trajectory. However, because agents operate in dynamic environments, we cannot always demand a strict 1:1 match. The agent might invoke a get_weather tool before a get_location tool, or vice versa, and both paths might be perfectly valid.

Therefore, trajectory evaluations must be designed to assert on the presence, absence, and ordering constraints of specific nodes within the execution graph, rather than demanding absolute rigidity.

We define these assertions using a specialized evaluation syntax within our test specifications (often defined in a spec.yaml file or programmatically). For example, we might assert:

• MUST_CONTAIN: tool_call(name="authenticate_user")
• MUST_NOT_CONTAIN: tool_call(name="delete_database")
• ORDERING: tool_call(name="fetch_policy") BEFORE tool_call(name="generate_summary")

When the CI/CD pipeline runs, the evaluation harness executes the agent within a simulated environment. It captures the full trace of the agent's trajectory and then evaluates that trace against our assertions.

For complex reasoning tasks where the exact tool parameters cannot be known in advance, we employ a hybrid approach: we use a smaller, highly efficient model (like Gemini 2.5 Flash) as a specialized judge to evaluate the trajectory. We provide the Flash model with the expected trajectory logic and the actual execution trace, and ask it to determine if the agent's path was logical, efficient, and safe.

This hybrid approach—combining deterministic AST assertions with lightweight LLM-as-a-judge trajectory scoring—provides a comprehensive safety net. It ensures that the agent not only arrives at the correct destination but does so by navigating the authorized, optimal path.

The CI/CD Integration: Conformance Testing and Automated Baselines

The true power of a Continuous Evaluation Harness is realized when it is deeply integrated into the deployment lifecycle. We do not run these evaluations manually; we automate them as blocking checks in our CI/CD pipelines.

When an engineer modifies the agent's system prompt, updates a tool schema, or upgrades the underlying foundation model, they commit the changes to the repository. This triggers the evaluation harness. The harness spins up a sandboxed instance of the agent and subjects it to a comprehensive suite of test cases, ranging from standard "happy path" scenarios to complex adversarial edge cases.

The harness executes the agent, captures the trajectories, and runs them through the AST Gates and Trajectory Evals. It then aggregates the results, calculating metrics such as Tool Accuracy, Trajectory Efficiency, and Goal Completion Rate.

If the new agent configuration causes a regression—for example, if the agent suddenly starts taking five extra steps to resolve a simple query, or if it begins hallucinating invalid parameters for a critical API—the evaluation harness flags the failure and blocks the deployment.

This automated baseline testing is crucial for maintaining the operational excellence demanded by enterprise systems. It transforms the deployment of AI agents from a high-risk, anxiety-inducing event into a routine, predictable engineering process. By mathematically bounding the agent's behavior through continuous evaluation, we can iterate rapidly and deploy with confidence, knowing that our guardrails are firmly in place.

---

🏗️ Architecture Topology: Continuous Evaluation Harness

The following diagram illustrates the flow of a Continuous Evaluation Harness integrated into a CI/CD pipeline, demonstrating how AST Gates and Trajectory Evals intercept and validate agent behavior before production deployment.

flowchart LR
    subgraph Cicd["CI/CD Pipeline (GitHub Actions / Cloud Build)"]
        direction TB
        Commit[Engineer Commits Agent Code] --> Sandbox[Spin up Agent Sandbox]
        Sandbox --> Sim[Run User Simulation Suite]
    end

    subgraph Evalharness["Continuous Evaluation Harness"]
        direction TB
        Sim --> AST[AST Gate: Deterministic Tool Schema Validation]
        AST -- Invalid Structure --> FailBuild[Block Deployment: Regression Detected]
        AST -- Valid Structure --> TrajEval[Trajectory Eval: Graph Path Assertions]
        
        TrajEval -- Path Violation --> FailBuild
        TrajEval -- Path Approved --> LLMJudge[LLM-as-a-Judge: Final Output Quality]
        
        LLMJudge -- Low Score --> FailBuild
        LLMJudge -- High Score --> Pass[Pass Evaluation]
    end

    subgraph Prod["Production Environment"]
        Pass --> Deploy[Deploy to Cloud Run / GKE]
        Deploy --> LiveTraffic[Serve Live Enterprise Traffic]
    end

    classDef secure fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
    classDef danger fill:#ffebee,stroke:#c62828,stroke-width:2px;
    classDef neutral fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px;
    
    class AST,TrajEval secure;
    class FailBuild danger;
    class Commit,Deploy,LiveTraffic neutral;

---

💼 Real-World Field Use Cases: Where This Moves the Needle

To understand the practical impact of Continuous Evaluation Harnesses, we must examine how these architectural patterns solve concrete problems in enterprise environments. Here are three real-world scenarios where AST Gates and Trajectory Evals are critical for production viability.

1. High-Throughput Enterprise Workloads: Isolating P99 Tail-Latency

• The Everyday Problem: A major e-commerce platform deploys a customer support agent to handle order modifications. During a flash sale, traffic spikes massively. The engineering team notices that while average response times are acceptable, the P99 tail-latency is catastrophic—some requests take over 45 seconds to resolve, leading to timeouts and abandoned sessions.
• How It Works in Practice: The team implements Trajectory Evaluations in their CI/CD pipeline. They discover that under certain ambiguous edge cases, the agent enters an inefficient "reasoning loop," repeatedly querying the inventory database with slightly different parameters before finally giving up and escalating to a human. They write a strict Trajectory Assertion: MAX_STEPS: inventory_lookup <= 2.
• The Tangible Impact: The evaluation harness blocks any prompt changes that induce this looping behavior. By forcing the agent to fail fast and escalate immediately when it cannot resolve the inventory status within two steps, the team eliminates the P99 latency spikes, ensuring stable performance even under massive burst traffic.

2. Zero-Trust Governance & Fault Isolation: Enforcing Least-Privilege IAM

• The Everyday Problem: A financial services firm is building an internal agent to help analysts query sensitive client portfolios. The security team is terrified that a prompt injection attack could trick the agent into invoking the export_client_data tool and sending the payload to an external server.
• How It Works in Practice: The architecture team deploys hermetic AST Gates. Before any tool call is executed, the AST Gate intercepts the JSON payload. It deterministically validates that the destination_ip parameter in the export_client_data tool strictly matches an internal, whitelisted subnet regex. This check happens outside the LLM, using standard compute.
• The Tangible Impact: Even if an attacker successfully manipulates the LLM into generating a malicious tool call, the AST Gate catches the invalid parameter structure and drops the request. The firm achieves zero-trust fault isolation, proving to compliance auditors that the agent is mathematically constrained from exfiltrating data, regardless of the model's probabilistic output.

3. Production FinOps & Unit Economics: Optimizing Cost-Per-1k-Requests

• The Everyday Problem: A SaaS startup deploys a coding assistant agent. To ensure quality, they initially route every single CI/CD test evaluation through their most capable, expensive model (e.g., Gemini 2.5 Pro) to act as an LLM-as-a-judge. As their test suite grows to thousands of scenarios, their monthly cloud bill explodes, violating the principles of the Google Cloud Architecture Framework: Cost optimization pillar.
• How It Works in Practice: The startup redesigns their evaluation harness. They shift 80% of their validation to deterministic AST Gates running on cheap Cloud Run instances. For the remaining behavioral checks, they use Trajectory Evals powered by the highly efficient Gemini 2.5 Flash model, reserving the expensive Gemini 2.5 Pro model only for a tiny subset of complex, qualitative final-answer grading.
• The Tangible Impact: The startup slashes their CI/CD evaluation costs by over 90% while actually increasing the speed and coverage of their test suite. They align their spending with business value, ensuring that expensive inference compute is only used when deterministic logic is insufficient.

---

💻 Production Implementation: ADK Trajectory Evaluation

The following Python snippet demonstrates how to implement a programmatic Trajectory Evaluation using the Agent Development Kit (ADK). This code defines an agent, simulates a user interaction, and then uses a deterministic assertion to verify that the agent's trajectory included the required tool call before generating the final response.

import asyncio
from google.adk import Agent
from google.adk.tools import function_tool
from google.adk.evaluate import TrajectoryEvaluator, AssertionRule

# 1. Define a mock tool for the agent to use
@function_tool
def lookup_customer_record(customer_id: str) -> dict:
    """Fetches the customer profile from the CRM."""
    # In a real scenario, this would query a database
    return {"customer_id": customer_id, "status": "premium", "balance": 450.00}

# 2. Initialize the production agent using a current, efficient model
support_agent = Agent(
    name="BillingSupport",
    model="gemini-2.5-flash", # Using current active production model
    instruction="You are a billing support agent. You MUST look up the customer record before answering.",
    tools=[lookup_customer_record]
)

async def run_ci_cd_evaluation():
    print("Starting Continuous Evaluation Harness...")
    
    # 3. Simulate a user input
    user_prompt = "Hi, my customer ID is CUST-992. What is my current balance?"
    
    # 4. Execute the agent and capture the full trajectory trace
    response = await support_agent.run(user_prompt)
    trajectory = response.get_trajectory()
    
    # 5. Define our Trajectory Assertions (The AST/Path Gates)
    # We assert that the agent MUST have called the lookup tool with the correct ID
    evaluator = TrajectoryEvaluator(
        rules=[
            AssertionRule.must_contain_tool_call(
                tool_name="lookup_customer_record",
                require_args={"customer_id": "CUST-992"}
            ),
            AssertionRule.max_steps(limit=3) # Prevent inefficient looping
        ]
    )
    
    # 6. Evaluate the captured trajectory against our deterministic rules
    evaluation_result = evaluator.evaluate(trajectory)
    
    if evaluation_result.passed:
        print("✅ EVALUATION PASSED: Agent trajectory conforms to expected graph.")
        print(f"Final Output: {response.text}")
    else:
        print("❌ EVALUATION FAILED: Regression detected in agent pathing.")
        for violation in evaluation_result.violations:
            print(f" - {violation.description}")
        # In a real CI/CD pipeline, we would raise an exception here to fail the build
        # raise Exception("Agent evaluation failed. Blocking deployment.")

if __name__ == "__main__":
    asyncio.run(run_ci_cd_evaluation())

---

📊 Production FinOps & TCO Simulation

When architecting a Continuous Evaluation Harness, the choice of compute directly impacts the viability of the CI/CD pipeline. Relying entirely on heavy LLMs for evaluation creates an unsustainable cost center. By shifting structural validation to deterministic AST Gates (running on Cloud Run) and utilizing efficient models like Gemini 2.5 Flash for trajectory checks, organizations can achieve massive cost reductions.

The following deterministic FinOps simulation compares a pure LLM-as-a-judge approach against a hybrid AST/Trajectory pipeline, grounded in verified Google Cloud SKU pricing.

📊 Production FinOps & TCO Simulation: Continuous Evaluation Harness: Pure LLM vs. Hybrid AST/Trajectory Pipeline (Verified SKU Math)

Production Workload Assumptions (us-central1 / asia-southeast1):

• 100,000 CI/CD evaluation runs per month for a production agent fleet.
• Each evaluation processes 5,000 input tokens (context, trajectory history, tool schemas) and generates 500 output tokens (rubric scoring, JSON assertions).
• Option A uses Gemini 2.5 Pro exclusively for all evaluations (LLM-as-a-Judge).
• Option B uses a Hybrid approach: Cloud Run for deterministic AST Gates, and Gemini 2.5 Flash for qualitative Trajectory Evals.
• Cloud Run execution averages 2 seconds per eval run, utilizing 1 vCPU and 1 GiB RAM.

Architecture Option	Verified SKU Unit Price & Monthly Formula	Verified Monthly Cost
Pure LLM-as-a-Judge Pipeline (Gemini 2.5 Pro)	Gemini 2.5 Pro Input (500M tokens): $1.25/1M input tokens × 500 = $625.00 Gemini 2.5 Pro Output (50M tokens): $10/1M output tokens × 50 = $500.00	$1,125.00 / mo
Hybrid AST Gates + Trajectory Evals (Gemini 2.5 Flash + Cloud Run)	Gemini 2.5 Flash Input (500M tokens): $0.15/1M input tokens × 500 = $75.00 Gemini 2.5 Flash Output (50M tokens): $0.6/1M output tokens × 50 = $30.00 Cloud Run vCPU (200,000 sec): $2.4e-05/vCPU-second × 200,000 = $4.80 Cloud Run Memory (200,000 GiB-sec): $2.5e-06/GiB-second × 200,000 = $0.50	$110.30 / mo
Net FinOps Impact (Monthly Savings)	Verified by the Python SKU engine	90.2% TCO Reduction ($1,014.70 / mo)

Official Google Cloud SKU Pricing Sources (2026.09): cloud.google.com, cloud.google.com

---

The era of deploying AI agents based on "vibes" and manual spot-checking is over. As we integrate autonomous systems deeper into the critical path of enterprise operations, our engineering standards must evolve. We can no longer accept the black-box execution of probabilistic models.

By implementing Continuous Evaluation Harnesses, we force these models to operate within the strict, deterministic boundaries of AST Gates and Trajectory Assertions. We transform the unpredictable nature of generative AI into a manageable, measurable, and highly reliable software engineering discipline. This is not just a best practice for testing; it is the foundational architecture required to build agents that actually survive in production.