Google Cloud Blueprint: Introducing Physical-Engineering’s Last Exam for agents — How Does It Work in Production?

TL;DR: Google Cloud has introduced Physical-Engineering’s Last Exam (PLE), a rigorous new benchmark designed to evaluate AI agents on complex, real-world mechanical, electrical, civil, and structural engineering tasks. Because frontier models currently score below 50% due to strict code-compliance and safety constraints, enterprise architects must move beyond simple prompt wrappers. By combining the newly released Agent Development Kit (ADK) 2.0 Graph Workflows, Vertex AI Gemini 2.5 Pro’s multimodal context caching, and Cloud Run Worker Pools, organizations can build deterministic, production-grade engineering agents that pass hard safety gates while optimizing unit economics.

What Google Cloud Shipped & The Enterprise Problem It Solves

In the rapidly evolving landscape of generative AI, the industry has reached a critical inflection point where standard academic evaluations no longer reflect enterprise reality. Today, the Google AI & Infrastructure team, in collaboration with Mercor, announced Physical-Engineering's Last Exam (PLE), a groundbreaking benchmark and open competition. Unlike previous evaluations that measure general knowledge or digital professional work, PLE specifically tests a frontier model’s ability to complete tasks that help build the physical world—designing skyscrapers, power grids, and computer chips.

Architecturally, the enterprise problem PLE exposes is severe: textbook benchmarks fall short for physical engineering. While frontier language models routinely score above 90% on academic tests involving single tool calls or self-contained design problems, they struggle immensely with the chaotic, multi-modal reality of a practicing engineer's workflow. PLE closes this evaluation gap by introducing 85 long-horizon tasks across mechanical (30 tasks), electrical (27 tasks), civil (18 tasks), and structural (10 tasks) engineering.

Instead of answering a single prompt, an AI agent must operate inside a containerized project folder—built on Mercor’s Archipelago environment—that mirrors a real-world engineering desk. Each task contains an average of 61 files (totaling 5,170 files across the benchmark), including dense PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models, and document revisions. The agent must reason through these files, identify governing building codes, run physical limit calculations, and generate production-ready deliverables.

The baseline results are a wake-up call for enterprise architecture: all frontier models score below 50%. Even the highest-performing model reached a mean score of only 48.3%. More critically, while top-scoring agents satisfied 91% of individual constraints, only 41.3% of their deliverables passed every hard safety and feasibility check required by licensed engineers. This data proves that non-deterministic, purely probabilistic LLM pipelines are fundamentally unsafe for physical engineering.

To solve this, Google Cloud has shipped the Agent Development Kit (ADK) 2.0, which introduces Graph Workflows. ADK 2.0 allows architects to weave deterministic code (hard safety gates) with adaptive AI reasoning. When paired with the multimodal capabilities of Vertex AI Gemini models (such as Gemini 2.5 Pro and Gemini 3.8 Flash) and the scalable execution environments of Cloud Run, enterprises can finally build agents capable of passing PLE-grade evaluations in production.

Real-World Use Cases: Where This Moves the Needle in the Field

To understand how this architecture translates from benchmark to business value, we must examine how physical engineering agents operate in the field. Here are three concrete industry use cases demonstrating how this stack solves everyday enterprise friction.

1. High-Throughput Enterprise Workloads: Automated Code Compliance Checking

• The Everyday Problem: In civil and structural engineering firms, licensed engineers spend hundreds of hours manually cross-referencing 3D BIM models and 2D CAD drawings against thousands of pages of municipal building codes. Human fatigue leads to missed constraints, resulting in costly rework during the permitting phase.
• How It Works in Practice: By deploying an ADK 2.0 agent on Cloud Run Worker Pools, the system can asynchronously process massive project folders. The agent uses Gemini 2.5 Pro to natively ingest the 3D BIM models and 2D CAD files alongside the municipal code PDFs. ADK Graph Workflows enforce deterministic routing: if the LLM proposes a structural load that violates the parsed building code, a deterministic Python function catches the error and forces the agent to recalculate before generating the final report.
• The Tangible Impact: Firms can isolate tail-latency and quota boundaries under burst traffic, reducing the initial compliance review cycle from weeks to hours, while ensuring that 100% of the deliverables pass hard safety gates before human review.

2. Zero-Trust Governance & IAM: Securing Proprietary CAD and IP

• The Everyday Problem: Manufacturing and aerospace companies want to use AI to optimize mechanical designs, but their 2D CAD drawings and material science spreadsheets are highly classified intellectual property. Running open-source agents on unmanaged infrastructure risks catastrophic data exfiltration.
• How It Works in Practice: The architecture utilizes Google Cloud's VPC Service Controls (VPC SC) to create a cryptographic boundary around the Cloud Run execution environment, Cloud Storage buckets, and Vertex AI endpoints. The ADK 2.0 agent operates under a strictly scoped IAM Service Account with least-privilege access, ensuring it can only read files for the specific task ID it was assigned.
• The Tangible Impact: Chief Information Security Officers (CISOs) gain mathematical certainty that proprietary engineering designs cannot be exfiltrated to external networks, satisfying strict defense and aerospace compliance mandates while still leveraging frontier AI.

3. Production FinOps & Unit Economics: Optimizing Cost-Per-Request on Massive Contexts

• The Everyday Problem: Processing 61 files per task (averaging 500,000 tokens of building codes, emails, and CAD metadata) for thousands of concurrent engineering tasks results in astronomical LLM inference costs. Re-reading the same 500-page International Building Code for every single task destroys unit economics.
• How It Works in Practice: The architecture leverages Vertex AI Context Caching for Gemini 2.5 Pro. The static building codes and standard engineering reference files are cached once per hour. The ADK 2.0 agent only passes the delta (the specific project files) as new input tokens. Furthermore, shifting the compute from standard Cloud Run Services (which bill for idle wait times during long LLM generations) to Cloud Run Worker Pools optimizes the compute layer.
• The Tangible Impact: As demonstrated in the FinOps simulation below, this architectural shift reduces the total cost of ownership (TCO) by over 45%, transforming an unscalable R&D experiment into a highly profitable production system.

Reference Architecture on Google Cloud

To build an agent capable of passing Physical-Engineering's Last Exam, we must design a topology that handles long-horizon tasks, massive multimodal file ingestion, deterministic safety gating, and strict auditability.

The following reference architecture illustrates how to orchestrate these components securely on Google Cloud.

flowchart LR
    %% External Triggers & Storage
    A[Cloud Storage] -->|Eventarc Trigger: New Project Folder| B
    
    %% Compute Layer
    subgraph ComputeLayer["Cloud Run Execution Environment"]
        B(Cloud Run Worker Pool)
        B -->|ADK 2.0 Graph Workflow| C{Agent Orchestrator}
        C -->|Deterministic Check| D[Python Safety Gates]
        C -->|Adaptive Reasoning| E[Vertex AI SDK]
    end

    %% AI Layer
    subgraph AiLayer["Vertex AI"]
        E <-->|Multimodal Prompt| F((Gemini 2.5 Pro))
        F <-->|Cached Building Codes| G[(Context Caching)]
    end

    %% Data & Audit Layer
    subgraph DataAuditLayer["Data & Audit Layer"]
        C -->|Tool Call: Query Material Specs| H[(Cloud SQL PG17)]
        C -->|Write Audit Log| I[(BigQuery)]
    end

    %% Security Perimeter
    subgraph SecurityPerimeter["VPC Service Controls & IAM"]
        B
        F
        G
        H
        I
    end

    classDef gcp fill:#e8f0fe,stroke:#4285f4,stroke-width:2px,color:#1a73e8;
    class A,B,F,G,H,I gcp;

Architectural Component Breakdown

1. Cloud Storage & Eventarc: The 61 files comprising a PLE task (PDFs, CAD, BIM) are uploaded to a Cloud Storage bucket. Eventarc detects the upload and triggers the agentic workflow asynchronously.
2. Cloud Run Worker Pools: Because PLE tasks are "long-horizon" and require extensive reasoning, standard synchronous HTTP requests will time out. Cloud Run Worker Pools provide a scalable, asynchronous execution environment capable of running background tasks for up to 60 minutes, with full support for network file systems (NFS/Cloud Storage volume mounts) to handle the 5,170 total files across the benchmark.
3. ADK 2.0 Graph Workflows: The Agent Development Kit (ADK) 2.0 orchestrates the logic. Instead of relying on a single, brittle LLM prompt, ADK Graph Workflows route the execution through deterministic Python safety gates. If the agent generates a structural design, the workflow forces it through a hard-coded physics validation script before proceeding.
4. Vertex AI Gemini 2.5 Pro: We utilize gemini-2.5-pro for its massive context window and native multimodal capabilities. Unlike older architectures that required brittle OCR pipelines to parse 2D CAD drawings, Gemini 2.5 Pro can reason over the visual and structural data natively. Context Caching is employed to store static engineering codes, drastically reducing input token costs.
5. Cloud SQL PG17 & BigQuery: Cloud SQL (PostgreSQL 17) serves as the operational database for the agent to query material specifications via Tool Calls. BigQuery acts as the immutable audit ledger, recording every reasoning step, tool invocation, and safety gate validation for licensed engineer review.
6. VPC Service Controls: The entire architecture is wrapped in a VPC SC perimeter, ensuring that proprietary engineering designs cannot be accessed from or exfiltrated to the public internet.

Step-by-Step Implementation

To implement this blueprint, we will use the Python Agent Development Kit (ADK) 2.0 and the Vertex AI SDK. The current year is 2026, and we are utilizing the active gemini-2.5-pro model.

1. Defining the ADK 2.0 Agent with Deterministic Tools

First, we define the agent and equip it with a custom tool that represents our "hard safety gate"—a deterministic function that checks if a proposed structural load complies with physical limits.

# requirements.txt
# google-adk==2.0.0
# google-cloud-aiplatform==1.80.0

import os
from google.adk import Agent
from google.adk.tools import tool
from vertexai.generative_models import GenerativeModel

# 1. Define a deterministic safety gate as an ADK Tool
@tool
def verify_structural_load(proposed_load_kn: float, material_yield_strength_mpa: float, cross_section_area_m2: float) -> str:
    """
    Verifies if the proposed structural load is safe based on material properties.
    This is a hard safety gate required by the PLE benchmark.
    """
    # Deterministic physics calculation: Stress = Force / Area
    # 1 kN = 1000 N; 1 MPa = 1,000,000 N/m^2
    applied_stress_mpa = (proposed_load_kn * 1000) / cross_section_area_m2 / 1_000_000
    
    # Apply a standard engineering safety factor of 2.0
    allowable_stress = material_yield_strength_mpa / 2.0
    
    if applied_stress_mpa <= allowable_stress:
        return f"PASS: Applied stress ({applied_stress_mpa:.2f} MPa) is within allowable limits ({allowable_stress:.2f} MPa)."
    else:
        return f"FAIL: Applied stress ({applied_stress_mpa:.2f} MPa) exceeds allowable limits ({allowable_stress:.2f} MPa). Redesign required."

# 2. Initialize the ADK 2.0 Agent
# We use gemini-2.5-pro for its deep reasoning and large context window required for 61+ files.
engineering_agent = Agent(
    name="ple_structural_engineer",
    model="gemini-2.5-pro",
    instruction=(
        "You are an expert structural engineering agent operating under the Physical-Engineering's Last Exam (PLE) framework. "
        "You must analyze the provided project files (CAD, BIM, PDFs). "
        "Before finalizing any design, you MUST use the `verify_structural_load` tool to ensure your proposed loads pass hard safety gates. "
        "If the tool returns a FAIL, you must iterate and redesign until it returns a PASS."
    ),
    tools=[verify_structural_load],
)

# 3. Execute the Agent (In production, this is triggered via Eventarc/Cloud Run)
def process_ple_task(task_prompt: str, project_files: list):
    print(f"Starting PLE Task Evaluation using {engineering_agent.model}...")
    
    # The ADK framework automatically handles the tool calling loop
    response = engineering_agent.run(
        prompt=task_prompt,
        # In a real implementation, project_files would be passed as multimodal parts
    )
    
    print("Final Deliverable Generated:")
    print(response.text)

if __name__ == "__main__":
    sample_task = "Design a steel support beam for a rooftop HVAC unit. The unit weighs 50 kN. The beam uses A36 steel (Yield Strength: 250 MPa) with a cross-sectional area of 0.005 m^2."
    process_ple_task(sample_task, [])

2. Deploying to Cloud Run Worker Pools

Because PLE tasks are long-horizon and require significant compute time to parse 61 files, we deploy this agent to a Cloud Run Worker Pool rather than a standard synchronous service. This ensures the agent has the execution time and background processing capabilities required.

# 1. Build the container image using Cloud Build
gcloud builds submit --tag gcr.io/YOUR_PROJECT_ID/ple-agent:v1

# 2. Create the Cloud Run Worker Pool
# We allocate 4 vCPUs and 8GiB of memory to handle large file processing in memory
gcloud run worker-pools create ple-agent-pool \
    --image gcr.io/YOUR_PROJECT_ID/ple-agent:v1 \
    --region us-central1 \
    --cpu 4 \
    --memory 8Gi \
    --max-instances 100 \
    --service-account ple-agent-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com \
    --vpc-egress all-traffic \
    --network default

# 3. Dispatch a job to the Worker Pool (typically automated via Eventarc)
gcloud run jobs execute ple-agent-pool-job \
    --worker-pool ple-agent-pool \
    --region us-central1

Production Readiness: FinOps, Quotas & Security Guardrails

Transitioning an agent from a local harness to a production Google Cloud environment requires strict adherence to FinOps, quota management, and security guardrails. The PLE benchmark explicitly evaluates cost efficiency, making architectural optimization mandatory.

📊 Production FinOps & TCO Simulation

When processing 10,000 engineering tasks per month, the choice of compute and LLM caching strategy drastically impacts unit economics. In our architectural evaluation, we compare a naive on-demand approach (Option A) against an optimized architecture using Cloud Run Worker Pools and Vertex AI Context Caching (Option B).

By caching the static building codes and standard reference files, we avoid paying the full input token price for every task. Furthermore, by utilizing Worker Pools, we reduce the execution time and idle billing associated with synchronous HTTP services.

📊 Production FinOps & TCO Simulation: Physical-Engineering Agent (PLE) Processing: 10,000 Tasks/Month (Based on the latest Google SKU information)

Production Workload Assumptions (us-central1 / asia-southeast1):

• 10,000 engineering evaluation tasks per month (e.g., mechanical, electrical, civil, structural).
• Each task processes an average of 61 files (PDFs, CAD, BIM) totaling ~500,000 input tokens and generates 5,000 output tokens.
• Option A uses standard Gemini 2.5 Pro on-demand pricing with Cloud Run Services (120s execution per task).
• Option B uses Gemini 2.5 Pro Context Caching (caching building codes and standard reference files) and Cloud Run Worker Pools (60s execution per task).

Architecture Option	Google SKU Unit Price & Monthly Formula	Estimated Monthly Cost
Standard On-Demand Processing	Cloud Run vCPU (120s * 4 vCPU * 10k): $2.4e-05/vCPU-second × 4,800,000 = $115.20 Cloud Run Memory (120s * 8 GiB * 10k): $2.5e-06/GiB-second × 9,600,000 = $24.00 Gemini 2.5 Pro Input (500k tokens * 10k): $1.25/1M input tokens × 5,000 = $6,250.00 Gemini 2.5 Pro Output (5k tokens * 10k): $10/1M output tokens × 50 = $500.00	$6,889.20 / mo
Optimized Context Caching + Worker Pools	Cloud Run vCPU (60s * 4 vCPU * 10k): $2.4e-05/vCPU-second × 2,400,000 = $57.60 Cloud Run Memory (60s * 8 GiB * 10k): $2.5e-06/GiB-second × 4,800,000 = $12.00 Gemini 2.5 Pro Cached Input (500k tokens * 10k): $0.3125/1M cached input tokens × 5,000 = $1,562.50 Gemini 2.5 Pro Cache Storage (0.5M tokens * 730 hrs): $4.5/1M tokens-hour × 365 = $1,642.50 Gemini 2.5 Pro Output (5k tokens * 10k): $10/1M output tokens × 50 = $500.00	$3,774.60 / mo
Net FinOps Impact (Monthly Savings)	Based on the latest Google SKU information	45.2% TCO Reduction ($3,114.60 / mo)

Official Google Cloud SKU Pricing Sources (2026.09): cloud.google.com, cloud.google.com

Quotas & Scalability Guardrails

When inspecting production topologies for high-throughput agentic workloads, quota exhaustion is the primary cause of failure.

• Vertex AI TPM/RPM Quotas: Processing 500,000 tokens per task across concurrent executions will rapidly exhaust default Tokens Per Minute (TPM) and Requests Per Minute (RPM) quotas. Enterprise architects must request quota increases for gemini-2.5-pro in their target regions and implement exponential backoff and jitter within the ADK 2.0 retry configurations.
• Cloud Run Concurrency: While Cloud Run scales automatically, Worker Pools should be configured with a strict --max-instances limit (e.g., 100) to prevent runaway scaling events from draining the Vertex AI quota pool. This creates a natural backpressure mechanism, queuing Eventarc triggers safely in Pub/Sub until compute capacity is available.

Security & Zero-Trust Guardrails

Physical engineering tasks involve highly sensitive intellectual property. The architecture must enforce a zero-trust posture:

• VPC Service Controls (VPC SC): The entire project must be enclosed in a VPC SC perimeter. This ensures that even if the ADK agent is compromised or hallucinates a malicious tool call, it cannot exfiltrate data to an external API or unauthorized Cloud Storage bucket.
• Dedicated IAM Service Accounts: The Cloud Run Worker Pool must execute under a dedicated Service Account (ple-agent-sa). This account must be granted the absolute minimum permissions required: roles/aiplatform.user for Vertex AI inference, roles/storage.objectViewer for the specific input bucket, and roles/bigquery.dataEditor for the audit log dataset. Broad roles like roles/editor are strictly forbidden in this topology.

By combining the deterministic orchestration of ADK 2.0, the multimodal power of Gemini 2.5 Pro, and the secure, scalable execution of Cloud Run, enterprises can finally deploy AI agents that pass the rigorous demands of physical engineering.