Build Multi-Agent Workflows with SageMaker and AgentCore

☁️Read original on AWS Machine Learning Blog

#multi-agent #observability #model-routing #awsamazon-sagemaker-ai-and-bedrock-agentcoreamazon-sagemaker-ai amazon-bedrock-agentcore strands-agents

💡Learn how to mix models across SageMaker and AgentCore while gaining token-level observability.

⚡ 30-Second TL;DR

What Changed

Combines OpenAI-compatible SageMaker AI endpoints with Bedrock AgentCore Runtime.

Why It Matters

The approach gives developers more flexibility to mix models and deployment environments within a single agentic system. Improved token-level telemetry can also make latency, usage, and behavior easier to diagnose in production.

What To Do Next

Prototype a two-agent workflow with Strands Agents, route each agent to an OpenAI-compatible SageMaker endpoint through Bedrock AgentCore Runtime, and add token-level telemetry.

Who should care:Developers & AI Engineers

Key Points

•Combines OpenAI-compatible SageMaker AI endpoints with Bedrock AgentCore Runtime.
•Enables specialized agents in one workflow to use different task-appropriate models.
•Adds token-level observability for SageMaker endpoints that Strands Agents does not instrument by default.

🧠 Deep Insight

AI-generated analysis for this event.

🔑 Enhanced Key Takeaways

•The integration leverages the Amazon Bedrock AgentCore Runtime to provide a standardized orchestration layer that abstracts the complexity of inter-agent communication and state management.
•By utilizing OpenAI-compatible endpoints on SageMaker, developers can migrate existing applications built on OpenAI SDKs to AWS infrastructure with minimal code changes while retaining access to custom-tuned models.
•The token-level observability feature is implemented via custom middleware that intercepts request/response payloads, enabling granular cost tracking and latency monitoring for non-Bedrock models.
•This architecture supports a 'bring your own model' (BYOM) approach, allowing organizations to deploy specialized open-weights models (e.g., Llama 3 or Mistral) on SageMaker alongside proprietary models in Bedrock.
•The workflow utilizes Amazon EventBridge for asynchronous event-driven communication between agents, reducing tight coupling in complex multi-agent systems.

📊 Competitor Analysis▸ Show

Feature	AWS (SageMaker + AgentCore)	Google Cloud (Vertex AI Agents)	Microsoft Azure (AI Agent Service)
Orchestration	Bedrock AgentCore	Vertex AI Agent Builder	Azure AI Agent Service
Model Flexibility	High (BYOM + Bedrock)	Medium (Gemini focus)	High (OpenAI + Open Models)
Observability	Custom/SageMaker Native	Vertex AI Observability	Azure AI Monitor
Pricing	Pay-per-token/Instance	Pay-per-token/Instance	Pay-per-token/Instance

🛠️ Technical Deep Dive

The AgentCore Runtime utilizes a directed acyclic graph (DAG) execution model to manage agent dependencies and task sequencing.
SageMaker endpoints are configured with the 'sagemaker-openai-adapter' container, which maps OpenAI API schemas to SageMaker InvokeEndpoint requests.
Token-level observability is achieved by injecting a custom Python decorator into the inference handler, which logs metadata to Amazon CloudWatch Logs and X-Ray.
State persistence between agents is handled via Amazon DynamoDB, providing low-latency access to shared context variables and conversation history.

🔮 Future ImplicationsAI analysis grounded in cited sources

AWS will standardize AgentCore as the primary orchestration framework for all generative AI workflows.

The integration of AgentCore into SageMaker signals a shift toward a unified control plane for both managed and custom-hosted agentic workloads.

Multi-agent observability will become a mandatory requirement for enterprise-grade AI deployments.

The explicit focus on token-level tracking for non-native models highlights the growing demand for cost and performance transparency in complex agent chains.