Back to discover
NewsBuild with AI

AWS shows decision models checking agents

Its example adds narrow probability-based checks alongside existing evaluators.

Amber glass channels divide ceramic spheres among three bowls beside a large inspection lens.
CurioFuse · AI-generated illustration

The AWS example connects Jev and Strands Decider to AgentCore Evaluations through custom AWS Lambda functions. These optional checks select predefined answers and return probabilities rather than written explanations. Existing evaluators remain alongside them. Before using a check to decide whether to release an agent, AWS recommends validating it against human labels on your own traffic, then repeating validation when the question or model changes.

TypeSafe describes Jev as a model that accepts unstructured state but produces typed decisions with probabilities. Developers define the possible output structure in advance. That makes its interface different from receiving a paragraph and extracting a decision afterward. The company lists classification, routing, scoring and branching as uses. Its guarantee about matching the output schema concerns the answer’s format; it should not be read as an independently measured guarantee that every decision is correct.

The Strands team positions Decider 2B for local experimentation on a CPU or GPU. It chooses among supplied options or assigns numerical scores, and the release includes weights, training data and scripts. Its limits are part of the choice: it cannot generate prose, making it unsuitable for chatbots, coding or document summaries. The team also says this approach handles complex problems less well than reasoning models.

Sources(3)

Sources & standardsReport a correction