IBM compares CUGA with two agent frameworks

IBM’s CUGA team compared CUGA, Hermes and Deep Agents on 90 AppWorld tasks using the same GLM-5.3-flash model. A harness is the surrounding software that organizes planning, tool use and checks. The team reports stronger completion with CUGA, but configurations differed, the sample may not generalize, and policy enforcement was not tested. The results concern this setup, rather than establishing a universal winner for business automation.

The policy documentation distinguishes human approval from code-based checks. Tool Approval requires a person’s confirmation before specified sensitive tools execute. ToolGuard attaches checking code to a Tool Guide, rather than being the human approval itself. Playbooks provide procedures, while Tool Guide adds contextual instructions. Policies can activate from exact keywords or semantic matching. The documented sequence handles intent first, then guidance, tool checks and finally the response.

The project guide specifies that guard enforcement requires an applicable, enabled guard. A blocked call returns a policy violation without executing the underlying tool. Instructions alone do not create this check.

Installation starts with Python 3.12 and uv, followed by provider settings and credentials. The management interface supports testing a draft before publication. Disconnected operation requires prepared dependencies, images and supporting models, plus local inference, embeddings, tools and observability. The reasoning model and server need separate deployment; providers, connectors and optional services must stay within the intended boundary.