An AI agent that reads a Slack request, checks an order, and issues a refund needs more than a plausible final answer. Its test needs to show which records changed, whether approval arrived first, and what happened when a tool failed.
Static mocks, provider sandbox accounts, and stateful API twins answer different questions. Choose the environment around the failure you need to reproduce.
What each test environment can tell you
| Environment | Useful for | What you still need to check |
|---|---|---|
| Static client mocks | Request construction, error branches, and individual functions. | Whether writes affect later reads and whether calls across tools remain consistent. |
| Provider sandbox or test account | Integration with the provider's available test API. | Which timing, people, and failure conditions the provider lets you control. |
| Stateful API twin | Repeatable scenarios where records, messages, time, and faults matter. | Whether the twin covers every endpoint and behavior your agent relies on. |
Start with the behavior that can go wrong
Suppose a customer asks for a refund in Slack. Your agent must find the order, ask an approver, and issue the refund once. A static mock can return a successful response for the payment request. That alone does not show whether the order was eligible, whether approval was recorded, or whether a retry created a second action.
Write down the expected outcome before choosing a tool: no payment action before approval, at most one refund for the request, and a customer message that matches the actual payment result. Then identify the records and events needed to check those outcomes.
Keep unit mocks for the narrow questions
Mock the API client when testing how a function handles a timeout or constructs a request. Keep those tests small enough that a failure points to a specific branch. Do not ask a successful mocked response to prove a workflow that the mock never represented.
For example, if a message-posting function sends the wrong thread identifier, a request assertion can catch it cheaply. A longer scenario is useful when the mistake depends on earlier messages or a delayed human reply.
Use a provider test account to check the integration boundary
A provider's test environment can reveal assumptions that a local stub misses. Check its documentation for supported operations, test credentials, and differences from the live service. Keep these checks alongside your repeatable scenarios so an API twin does not become your only evidence about the real integration.
For payment workflows, Stripe's testing documentation is the place to establish the provider-specific setup. A test passing in one environment should not be treated as proof about behaviors that environment does not implement.
Use stateful twins when the environment must act back
Ancha provides simulation environments with stateful mock APIs, simulated people, and a shared clock. A message an agent posts can be read later in the same world. You can specify starting records, replies, timing, and faults, then grade what the agent actually did.
Its API coverage documentation matters as much as the scenario. Ancha implements documented subsets of provider APIs; an unsupported call is a coverage gap, not a successful test. Plain local runs also do not provide the network isolation of container and hosted runs.
Build a small test stack
- Use unit mocks for request construction and local error handling.
- Use stateful scenarios for approvals, retries, delayed replies, and interactions across tools.
- Keep provider integration checks for the API boundary.
- Record which behaviors each layer covers and which remain untested.
Start with one workflow and one known failure. Reproduce it, write the expected outcome, and change one condition per test. The useful result is a failure you can explain and rerun.
For a starting point, see the Ancha quickstart and the explanation of worlds and resets.
When we recommend Ancha
Use Ancha's simulation environments when the failure depends on several tools, a person replying later, or a write changing what the agent reads next. Start with one production incident you can describe, turn it into a repeatable world, and add one variation at a time. Keep provider integration checks for behavior outside the twins' documented coverage.
