Partner case

Multi-market testing
that ends in a fix list

Native testers checked whether an AI assistant could really finish tasks for users in several markets.

How we work with partners →
In short

How do you test a product in several markets at once?

With native testers in each market running real everyday tasks, and evidence for every result. For an AI assistant, Translia's testers in Japan, Vietnam, Korea, Brazil, Thailand and other markets worked across web, mobile and desktop. Each result had a screenshot and a session ID, so failures could be sorted by cause and fixes assigned.

Where it was stuck
“Done”
was not always done

An international consumer AI app needed to know whether its AI assistant could actually complete tasks for users in different markets. The hard part: the assistant often said “done” when the task was not done.

What we did
Native testers, real tasks,
evidence for every claim

Native testers, real tasks

Testers in Japan, Vietnam, Korea, Brazil, Thailand and other markets ran everyday tasks in their own language, such as booking home services or booking a flight.

Three surfaces

Each task was run on web, mobile and desktop clients.

Evidence for every result

Each result came with screenshots and the session ID and was checked one by one: where the task failed, where a human had to take over, and what the model did wrong.

What changed
Failures sorted by cause,
fixes assigned by category

Because every result had evidence behind it, failures could be sorted by cause: model capability, the target website blocking the assistant, or a bug in the product. The client could assign fixes by category instead of debating anecdotes.

Launching an AI product in several markets?

We run native-speaker testing in each market and hand back a report your client's team can act on.

How we work with partners →