ConvoProbePlaywright-style visual testing for multi-turn conversations. No code.
Those are two different things, and pretending they are the same is how bad answers reach customers.
"Should do" and "does" are two different things.
You know, roughly, what your chatbot is supposed to do. You do not know, precisely, what it actually does. Pretending they are the same is how teams ship chatbots that quietly disappoint users for weeks before anyone notices.
Most chatbot QA today is a vibe.
Someone opens the bot, asks three or four questions, reads the answers, and says "looks fine." That is not testing. That is hoping. And one person checking a handful of flows cannot cover the thousands of conversation paths real users will take.
"Does it work" is the wrong question.
The honest question is "how would I know if it stopped working." If the answer is "a customer would tell us," your chatbot is already worse than you think it is. You just have not measured it yet.
You wouldn't ship a product without testing it. Why ship conversations without testing them?
Define the conversations your chatbot must handle correctly. ConvoProbe runs them automatically and tells you exactly which ones fail — before a single user is affected.
Accuracy, consistency, goal achievement — scored and tracked. Stop relying on a PM clicking through 3 conversations and calling it QA.
Changed a system prompt? Run your full scenario suite in one click. See a diff of what improved and what broke. No surprises in production.
1 in 10
Conversations contain errors
Most teams don't know their chatbot's actual failure rate until they measure it
Hours → Minutes
QA cycle per release
What took a team half a day now runs automatically
10 min
To your first test result
Connect your chatbot, generate scenarios, run them. No setup meetings required.
No SDK. No code in your repo. No integration project.
Paste the endpoint URL and key. That is the entire integration.
Drag turns, branch on conditions, set what a good answer looks like. Or import your Dify DSL and let ConvoProbe draft the first pass for you.
Every turn is scored. Every failure shows you what the bot said versus what it should have said.
Your bot tells a customer the wrong return policy. They show up at the store. Now you have a complaint and a broken promise to fix.
Your bot recommends an out-of-stock product or hallucinates a discount code. The customer tries to check out. It doesn't work. They leave.
Your HR bot gives wrong information about leave policy. An employee makes a decision based on it. Now legal is involved.
What You Do Today
Chat with your bot 3 times and say "looks fine"
No idea if last week's prompt change helped or hurt
Find out about failures from customer complaints
Spend half a day on QA before every release
With ConvoProbe
Run 100+ conversation scenarios automatically
See a scored diff after every prompt change
Find failures before they reach a single user
Ship the same day you make changes
Start with the Free plan. Every feature is included.
$0
Everyone starts here.
What's included
Details coming later.
Deep dives on chatbot testing, multi-turn evaluation, and LLM application quality.
A real test where a Dify chatbot built on GPT-5.2 collapsed from 97 to 22 in the same conversation — plus 5 practical techniques to detect and prevent multi-turn quality degradation.
How I designed multi-turn conversation scenarios for a Dify chatbot, ran them automatically via the API, and measured response quality quantitatively.