The bot’s settings had two limits on the length of the conversation. Neither of them ever triggered / Habr

If both conditions are met at once, the reason is recorded as the message limit. When the company’s phrase template is empty, the default phrase is sent. And one more minor detail: the old branch, when handing off, also set the message’s rating to “meaningless,” which skewed the analytics. Now the rating remains the one assigned by the model.

Tests from the specification

The behavior was first described through scenarios in the specification, and then each scenario became a test. There are 16 of them; here are some of the names as they are:

passes the agent answer through when no limit is reached hands off once max_messages were already processed, even for a meaningful message still answers with the agent on the last allowed message hands off on the Nth non-meaningful message in a row, counting the current one does not hand off on the first non-meaningful message a meaningful message breaks the non-meaningful streak hands off when both limits are reached (old AND case) uses the default phrase when consult_phrase_template is empty

The test marked “old AND case” checks the only case in which the old condition actually triggered (the broken branch never got reached anyway). I left it in intentionally: the old condition is included in the new one as a special case.

The other scenarios describe what clients saw in the settings and considered to be working. Hence the main rollout risk. Companies with small limits will see the bot go silent and call a manager for the first time, and for them this will look like a new feature rather than a fix.Therefore, before enabling it in production, they need to be notified, and this is recorded as a separate item in the rollout plan.

The error itself is trivial—using AND instead of OR; this happens to everyone. What bothered me more in this story was something else: the setting that the client sees did not have a single automated check. Now the rule is simple: if a number appears in the interface, its behavior has a test, and the test is named so that its title clearly conveys what is promised to the client.