When you build a workflow step that asks the AI to both explain its thinking and return a structured result — like a true or false value — you may occasionally see the written reasoning reach the correct conclusion while the final structured field says something else. This is a known limitation of steps that combine reasoning and structured output, and it's straightforward to prevent once you understand why it happens.
Important: if a step's result feeds directly into a decision that affects the customer, such as approving a refund, a credit, or a compensation payout, treat this as a class of risk rather than a one-off bug. Follow the guidance below instead of relying on a single AI step's boolean output for that decision.
Why does this happen?
When one step asks a model to both work through logic in natural language and commit to a structured field in the same response, the two parts aren't generated in lockstep. This shows up most often in steps that compare two values against each other, such as checking whether two dates match or whether an identifier has already been seen before.
The more values a step has to compare, or the more nested its output schema is, the more room there is for the written reasoning and the final field to drift apart.
How do I keep a step's reasoning and its output in sync?
Split the reasoning step from the decision step
Instead of asking one step to explain its thinking and decide in the same response, break the work into two steps: one that reasons through the comparison in natural language, and a second, simpler step that turns that reasoning into the structured result. Isolating the decision on its own reduces the chance that the two disagree.
Move exact-match comparisons to a Code step
For comparisons that have one correct answer — two dates matching, two identifiers matching, a value already existing in a list — a Code step is more reliable than asking the AI to compare the values itself. Have the AI extract or normalize the values if needed, then hand them to a Code step that performs a deterministic check.
Keep the output schema simple
If a step is asked to return several variables or nested logic at once, simplify it. Fewer moving parts in a single step's output reduces the chance that any one field drifts from the reasoning behind it.
What does a Code step comparison look like?
A Code step for an exact date match might look like this:
function (date1, date2) {
if (date1 == null || date2 == null) {
return false;
}
date1 = String(date1).trim();
date2 = String(date2).trim();
var isValidFormat = /^\d{4}-\d{2}-\d{2}$/;
if (!isValidFormat.test(date1) || !isValidFormat.test(date2)) {
return false;
}
return date1 === date2;
}The same pattern works for duplicate checks: pass in the two identifiers or fingerprints you want to compare, and let the Code step return true or false based on an exact match, rather than asking an AI step to judge the comparison.
What if I need the AI to prepare the values first?
It's fine, and often necessary, to use an AI step earlier in the workflow to extract or normalize values — for example, pulling a date out of an uploaded image, or standardizing a date format. Keep that step's job limited to extraction, and pass its output into a separate Code step for the actual comparison. Splitting extraction from comparison keeps each step doing the one thing it's best at.
Note: this pattern applies to any workflow step where the outcome depends on comparing two specific values, not just date or duplicate checks.
Questions? Contact support@assembled.com and we'll be glad to help.
Comments
0 comments
Article is closed for comments.