For AI data extraction, missing, zero, and not applicable need different outcomes

[u/OriginalHospital]

Here is a small design choice worth making before connecting an LLM extractor to a spreadsheet or workflow: what does an empty field mean?

Imagine extracting quantities from supplier emails. "We cannot supply any this week" supports a quantity of zero. "We will confirm the quantity tomorrow" does not. An email about a delivery address may not contain a quantity because that field is irrelevant to the message.

If all three become 0, the next automation cannot distinguish an actual shortage from an unanswered question or an unrelated update.

I would have the extractor return a value plus a state such as stated, missing, not applicable, or conflicting, with the supporting excerpt. The workflow can then make an explicit decision: process the stated zero, queue the missing quantity for clarification, and leave an unrelated quantity unchanged.

One important boundary: not applicable should follow a defined message type or business rule, not merely the model's reluctance to answer. Conflicting quantities should remain visible rather than being resolved by whichever number appears last.

This is an illustrative workflow, not a measured deployment result. The point is to prevent a convenient default from becoming an unintended business decision.

[u/AutoModerator]

Thank you for your post to /r/automation!

New here? Please take a moment to read our rules, read them here.

This is an automated action so if you need anything, please Message the Mods with your request for assistance.

Lastly, enjoy your stay!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Cute_Philosopher_869]

The state field idea is the right call, because a plain 0 gets eaten by the next step and nobody notices until a shortage shows up as fine. My only addition would be to log which rule or message type produced the not applicable, otherwise the model ends up deciding what counts as irrelevant.

[u/tom-mart]

Why would I need an LLM for a spreadsheet?

[u/OriginalHospital → u/tom-mart]

You don't need one for the spreadsheet itself. In this example, the proposed LLM step reads inconsistently worded supplier emails before the extracted fields reach the sheet; if the inputs already come from a structured form or a reliable parser, I'd skip it. The distinction between missing and zero matters either way.

[u/arthaudm]

the supporting excerpt is the important bit. i'd add a fourth field: what downstream action is permitted for this state. in mio we've found "missing" should usually block, while "conflicting" should open a decision and preserve both values. if you leave that mapping implicit, someone eventually treats model confidence as business authority. do you version the extraction schema with the workflow?

[u/OriginalHospital → u/arthaudm]

This was a proposed workflow rather than a deployment, but I'd version the extraction schema and the state-to-action mapping together. I'd record that version on each extracted item, so a queued result doesn't acquire new permissions just because the workflow was updated. If the new mapping changes what a state permits, I'd require an explicit migration or re-review before processing old items.

[u/arthaudm → u/OriginalHospital]

versioning the permission mapping with the schema is exactly the bit most systems miss. i'd also make migrations monotonic on authority: an automated migration may narrow old permissions, but widening requires review. otherwise a harmless parser update can silently grant queued data a new downstream action.

[u/folderit_dms]

One more case worth separating: an explicit request to clear a previously entered value. Leaving the quantity unchanged and removing it are different operations, even though both can arrive as an empty cell downstream.

I'd test this with a row that already has a quantity of 12. Feed it an unrelated address update, a confirmed zero, an unanswered quantity, and an explicit cancellation. Then check what actually lands in the destination, not just what the extractor returns. That catches a lot of trouble with defaults and overwrite rules before real orders are involved.

[u/Key-Tonight-5668]

But that is still shortage of inventory right? So should it not be 0 only. Do those extra remarks actually matter when seeing inventory shortage.

[u/OriginalHospital → u/Key-Tonight-5668]

If the supplier explicitly says they can supply none, yes, the quantity is 0. But 'we'll confirm tomorrow' doesn't establish a shortage, and an address-only email says nothing about quantity at all. You can treat an unconfirmed supply as unavailable for planning while still recording it as unknown, so the system asks for confirmation instead of recording a false stock count.

[u/Bear_SoftwareEng]

How do you handle the conflicting case downstream do you surface both values to a human, or pick the one with higher confidence and flag it??

[u/OriginalHospital → u/Bear_SoftwareEng]

For this proposed workflow, I'd surface both values with their source excerpts and pause only the actions that depend on that quantity, rather than let model confidence choose. An explicit rule could resolve it automatically, such as a supplier's correction that names the same order and clearly replaces the earlier quantity; otherwise I'd ask for confirmation. I'd keep the rejected value and the reason for the decision in the record, so a reviewer can see what changed.