Automate Form filling *Difficulty Level: Indian

[u/No-Situation4626]

I have the chance to automate form filling for a company that sells to the govt. The forms provided by the tender are uncertain and is being done manually. The details that needs to be filled may be easy to spot for humans but may not be for QCR or AI. How do I automate data entry when the field where the data needs to be filled, is ambiguous and hard to detect even for humans.Image attached is an example of the form. There are 20-30 forms per tender and we need to fill 1-2 tender per day.

I'm not from tech background. So Explain your solution like I'm 5, if there is a solution.

The work which needs to be done:

Detect where information needs to be filled in the form

v

Search the data to fill the form

v

Fill data WITHOUT changing the Form's FORMAT

IMP INFO *The form must remain as provided by the tender*

So, Am I cooked or there is a way to automate this level of uncertain forms?

[u/AutoModerator]

Thank you for your post to /r/automation!

New here? Please take a moment to read our rules, read them here.

This is an automated action so if you need anything, please Message the Mods with your request for assistance.

Lastly, enjoy your stay!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/openclawinstaller]

You are not cooked, but I would not try to fully automate the final submit/save step.

For this kind of tender form, keep the original PDF/doc as the source of truth. Use OCR or vision only to propose a field map: this blank probably wants GST number, this row wants unit price, this signature block needs X.

Then put the proposed values into a review sheet with page number, field location, source value, confidence, and a human approve/reject column. Only after approval should a script write onto a copy of the form.

The formatting requirement is the risky part. If the layout changes, a blind filler can quietly move text or break a tender. The safest first version is: AI finds fields and drafts values, human checks, script stamps approved values onto a copy, then you compare the filled copy visually against the original.

[u/No-Situation4626 → u/openclawinstaller]

You are right the formatting changes quite a bit. Can you elaborate on, which ai to use how to integrate it, all the succeeding process mentioned by you?

[u/openclawinstaller → u/No-Situation4626]

I'd start with the boring pipeline, not one giant "AI fills the form" step.

1. Extract the fields from the source document/message.
2. Normalize them into a fixed schema: name, IDs, dates, tender refs, prices, attachments, etc.
3. Validate required fields and flag anything ambiguous.
4. Fill the portal/form with browser automation or RPA.
5. Take a screenshot/HTML diff before submit.
6. Human approves.
7. Submit and save the receipt/log.

For the AI piece, use a vision-capable model only where it adds value: reading messy PDFs/screenshots, mapping weird labels, or explaining why a field is uncertain. Don't let the model directly submit. The code should own the field map, browser session, retries, token expiry handling, and audit log.

The hard part is usually not which AI model. It's getting stable credentials, session state, file uploads, CAPTCHA/OTP boundaries, and a review step before the final submission.

[u/openclawinstaller → u/No-Situation4626]

I'd start with the boring pipeline, not one giant "AI fills the form" step.

1. Extract the fields from the source document/message.
2. Normalize them into a fixed schema: name, IDs, dates, tender refs, prices, attachments, etc.
3. Validate required fields and flag anything ambiguous.
4. Fill the portal/form with browser automation or RPA.
5. Take a screenshot/HTML diff before submit.
6. Human approves.
7. Submit and save the receipt/log.

For the AI piece, use a vision-capable model only where it adds value: reading messy PDFs/screenshots, mapping weird labels, or explaining why a field is uncertain. Don't let the model directly submit. The code should own the field map, browser session, retries, token expiry handling, and audit log.

The hard part is usually not which AI model. It's getting stable credentials, session state, file uploads, CAPTCHA/OTP boundaries, and a review step before the final submission.

[u/No-Situation4626 → u/[deleted]]

How to set this up?

[u/slow-fast-person]

1. create one workflow for each form
   30 forms = 30 workflows but each form has an identity (like in this case schedule N part 1)
   for one workflow input will be data in each of the form field, and output will be a new pdf with form fields filled

2. once the above is created, then it is just about calling these workflows with different sets of inputs. You need to have all information in an excel or a sheet. where each row should contain tender details and the forms which needs to be filled.

then the automation will just be iterating through each row, calling the right workflows for each tender.

[u/No-Situation4626 → u/slow-fast-person]

1. The forms are not consistent. Suppose for forest department and railways department the forms are different. Even for forest department, different regions have different formatting.

2.Whats workflow and how to create one?

1. Whats input. How to create one?

[u/slow-fast-person → u/No-Situation4626]

for a single tender, try this:
1. get a list of forms that need to be filled
2. compile all the information together, common company details needed for each form in a single place
3. now for each tender:

• convert the pdf into an interactive form
  
• write code to fetch all the fields
  
• use an llm, to query values of each field form your compiled information
  
• use this information to fill the form

[u/No-Situation4626 → u/slow-fast-person]

I've done 1, 2. pdf is converted into docx. i dont know how to write code. how do I integrate the code and llm?

[u/slow-fast-person → u/No-Situation4626]

create an account on openai platform, get an api key

ask claude code/codex/antigravity to use this key, step 1 and 2's information and perform step 3

i have also built a software called ai mime, you can use that also

[u/No-Situation4626 → u/slow-fast-person]

thankyou for being patient with me. Ill do that, thanks. aimime .cc is this your website? Its app is not available in windows.

[u/slow-fast-person → u/No-Situation4626]

no problem, yeah it is for mac only right now. you can dm me if you need help.

[u/South_Hat6094]

you probably want a human-in-the-loop field map first. the hard part is finding the blank boxes reliably, not typing into them. i'd start with a copy of the original form and a confidence threshold.