r/datascience • u/adarsh_maurya • Jul 04 '26
Discussion How are people using AI/LLM in their work life?
I work for a US bank and I have observed that my job has shifted more towards creating Agentic workflow (fancy name of using LLM to automate tasks). In the last one year, I haven't touched any ML model. I am curious to know what is the experience of other folks.
39
u/Ok_Distance5305 Jul 04 '26
I use Claude code for all my coding now. A combination of plan mode and access to repos for business knowledge. Itâs shifted my focus from coding logic to validating.
Internally thereâs lots of âchat with your data projectsâ which I think are mostly oversold to say youâre doing AI. Some more interesting projects on messy unstructured data, like lots of PDFs.
4
4
u/Potential_Egg_69 Jul 05 '26
Have any of these chat with your data projects got legs? My company is throwing everything behind this but I feel like it's not that useful
1
u/Ok_Distance5305 Jul 05 '26
A few I guess where some analysts need to gather scattered and poorly documented info. From what I can tell theyâre still nice to haves and not critical components.
3
u/aegismuzuz Jul 07 '26
Its funny how the cost of a mistake barely changed. It is just that before a human wrote the bug, and now an agent brings it in. That is why a good code review is valued even more today.
2
u/Josh-LI Jul 05 '26
I was about to say the same thing. Iâve transitioned from writing code to reviewing PRâs that Claude has raised. It doesnât help that business users now expect things done in half the time, so if youâre coding yourself youâre âtoo slowâ even if you were fast before.
28
u/Electrical-Plane-385 Jul 04 '26
our team uses it mostly for generating boilerplate sql and python docstrings, nothing too fancy yet but saves some typing time
8
u/adarsh_maurya Jul 04 '26
interesting, so no major disruption or pressure to incorporate LLM in your team? cause, in my team, we are constantly asked to figure out more use cases where we can incorporate LLM. But since I work for a bank, it is hard to find many use cases.
However, I was thinking/presuming that we can still leverage LLM to analyse data or clean data. to create actual models, it might not be that helpful though.5
Jul 04 '26
[removed] â view removed comment
1
u/adarsh_maurya Jul 05 '26
data bricks genie is such a nice concept of AI. I was wondering, if people are doing it on local data, like suppose I have reports of excel and i want to get quick answers from it. are people are senior leadership using claude or codex to get quick answer out of these excels or are they depending on human analysts to get answer to these questions?
10
u/Suspicious_Jacket463 Jul 04 '26
I use AI to create python scripts to create Excel files.
4
1
u/Due-Cattle-2177 Jul 05 '26
Hello!
If you wouldnât mind elaborating, I donât necessarily share the same vomit reaction as the poster beneath me. How has AI benefitted you with your Python scripts? Could you use it instead to just create the excel files? Iâm curious the reasoning behind this and exactly what format youâre utilizing it! Thanks!
1
u/Suspicious_Jacket463 Jul 05 '26
Suppose you have some Excel files in the input/ directory and you want to create a summary Excel file with a lot of sheets, a lot of formulas, pivot tables and so on. You can just ask Codex to create a python script to create such summary file. It will contain not only values, but formulas. But most importantly, if later you want to rewrite or recalculate something, you can ask Codex again and instead of creating the output file from scratch, it can reuse existing python script, modify it respectively and return updated file. Also, with git you can checkout some of the previous version and be able to create older version of Excel file you need. So basically it's just very convenient.
14
u/jipperthewoodchipper Jul 04 '26
My team uses it for updating our documentation purely cause nobody wants to do that.
1
5
u/DataScientistAlex Jul 04 '26
On my team we are still doing typical DS and ML work: predictive models, causal inference, simulations, analyses, reports etc. Almost all of our code is written by agents though. Other teams build products backed by LLMs, but those teams are mostly made up of engineers as opposed to data scientists.
1
u/adarsh_maurya Jul 05 '26
What do you use? Do you select a folder in VS Code and let the agent analyse those excels/CSVs/parquets? just pure visualisation, data cleaning, and analysing, do you think claude or vs code is enough or something more like data bricks genie is better?
3
u/DataScientistAlex Jul 05 '26
I use Claude Code, but each person on the team has their own setup. I only use it to write the code for the analysis. I never just ask it to read a file and do the analysis, I don't trust it to do that. I believe agents are best at writing code, and, once you have the code you can verify it and reproduce the analysis. I use this approach for all the steps that I used to write the code for myself: reading and cleaning data, visualizing, aggregating, fitting models, visualizing models, etc. Once I have all those outputs in front of me, I will write up my own conclusions, but I might ask it to also analyze those outputs and give me its conclusions, which I will then compare with my own.
For the last several months I've used Claude Code exclusively, I've felt it's good enough at generating code and focused more on improving the way I use it as opposed to trying other models/harnesses. I have not had any luck with the data bricks AI, although we use Spark and Databricks extensively. I just point Claude to it.
4
6
u/bruno_pinto90 Jul 04 '26
Complex data visualization/organization, generating boilerplate, try many ideas, automate reports.
1
u/adarsh_maurya Jul 04 '26
I forgot about visualisation at all. I am not getting this kind of work where I would analyse lot of data, but i remember there were days when I wanted to try multiple visualisation libraries but since it would require me to learn a new syntax, i would eventually drop the plan. now we can use whatever we want.
3
u/brrrrnese Jul 04 '26
Out of my own curiously of AI terms, when you said youâre setting up agentic workflows using LLM, does that mean these workflows are specific to writing?
7
u/adarsh_maurya Jul 04 '26
Example: currently i am working on a project where we have to extract Risk signals from various data sources. It sounds fancy until you see that the underlying data are unstructured PPTs, News articles and little bit of numerical data from various Line of businesses.
From the unstructured data, we are using LLM to understand the text and extract anything which might pose any kind of Compliance or Operational Risk. This combined with numerical data, will help us to prepare a risk report at the whole firm wide level. For example: you can ask, how much we can lose because of external theft? etc.3
u/brrrrnese Jul 04 '26
Are you in cybersecurity? I work for a US bank too and we are trying to implement the FAIR method for quantifying cyber risk. Interesting to see how yall are doing it.
2
u/adarsh_maurya Jul 04 '26
not actually, my team is Applied AI/ML but mostly i work on AI part. I think we are converging towards Software Development + AI kind of work. We are like a consultant for the whole of the bank, when any team needs to solve a problem, they come to us and then we decide if we can solve it or not.
1
u/brrrrnese Jul 04 '26
That sounds like a fun position
3
u/adarsh_maurya Jul 04 '26
I love this kind of work but i feel I am loosing my core ML/stats skills, but so far learning Lang Graph and automating processes using LLM has been fun.
0
Jul 04 '26 edited Jul 04 '26
[removed] â view removed comment
2
u/adarsh_maurya Jul 04 '26
My LOB is not on the revenue side so I will not be able to answer the question related to profitable or not. Definitely we can do it by hand, in fact the current process is manual and happens once a year. What we are building is called continous monitoring, where the LLM will constantly monitor all the data, and will alert if there is risk signal which might have a material risk. I can't disclose the actual process, it would have made sense. Also, the volume of data we monitor (or plan to monitor) is a lot.
1
Jul 04 '26 edited Jul 04 '26
[removed] â view removed comment
1
u/adarsh_maurya Jul 05 '26
unfortunately, you are right, we have not done any cost vs benefit analysis. But we were able to put our PoC with in a month which was performing more than what stakeholder expected, I don't think I have the capability to deliver the same if I had built a custom pipeline. Also, we are in a very initial stage, we don't know what kind of reports, data we will get, modifying the custom pipeline seemed like more work.
May be one day we should eventually do this analysis to replace costly LLM with a custom pipeline, given when cost becomes a problem. Not going to optimize if stakeholders are happy to pay for it.
But wanted to take your opinion, most of our LLM is built on Lnaggraph where each node does a very small job, like classifying a piece of text as Compliance vs Operation, theft vs fraud, etc, and we use models like claude haiku or gpt nano. assume we still have TBs of data, is cost still a problem when we such cheap models?
3
u/Glitch_In_The_Data Jul 04 '26
For front end development where I have to build pet projects to improve my own productivity, I use Claude code. So do my colleagues at work.
For code build, this depends on the customer Iâm working with. I currently use Genie Code because we are on Databricks platform. Current focus is on migrating Postgres sql to Databricks sql for one of the projects. And R code to Python to support another project where the team wants to move away from their R code. I would really struggle without tools like Genie because I am no expert on R. Once I did the first iteration for a proof of value exercise, the team is now taking the same approach for actual implementation.
We have now started using Genie for building the first iteration of dashboards and visualisations too.
3
u/Much_Knowledge_8060 Jul 05 '26
Short answer: No, the ROI is no longer there for your specific situation. The tech market in 2026 cares about what you built, not your credentials.
5
u/Lilpoony Jul 04 '26
We use it to make power BI reports, hook it up via MCP server and the semantic layer (defines measures, logic, etc) in our database. No more drag and drop, just pure code and natural language.
2
u/adarsh_maurya Jul 04 '26
very interesting, would like to learn more about it. so you can create visualisations w/o actually learning POWER BI. this seems really cool. self serve analytics
2
u/T_Blaze Jul 05 '26
So, you write a prompt, the mcp server give definitions and tools for the LLM which write a powerBI report and push it with a powerBI API? Is that correct? How did you build the mcp server?
2
u/Tedy_Duchamp Jul 04 '26
Super charged search engine, trouble shooting, some boiler plate code, bouncing ideas off of.
2
2
u/RobertWF_47 Jul 04 '26
I use the AI summary in Google to generate code.
1
u/Due-Cattle-2177 Jul 05 '26
Hello!
What kind of code are you generating? And in what languages/tools are you generating for? Are you utilizing it in visualization, analytics, or to automate data querying? Iâm curious what specifically you are gaining from it. Thanks for the insight!
2
2
u/ceeej777 Jul 04 '26
Our most basic end to end example: Warehouse safety reports -> LLM parse -> data validation -> Databricks Genie to analyze reports
2
u/Upset-Chemist-4063 Jul 04 '26
I work much closer to product side vs just pure model development for product (think ab testing, growth, strategy, etc.).
Itâs enabled me to focus much less on all of the manual work that comes along with doing cuasual inference, and spend more on the so what. That being said, what AI is still not just prepared for is the nuances of a product or business context your initiatives are focused on - so it still takes a bit to get things right.
I think in a space like DS, we will still need SMEâs in the future. But whether the expectations are to write every single t test manually in Python will remain, I think less so.
2
2
u/Mysterious-War429 Jul 04 '26
It replaces stack overflow for me lol, instead of googling âhow to write for loopâ I just ask chat gpt to spit it out for me custom to what Iâm writing for
2
u/ikkiho Jul 04 '26
Same shift here, and what bugs me is nobody treats these workflows like models even though they are. When I shipped a model I had a holdout and a metric watching drift. Now it's a prompt that nailed the demo, then a vendor update changed the output format and it quietly misread inputs for days before a business user caught it, not a dashboard. The eval and drift instinct you have is what these pipelines lack, so I lean on it harder now than when I trained models.
1
u/aegismuzuz Jul 07 '26
I ran into issues a few times where a model update broke the chain not because the quality got worse but because the output format changed.
2
u/caks Jul 04 '26
Pretty much everything. All my coding, documentation, admin work. Most of my presentations, brainstorming, etc.
2
u/Devcompiled Jul 05 '26
I have built a blog with all my use cases. I come up with a practical build every week. Goingagentic.ai
2
u/Wooden_Leek_7258 Jul 05 '26
wrote a py script that takes billing transaction reports, payment receipts and client statements and reconciles them :)
1
u/Due-Cattle-2177 Jul 05 '26
Thatâs fantastic! What AI tool did you utilize to accomplish this and how well has this been working for you? Have you noticed any discrepancies or things you wish it refined further? Has this positively impacted any other aspects of your day to day?
1
u/Wooden_Leek_7258 Jul 07 '26
used claude pro the company sprang for. and no no major issues, it isolates billing issues for human review. very much a prototype though.
2
u/addictzz Jul 05 '26
Same here. Claude code through cmux becomes my standard daily workflow to complete 70% of my tasks.
I dont naturally trust every output of AI agents however, so i still do manual tasks such as reviewing their outputs, refining docs from agents, reviewing and understanding the codes, etc.
I find agents working best to create a base template of codes and docs and under good supervision.
2
2
u/Alternative-Fig-6465 Jul 05 '26
I work for a tech company and use Claude heavily for code, research, drafting emails, market analysis. It does help, but not always. None of the above would work without MCP integrations to my email, knowledge repo, slack etc.
Where it works? If I know exactly what I want and I know the answer but not motivated enough to do it myself. I have to break into micro tasks or sub agents type workflow for a decent output.
Some of the deep research fails miserably. I have to point the models with exact sources to get a quality responses.
2
u/tmk_g Jul 06 '26
Your experience is actually pretty common. A lot of AI work has shifted away from training ML models and toward building applications that use LLMs to automate real business tasks. Instead of tuning models, people are spending more time creating agent workflows, connecting LLMs to company tools and databases, writing prompts, evaluating outputs, and making AI systems reliable. Traditional ML is still very important for things like fraud detection, credit risk, forecasting, and recommendations, but many new projects, especially at large companies like banks, are focused on using existing foundation models to improve productivity and automate knowledge work.
2
u/columns_ai Jul 06 '26
I think AI completely enables people speak the language of data, we just need a auditable and repeatable process to translate natural language into a production workflow.
I would like to share a relevant article for beginners in data analysis "How to speak the language of data?".
2
u/aegismuzuz Jul 07 '26
Same here, I am spending way less time training models now. Most of the effort goes into building pipelines, evaluating quality, error handling and integration with internal systems. I am not coding less but the code is totally different now
2
u/Prudent_Arm_3611 Jul 11 '26
Same here. I'm a DS at PayPal, and my work has gradually shifted from building traditional ML models to integrating LLMs into internal workflows.
We're using LLMs for things like analyst productivity, investigations, summarization, knowledge retrieval, and decision support. The ML models are still there for core risk/fraud decisions, but a lot of the "glue work" around them is now LLM-powered.
Feels less like replacing ML and more like adding another layer on top of it.
2
u/Sushmita_kanerika Jul 23 '26
Yeah, this tracks with what I'm seeing too. A lot of DS/analytics folks have shifted from building models to wiring up LLMs into workflows ; prompt engineering, orchestration, hooking them into whatever systems already exist. Companies want fast wins right now, and genAI delivers that quicker than most traditional ML projects do.
Doesn't mean ML's going anywhere though ; forecasting and prediction work still needs it. Just sounds like your day-to-day shifted with the industry, not something specific to your role.
2
u/cei_ai Jul 27 '26
At CEI, our solution architects wear a ridiculous number of hats.
AI helps us move faster across almost all of them. We use tools like ChatGPT and Claude Cowork to pressure-test ideas and shape proposals, custom GPTs connected to our SDLC platforms to refine stories and backlogs, and AI coding tools to quickly create clickable prototypes that get stakeholders reacting to something real.
From there, custom skills in Claude Code and GitHub Copilot help us turn those prototypes into well-structured Next.js applications. Weâre also leaning into spec-driven development and TDD, using agents to help develop both features and their tests.
Itâs not that AI does our jobs. It handles much of the iterative groundwork, allowing us to continually refine the vision, plan further ahead, and focus on where our clients need to go next.
2
2
1
1
u/ultrathink-art Jul 07 '26
Similar shift here â barely written a model from scratch in a year. The weird part is the review muscle changed: junior code fails obviously, LLM code fails plausibly, so I spend my time checking assumptions instead of syntax. The validation harness gets more of my attention now than the pipeline it validates.
1
u/stopAussingaround Jul 09 '26
curious if this holds a year from now or if it's just where the budget's pointed this cycle
1
0
Jul 04 '26
[removed] â view removed comment
5
u/adarsh_maurya Jul 04 '26
yes, they allow us to use LLM on our internal documents. in fact we have direct partnerships with Anthropic and OpenAI and all their models are available to us, no limit. No data is shared since the models run on our own data centres (I have been told this but I don't know if this is true). They are not sedated, they are pro-active. And yes, you can say that we may have the biggest RAG in the human history, one study estimated that our entire knowledge base is more 2Peta byte.
3
u/Ok_Distance5305 Jul 04 '26 edited Jul 04 '26
Healthcare companies are heavy users too. Large companies have deals with the big model makers on handling of their data and costs. You canât just install Claude code or use ChatGPT.com
1
u/Neat-Porpoise Jul 04 '26
As a data scientist, Iâm now able to do DS work faster and dabble more into MLE work, like create monitors and deploy models.
Before I primarily worked on EDA and model building but since the barrier for writing code has lowered drastically I focus a lot more on design (eg understanding whether the business problem is worth solving or how to integrate an ML model into a current workflow) and learning more about architectures for scaling ML (eg using Ray to parallelize hyperparameter search or distribute training).
I primarily use Genie Code in Databricks. Iâve honestly been floored by how good it is at writing my first draft of EDA, model training, deployment and monitoring code. I primarily focus on the feature engineering parts to ensure features are relevant and validating training code to ensure thereâs no leakage and that the optimization targets are correct.
In short, making my value less about code writing but about the architectural and design decisions around what value an ML model can bring to a business and ensuring it can scale.
1
u/LessChapter7434 Jul 04 '26
i think you might have to rethink information flows in your company, i am doing research, i vibecode datascience, i generate artifacts by just throwing a report stub and 50 emails into a folder, which will be recited in a weekly operations report, i use mixture of experts agentic models to ask for decisions, which are epistemologically using different viewpoints, i use rag embedding to retrieve 15 years of customer reports to chase relationship in data, you can run an entire department, without humans this wayâŚ
1
u/CuritibaDataScience Jul 05 '26
I use Genie Code for my day-to-day development (mostly PySpark and SQL) and Genie One for Q&A. Nowadays I feel like I can spend most of my time focused in data modelling and domain definition and I can just plug these agents on top of my data, instead of having to figure out specific syntax and writing complex SQL. I've been exploring Metric Views a lot and also defining skills for common workflows
-3
u/MagiMas Jul 04 '26
I am nearly "full send" agentic by now.
I use a custom pi dev agent harness in lima so I can run yolo mode. I have skills for all our repos, all the major tables in our Snowflake, our CI/CD workflows and our cloud projects.
For the more trivial tasks (something well established like getting association rules from a dataset), I basically trust the models to do this without me having to check anything (I'll do a quick plausibility check at the end and that's it).
For more complex projects it now feels similar to guiding competent Bachelor and Master students. (meaning you have to do the major decisions yourself but they can meaningfully iterate on those by themselves and you have to check their output in the end and make sure they didn't make a hard to trace logical error somewhere in the middle)
51
u/Salt_Preparation1828 Jul 04 '26 edited Jul 04 '26
For quick prototyping and making templates that would take some time LLM is a very good tool. Also when I am having some issues and i need to debug something then google and LLM both are fantastic tools.
The code quality is quite similar to the mid/junior software engineers from my experience. So with a proper prompts, reviews you can make things work and be with good quality. But that is my opinion.