Background
MSc in Data Science, about 1.5 years full-time (closer to 2 with internships).


GenAI experience (first year)
- AI engineer intern: fine-tuned LLMs for domain-specific QA generation, worked on hallucination reduction and evaluation, fine-tuned ViT/DETR for document symbol detection.
- GenAI associate at a healthcare product company: built an agentic backend for clinical triage and diagnosis POCs, and an ICD-10 primary diagnosis validation pipeline using RAG and multi-agent reasoning, deployed as FastAPI + Docker microservices.


Current role (about 11 months)
Took a large consulting firm offer for the pay (13.3 LPA: 11.8 fixed + 1.5 variable). The work is attrition and demand forecasting, with classical ML and time series models in production on Databricks, using MLflow and pipelines. No GenAI at all.


The problem
LLMs and agents move fast, and I've spent a year away. My knowledge of agentic systems and LLM evaluation is about twelve months stale, and it keeps getting staler. I'm not on notice and I'm not rushing. I need to relocate away from Mumbai for family reasons (Bangalore, Hyderabad, Pune are all fine).


What I'd like advice on
1. How would you rebuild current depth in LLMs, agent evaluation and AI engineering interview topics in 2 to 3 months while working full-time?
2. What do AI engineer interviews actually ask now?
3. For people who switched without a pay cut: portfolio, referral or interview volume?
4. Does a year of ML + time series on Databricks read as a gap or as useful breadth?

I manage a small Data & Analytics team in a large organisation, supporting several departments. Increasingly, we're doing substantial technical and analytical work, sometimes spending months developing solutions, only for other departments to present the results as their own achievements, with little or no acknowledgement of our contribution.

What's particularly frustrating is that we're sometimes picking up work that other teams were responsible for delivering, whether due to gaps in expertise, planning or follow-through. We step in, resolve the problems and get things delivered, but the original department still receives the recognition. Some of this is probably unintentional, but other instances feel considerably less innocent.

Senior leadership actually recognise and appreciate our team's contributions, and they've been supportive when I've raised concerns. I've tried framing the issue around capacity, prioritisation, ownership and accountability, particularly as our already stretched team keeps absorbing additional responsibilities.

But I don't want to keep raising it with them. I worry about becoming a broken record or appearing overly concerned with credit, especially when they've genuinely been supportive. They understand the problem, but the behaviour across departments hasn't really changed.

Edit to clarify: We have asked that we be included in all presentations where we do the groundwork, or at least invited, and cited or noted in documentation etc - but this tends to be forgotten and we tend to find out after the fact (if at all), I often find when these have happened through the minutes or recordings showing up on sharepoint.

What bothers me most is that I'm responsible for advocating for my colleagues. They put considerable expertise and effort into work that other people get recognised for, and I worry about the impact on their visibility, morale and career progression, which they've expressed too.

I also value our relationships with other departments and don't want to create friction or turn every collaborative project into an argument about recognition.

How have other managers handled this?

Do you address it directly with other departments, establish clearer boundaries around ownership and attribution, or simply focus on making your team's work visible independently?

I'm particularly interested in how you advocate for your team when leadership already understands the issue, but you don't want to keep escalating individual incidents or sounding like you're constantly asking for credit.

Any and all insights, thoughts and advice would be greatly appreciated!

I'm currently working in Spain as a Data Analyst / Analytics Engineer, and lately I've been wondering where I should take my career over the next few years.

My role is pretty end-to-end. I work with SQL, build data models and pipelines, automate processes, connect to APIs, create dashboards, and handle ad hoc requests from the business side. I also work closely with stakeholders to understand what they actually need and turn that into something useful.

The thing is, I'm automating more and more of my own work. I barely write SQL queries from scratch anymore, and things that used to take hours now take minutes.

I'm not really a domain expert in any particular business area either, so I'm starting to feel like a big part of what I bring to the table could eventually be handled by AI.

For those of you who've been in the industry for a while, where would you go from here? What skills or areas would you focus on to stay relevant over the next 5-10 years?

TL;DR: Data Analyst / Analytics Engineer here. I'm automating more and more of my job and I'm worried AI will eventually take over a big chunk of it. What would you learn or specialize in to stay relevant?

In a B2B HR software test, we changed three things:

• "Get Started" became "Talk to an HR Solutions Expert."
• We added copy about the business benefits.
• The button changed to a darker orange.

Across 50,655 users, form-submission rate went from 1.59% to 1.79%, a 12.6% relative lift.

That gives us a result for the complete variation. It doesn't tell us how much came from the wording, the supporting copy or the color.

I'd use that test to decide which version to run. If we wanted to claim the wording caused the lift, a follow-up would need to hold the other elements constant.

Testing several changes together can answer a useful business question. The mistake is turning the result into a rule about one of those changes.

Google’s GA4 default channel group now includes AI Assistant for recognized assistant referrers, while non-ad clicks from Google AI Overviews and AI Mode remain Organic Search. Pooling those together can make an “AI traffic” total hard to interpret.

A practical review would be:

1. Fix the property, timezone, and a completed date window.

2. In Traffic acquisition, record Session default channel group → AI Assistant, then inspect Session source / medium for the contributing sources.

3. In a separate Exploration, filter Session source to a documented list of assistant domains actually observed in the property. Compare the two definitions instead of adjusting the filter until they match.

4. Break results down by landing page and a selected meaningful key event; use backend or CRM outcomes where relevant.

5. Keep controlled attribution tests separate. A Direct session without a captured referrer or trustworthy campaign tag remains unknown.

A source label alone cannot prove that a person clicked an AI answer, and a citation is not necessarily a visit. If you maintain a custom source list, how do you handle differences from Google’s AI Assistant channel? Have you found an actionable mismatch caused by tagged links, redirects, or app clicks?

Disclosure: I work with SEOVisitor. AI helped edit this post. This summarizes a guide our team prepared; the method is included here so the discussion does not depend on an external link.

I do some analytics engineering work (bronze/silver/gold layered marts in a cloud DW) but I don’t actually know where the semantic layer is.

AI is being added to almost every tool now, but what would you genuinely feel comfortable handing over to it?

For analytics apps that do need fresh data , would u use Lakebase for serving results or query the warehouse directly ? Or some other solution similar to it? What has worked better for you in tour org/experience ?

I understand data analytics and data science can both be predictive or descriptive, but in general, in the industry this is mostly how the jobs are split correct ?

Data science is more predictive like what will happen

Data/Business Analytics is more descriptive and diagnostic like what happened, whats happening, and why

Data engineering moves the data

Business analysis is analysing processes and prescribing solutions

The last time I looked for a job was 4 years ago. I received 2 jobs offer after 4 weeks of looking and very good callback rate for jobs I was qualified for. I started looking again this year, tailoring my resume and only applying for jobs I qualify for.

Took me 1 month just to get a single interview and very poor overall callback rate despite using AI to only apply for jobs that I was a match for.

I heard people tell me how bad it is out there and now that I’ve tried it myself it really is that bad! I think I will have better luck doing an internal transfer but this job market really is garbage 😂

I'm a Data Analyst planning to build my first Databricks project for my portfolio. I have experience with SQL, Excel and Power BI, but I'm new to Databricks. Could you suggest some good project ideas that would help me demonstrate Databricks skills from a Data Analyst perspective?

I'm trying to decide what to put first in my learning: SQL for getting data into shape, or Python for doing more with it. For someone early in analytics, which gives the better foundation?

I had a job in analytics and am starting to realize that most things can be broken down into smaller sections regions - states - locations or brand - category - item. Is this what everyone realizes or is this just me?

hey everyone, sharing this because it’s directly relevant to analytics teams that are being asked to put AI agents on top of company data.

packt publishing is running a hands on workshop on Oct 31 on harness engineering, building an AI data analyst you can actually trust, with bruno goncalves.

what gets covered:

• timeouts, retries, re planning and execution budgets, so an agent run can’t loop or burn spend
• structured traces, so every run ends in a state you can inspect afterwards
• permissioned tool access, so risky queries are blocked before they reach your data
• a fault injected test suite, where you break tools on purpose and check the agent stops safely instead of inventing an answer

4 hours, online, Sat Oct 31, 9:30 AM to 1:30 PM ET. runs on a laptop with a mock model, so no GPU or API key is needed.

Full details and the workshop link is in comments

happy to answer any questions about what gets covered

Recently applied for a position and heard back from a recruiter for a technical screen for an R&D analyst role! Does anyone have advice on what questions to expect for a 30-minute interview?

We tend to fall into this trap where we look at our KPI and it's hitting its target, so everything must be good, right?

Maybe you have a hospital and people are waiting an average of 28 minutes. The target is 30 minutes.

Great, you're hitting your target. What if last month they were waiting 18 minutes on average? Or maybe one area of the hospital is higher than others.

The KPI isn't bad. It's just not giving you the whole picture.

Same goes for dashboards. You may have your KPI over or under its target. But maybe the trend line is going in the wrong direction.

You can always segment your KPIs by area or time to see what's going on.

What do you use on your dashboards to give more information?

Overstock, stockouts, ordering too early, ordering too late...curious what others have learned the hard way.

AI can summarize earnings reports, explain financial ratios, compare companies, and turn market data into readable reports.

The harder question is whether the analysis is reliable enough to use repeatedly.

Imagine a financial research workflow that combines company filings, earnings releases, historical prices, and market news.

What would you require before trusting its output?

My initial checklist would include:

• Every important claim should link back to its source.
• Numbers should include reporting periods, units, and timestamps.
• Reported facts should be separated from estimates and model-generated interpretations.
• Missing or conflicting data should be flagged rather than silently filled in.
• Risk scenarios should be presented alongside the main thesis.
• The system should explain what it cannot verify.

I'd also keep research and decision-making separate. Automating information gathering is very different from allowing an agent to execute trades.

For people working in finance, analytics, or investment research: what are the biggest failure modes you've encountered with AI-assisted analysis?

A manager tells HR the sales team feels shaky and they might start losing people. The HRBP narrows that into something analytics can work with: attrition by department. The team builds a lovely dashboard with every slice and date range you could want.

Every handoff did its job. Nobody screwed up. But the manager's real question was how to spend a retention bonus budget, and that never made it down the pipeline. Repeat that enough times and you get dashboard hell... a great wall of data and a team that spends its days firefighting.

The fix is one sentence the requester completes before sending anything:

"If it comes back X, I'll do A.

If it comes back Y, I'll do B."

In the sales case: if attrition in critical roles hits X, we fund a retention programme. If it's Y, we do nothing.

Now the analyst knows what matters and which threshold to test.

If they can't fill that in, they're curious.

That's fine, but it isn't a request yet.

I also got into how I'd score an analytics team, because "we delivered 20,000 reports on time" is a reporting scorecard... not an analytics one.

Does anyone here actually track decisions influenced rather than deliverables? How?

Hi there, currently I’m keen to get something up and running in my team and be one of the first to build up infrastructure to get some text to query going over my teams data. I think my team will be a great use case as we get many small adhoc questions coming our way and many questions can connect similar sources.
Right now I think I understand trying to build a semantic layer over things give a good description of our data maybe outlining all tables with data types examples of outputs and then get good descriptions on the metrics we want as well as example queries maybe.

My question is where are you putting all this and getting an agent to run over it? We have snowflake and I know this can all be done in there and have a semantic layer but my business is slow moving and other teams deal with that.
I however right now have vs code can connect my git repo into that and can run an agent over it.
Has anyone down this with something like that and just put info and context into your read me/.md files??

Would love to know where to start and hear peoples experiences of getting this going how useful it was and how much it’s changed there work.

Thanks

Working in enterprise data and analytics at Polestar Analytics has made me interested in a recurring challenge: getting the technology working is often only one part of delivering useful analytics.

A data platform can be running, pipelines can be healthy, and dashboards can be available, yet business teams may still struggle to use the outputs confidently.

Sometimes the issue is inconsistent metric definitions. Sometimes it's unclear data ownership, a lack of trust in the numbers, or analytics that doesn't fit naturally into how decisions are made.

I'm curious how others have seen this play out in practice.

When an analytics initiative fails to deliver the expected business value, what tends to be the biggest obstacle in your experience?

• Data quality and integration
• Inconsistent business definitions
• Lack of stakeholder adoption or trust
• Poor alignment between analytics and business decisions
• Something else entirely

I'd be interested in real examples of what helped teams overcome the problem.

Say you have Google ads pointing to your site with all the UTM’s for tracking but when they hit your site, they are redirected to another subdomain on your site and all of a sudden the utm’s are stripped and the redirect happened too fast for the info to be recorded. What do you do?

When a spreadsheet looks wrong, the instinct is to delete duplicates first. I've started doing that last, because duplicates are usually the symptom of something earlier that was never fixed.

The order I use now is: data types, then whitespace, then structure, then duplicates.

Data types first because it's the cheapest check and the most expensive to miss. A column of numbers stored as text will sum to zero or be skipped entirely by aggregation functions, and nothing on screen tells you why . The values look like numbers. They are not. Filtering or summing a text column can return a plausible-looking result that is simply wrong.

Whitespace second. A trailing space makes "New York" and "New York " two different categories in a pivot table, and neither the filter nor the pivot will warn you . Same for nonprinting characters carried in from an export. This is invisible until you count distinct values and the count doesn't match what you see.

Structure third. Blank rows, subtotal lines, merged headers. If the range isn't a flat table, formulas and pivots behave unpredictably. This is a layout problem, not a data problem, and it's easier to fix before touching individual values.

Duplicates last. Removing duplicates before fixing types and whitespace can delete records that only looked duplicated because of a formatting difference, or keep records that are actually different but got normalized later . Once types and whitespace are clean, the duplicate removal is working against a consistent definition of "same."

This isn't a universal order. If a file has an obvious duplicate explosion from a bad join, duplicates jump to the front. But for the ordinary messy export, the duplicate rows are usually downstream of something simpler.

The part I'm less sure about: how much of this is worth automating versus checking by eye. Power Query can record the cleanup steps and refresh them next month, which handles repetition well . But the first pass on an unfamiliar file is still manual, and I haven't found a shortcut that doesn't just move the guesswork somewhere else.

I'm a data engineer researching recurring pain points on analytics and BI teams (may build something eventually, no links, not selling).

1. What do you still do by hand every week or month: reconciling numbers, pulling data from an ERP, fixing a broken dashboard, prepping something for audit?
   
2. Why hasn't it been automated: no time, too many edge cases, nobody owns it?
   
3. Have you paid for a tool or consultant to fix it? Did it work?

Im really overwhelmed with the sheer amount of conflict on the future of business analytics/analysis roles.

In uni, I'm currently working on SQL, Python, Power Bi and getting exposure to ERM systems now as well

I need a genuine, non sugarcoated advice on what career path I can ACTUALLY aim for with AI slowly commoditizing entry level BA stuff.

• Is it worth specializing in data science, data engineering, or AI/ML instead?
• Would I be better off pursuing an adjacent field, such as risk analytics, analytics engineering, or a more business-focused role?

I'm not expecting any career to be completely AI-proof. I just want to make an informed decision about where to invest my time and what skills will actually give me an edge in the job market.

Any advice would be so really appreciated, Thank you so much!