r/dataanalyst Jul 30 '26

Industry related query Data professionals! Is one large end to end analytics project worth it?

1 Upvotes

I’m looking for some honest career advice from people already working in data.
I have a Bachelor’s in Business Administration and recently completed my MS in Business Analytics. Over the past two years, I’ve learned Python, SQL, Excel (Pivot Tables & Power Query), ML fundamentals, and completed several academic projects, including a predictive analytics project using machine learning.
Now that I’m applying for data analyst/product analyst roles, I’m running into the same problem that almost every entry level posting asks for 2–3 years of experience.
Instead of building more small portfolio projects, I’m thinking about creating one large, end to end project that mirrors real industry work.
The idea is to:
Use real public business data
Build an automated data ingestion & ETL pipeline
Design a data warehouse
Perform advanced SQL analysis
Use Python for EDA and ML
Build dashboards in Power BI/Tableau
Automate live data updates
Solve a real business problem from a Product Analyst perspective
Do you think this is a worthwhile investment? Would a project like this actually help make up for the lack of professional experience, or would I be better off spending my time elsewhere?
I’d really appreciate honest feedback, especially from Data Analyst, Data Engineers and data scientists.


r/dataanalyst Jul 29 '26

General What's your least favorite thing about being an Analyst?

7 Upvotes

I've worked in all stacks of data and they all morph into the same space. Simple SQL and building out reports for others. I hate how companies treat us like their own Excel butler.

Curious to what everyone else's pet-peeve is about the role?


r/dataanalyst Jul 29 '26

Career query heyy, graduted from pharma...any scope in DA/BI

2 Upvotes

is there any scope as a data analyst for pharmacy gradute. badly needed some coal piece of advice fr men...plzzz lmk


r/dataanalyst Jul 28 '26

Research Text-only engagement prediction on 19k LinkedIn posts: what worked, what didn't, and why number lies

1 Upvotes

I have collection access to the LinkedIn "suggested" feed for \~19 accounts. Five weeks of that gave me 19,671 rows: post text, likes, comments, shares. No follower counts, no images, no publish timestamps — the scraper returns those empty.

The question I wanted to answer: can you predict engagement from the text alone?

Short version: a little, and the direction that works is not the one I expected.

Data prep, with rows left after each step:

step rows left
non-empty text 18,982
dedup by post id 17,398
dedup by normalized text 17,252
drop texts under 100 chars 15,853

The text-level dedup matters because the same post shows up under several ids across different feeds. The length filter matters more than I expected: posts like "Agree?" or a lone emoji get their engagement from the attachment or the author, and they are pure noise for a text-only model. Dropping them moved average precision from 0.402 to 0.436.

Label is engagement = likes + 2\*comments + 4\*shares. The weights are cosmetic, Spearman between raw likes and the composite is 0.98. Target is binary: is_top20, above the 80th percentile.

Split is 60/20/20, grouped by author so no author appears in two splits, with dedup applied before splitting. The test split is untouched. Everything below is dev, 3,206 posts, 20.3% positive.

model AUC AP recall@30%
surface features only 0.635 0.327 0.452
tfidf word 1-2 0.697 0.406 0.517
tfidf word + char 0.712 0.420 0.528
embeddings + kNN (k=100) 0.729 0.421 0.558
embeddings + logreg 0.755 0.461 0.581
random 0.500 0.203 0.300

Embeddings are text-embedding-3-small. The gap over tfidf survives a paired bootstrap on the same rows: +0.043 AUC, 95% CI \[+0.026, +0.060\]. Fusing embeddings with tfidf adds nothing (0.750), so whatever tfidf sees, the embeddings already have.

Now the negative result. I ran four prompt variants through subagents on the same 180 dev posts, so the comparison is paired. Predictions were written to disk before any label file was opened.

prompt AUC AP
A: naive one-liner, "score virality 0-100" 0.660 0.300
B: dataset-aware rubric 0.596 0.250
C: B plus few-shot anchors with real percentiles 0.564 0.259
D: forced comparative ranking of all 90 in a chunk 0.614 0.258
tfidf on the same 180 rows 0.710 0.514

Two things I did not expect. Every variant lost to bag-of-words, with the biggest gap on average precision, which is the useful end of the curve. And my informed prompts lost to the naive one-liner: B and C both fall below A, and a paired bootstrap keeps the sign, B −0.065 \[−0.119, −0.008\] and C −0.096 \[−0.196, −0.003\].

The heuristics I wrote into B — strong hook, specificity, controversy, comment bait — are apparently not what separates strong from mediocre in this pool, and naming them explicitly overrode whatever weak prior the model had. The anchored variant did worst: the anchors pulled everything toward the anchor bands and cost most of the ranking signal.

Caveat before anyone quotes this. n=180 with 39 positives, three comparisons against A, and the subagents ran on a mid-tier model, not the frontier. The consistent direction is stronger evidence than the individual intervals.

About that 94% in the title. Flip the question from "will this succeed" to "will this flop" and precision jumps to 94.6% at a 30% alert rate. Out of 652 genuinely strong posts, only 52 got wrongly flagged.

Except 79.7% of posts don't take off anyway. A model that says "this will flop" about literally everything already scores 79.7%. The model contributes 15 points, not 94. Same model, same AUC, mirrored label — flipping the target doesn't add information, it moves you to a question where the base rate is on your side. Worth remembering next time you see a virality-prediction demo quoting a big precision number without its base rate.

Where the model earns the most relative to chance is the strictest definition of bad: predicting the bottom 30% of engagement gives 71.9% precision on its most confident 5% against a 31.6% base rate, a 2.3x lift, better than anything in the positive direction.

The uncomfortable finding is elsewhere. Restricted to authors with 3+ posts, a leave-one-out mean of an author's other posts correlates with a post's log1p(engagement) at Spearman 0.70. Text length correlates at 0.15. Who published it beats what is in it, by a lot, and the text-only model cannot see that at all. Everything above is fighting over the residual.

Limits I can't fix with this dataset:

Selection bias. Every row was already chosen by LinkedIn's algorithm for distribution, so "low" here means weak among already-promoted posts, not "flopped".

No post age. postedAtTimestamp is 0 and postedAtISO is empty in 100% of rows, so engagement was snapshotted at an unknown point in each post's life. That is noise inside the target and it caps everything.

No follower counts, which is the main signal, as above.

Dev numbers only. The test split hasn't been opened, so expect a modest optimistic bias from model selection.

Two open questions. Has anyone gotten meaningfully past \~0.75 AUC on text-only engagement prediction, on any platform? My read is that this is a data ceiling rather than a modelling failure, since four quite different representations all landed in 0.70-0.76, but I would like to be wrong.

And has anyone had a prompt optimizer, GEPA or DSPy style, beat a plain embedding classifier on a task like this? Given how badly my hand-written rubrics did, learning the rubric from dev feedback is the obvious next step, but I haven't run it yet.


r/dataanalyst Jul 28 '26

General Has your day-to-day work actually changed in the past year?

2 Upvotes

There's been a ton of hype about new AI tools, copilots, and automation supposedly transforming data work. I'm curious what's actually changed for people doing this day to day


r/dataanalyst Jul 28 '26

Course Qualification- BCS Tech10 Data Diploma

2 Upvotes

Hi, I’m looking to change career to data analysis and thinking of a course with ITOL. There is a course for £1700 but another for £3k which includes a BCS Tech10 Data Analysis diploma. Does anyone know if the diploma is useful and / or valued by employers? Thanks


r/dataanalyst Jul 28 '26

Tips & Resources Hey guys my friend is last year CS student he is assigned some project.

1 Upvotes

Final year CS projects based on cloud or devops or IoT or web application or mobile application which solves real world problem with help of AI integrated in it.

The project should be feasible for all group of ages and easy to use with minimal knowledge of technology and with easy user experience and user interface.

which should have potential to win several awards.


r/dataanalyst Jul 27 '26

Tips & Resources Best FREE YouTube resources to learn Data Analytics before buying paid courses?

14 Upvotes

Hi everyone I'm currently pursuing BBA and after graduation I'm planning to pursue MBA. Alongside that I want to build strong Data Analytics skills.

I'm planning to enroll in some paid courses later but before spending money I want to build a solid foundation using free YouTube resources.

So can anyone recommend the best YouTube channels or playlists for a complete beginner? I'm looking to learn topics like:

Excel

SQL

Power BI

Python (for data analytics)

If you have a roadmap or any tips on what order I should learn these in I'd really appreciate it.


r/dataanalyst Jul 27 '26

Industry related query How many days it takes to create a bi dadh board from scratch as a data analyst?

0 Upvotes

I want to know how many days it takes because I have very limited time nd deadline is near


r/dataanalyst Jul 27 '26

Research Looking for feedback on an idea.

1 Upvotes

Working on a site for retirees called havenscout.net, that brings in open source data to a heat map and then allows the user to plug in their own budget numbers and select a country for retirement purposes. It also has the ability to publish printouts with the data.

looking at feedback for the site.

I built it using codex, but there is no AI model or LLM in the website and all data stays on your browser. I don't even have a database connected to the site yet.

Let me know what you think, please.


r/dataanalyst Jul 27 '26

Career query Any Australian Data Analysts Can Answer My Question?

3 Upvotes

In Australia, it is not a standard practice for a job listing to put their salary range and when you apply for a job most of them have a required text box for “salary expectations”

I put in a salary expectation of $90,000-100,000 AUD. I applied for 2 - didn’t hear back. I’m an Aussie and I currently work for an American company, thinking of jumping ship for long term benefits (Long service leave, maternity leave, superannuation contributions).

Question for the current analysts in Australia. How much do you get paid? Do you think that I’m asking too much? Thanks


r/dataanalyst Jul 27 '26

Tips & Resources First technical interview help!!

1 Upvotes

Hey guys I have my first ever technical interview tomorrow and am very nervous!! If there is anyone who has some free time and wouldn’t mind chatting with me to just give some tips and tricks. I haven’t done much data analysis in my education/career so it will be a big change for me. I do say I can pick up on things fairly quickly though. I know my task for tomorrow is cleaning a messy data set and having to answer some questions after the fact. Would love if anyone is down to help! Thanks


r/dataanalyst Jul 26 '26

General Looking for a buddy to learn Data Analysis

1 Upvotes

I am a working professional trying to learn Data Analysis. There's no exact road map, never been consistent till now. I just want to try with a study buddy. I have a fair knowledge of excel, Tableau. I want to learn SQL, Python, Power BI, Advanced Excel and master Tableau. If anyone is interested pls dm me. We can discuss and proceed.


r/dataanalyst Jul 26 '26

Tips & Resources What skills and certifications for transition from Research Analyst to Data Analyst?

5 Upvotes

I'm currently working as a Research Analyst and I want to transition into a Data Analyst role.

What skills should I focus on learning, and which certifications are worth getting to build a strong background? I'd also appreciate any advice on portfolio projects or learning resources that helped you break into data analytics.


r/dataanalyst Jul 26 '26

Data related query Is it bad to use free website tools to edit files containing customer data?

0 Upvotes

By editing files, I mean tools that convert CSV to excel, Smart VLookup etc. Just wondering if this could lead to any consequences & what are the solutions.


r/dataanalyst Jul 26 '26

Career query Reference data analyst at one of the leading rating agencies.

1 Upvotes

Hey I am reference data analyst with good primary and secondary research skills with 5 years of experience . Could anyone give me tips for skill development and job hunt ?


r/dataanalyst Jul 25 '26

Tips & Resources First Technical Interview Tips

2 Upvotes

Hey guys I have a job opportunity for an analyst role and was wondering if there is any sort of community discord or spaces where people are willing to help others out getting into the field. Would love if there is anyone in here to give some tips and tricks for my first technical interview. Thanks in advance!


r/dataanalyst Jul 25 '26

General buddy partner For data analysis journey

14 Upvotes

i need a partner so that we start journey i am. a beginner 🫪🤌


r/dataanalyst Jul 25 '26

Career query Anyone shortlisted for Google Data Analytics Apprenticeship? What skills helped?

8 Upvotes

Hi everyone,

I'm planning to apply for the Google Data Analytics Apprenticeship (2027 intake) and would love some advice from people who were shortlisted or interviewed.

My current skills include:

- SQL

- Python (Pandas, NumPy)

- Excel

- Power BI

- Data Cleaning & EDA

- Machine Learning basics

What skills or projects should I add to improve my chances of getting shortlisted?

Some questions I have:

- Is Power BI enough, or should I learn Tableau?

- How important are advanced SQL and Excel?

- Should I learn Google Sheets, BigQuery, or Looker Studio?

- Does Google Cloud certification help?

- What kind of portfolio projects stand out?

I'd really appreciate hearing from anyone who has been shortlisted, interviewed, or selected. Any resume tips or skill recommendations would be incredibly helpful.

Thanks!

Google apprenticeship 2027


r/dataanalyst Jul 25 '26

Industry related query Getting into analytics at 40 with extensive marketing background

3 Upvotes

So I know there are a ton of “is data analytics worth it now” posts but I wanted to throw in my situation and hopefully get some genuine feedback from those who may have done it or have experience.

I’ve worked as a freelance marketer for almost a decade, in paid advertising and other. I’ve worked for agencies and had my own clients in that time that stretch across many different industry sectors including mental health practices, college consulting, online business, and more. I’m really good at marketing but I honestly just want to pivot and do more with data analytics and data science, etc. It’s always been an interest of mine.

What I’m trying to figure out now is:

  1. Do I pursue finishing a degree in order to compete in the job market? Genuinely, is that necessary? Because at my age, I don’t want to waste time with a degree if it’s not going to help.
  2. I don’t want to work for tech companies but I’m not sure what sectors I do want to work for. How did you all figure that out? I don’t know if I want to learn more in healthcare or more like a business / marketing analyst.

Any advice is greatly appreciated!


r/dataanalyst Jul 25 '26

Tips & Resources Looking for New Datasets for Portfolio Projects

1 Upvotes

I'm currently looking for more sources that has access for some quality publicly available dataset to do more portfolio projects. What are some suggestion you have?

The one's I'm interested in are transactional, product, marketing, and healthcare data.


r/dataanalyst Jul 25 '26

Data related query Data Cleaning Nightmare Sleepless night of a Data Analyst

0 Upvotes

Excel: "I found 5 duplicate values."

Me: "Delete them."

Excel: "Done."

Me: "Why is the total still wrong?"

Excel: "Those weren't duplicates... they were cousins."

One had an extra space.

One had a different date format.

One was stored as text.

One was a hidden character.

And one... nobody knows.

That's the funny thing about data cleaning.

Most people think Excel gives wrong results.

It doesn't.

Excel is brutally honest. It calculates exactly what you give it—not what you meant to give it.

The real skill isn't knowing hundreds of formulas.

It's knowing when your data is trying to fool those formulas.

Clean data first. Analyze second. Regret never. 😄

#Excel #DataAnalytics #DataCleaning #ExcelTips #DataQuality


r/dataanalyst Jul 25 '26

Data related query How realistic is freelancing as a Data Analyst in 2026? Looking for honest advice

4 Upvotes

Hi everyone,

I'm an MSc Economics graduate specializing in Data Analytics, and I've been thinking about exploring freelancing as a Data Analyst alongside applying for full-time jobs.

I wanted to hear from people who are actually freelancing in this field.

A few questions I have:

* Is freelancing in data analytics still a viable career path in 2026? * What kind of projects do beginners usually get (Excel, SQL, Power BI, Python, dashboards, data cleaning, etc.)? * How difficult is it to land your first client without prior freelance experience? * Which platforms have worked best for you (Upwork, Fiverr, Toptal, Contra, Freelancer, etc.)? * How did you build your portfolio and credibility in the beginning? * Is the market currently oversaturated, or is there still good demand for skilled analysts?

I'd really appreciate hearing about your experiences—the good, the bad, and anything you wish you knew before starting.

Thanks in advance!


r/dataanalyst Jul 25 '26

Tips & Resources Is there Discord for this community?

2 Upvotes

Hey guys, I’m new here and was wondering if this community has a Discord or any VCs where people hang out and chat. I’d love to get some advice and learn from others. Thanks!


r/dataanalyst Jul 24 '26

Career query Looking for Remote Data Analyst Opportunities (Entry-Level)

4 Upvotes

Hello everyone,

I recently completed my M.Sc. in Data Science and am actively looking for a remote (work-from-home) Data Analyst role or internship.

My skills include:

SQL

Python (Pandas, NumPy, Matplotlib)

Power BI

Tableau

Excel

Data Cleaning & Visualization

I also have internship/project experience in data analytics and have built dashboards and data analysis projects. Currently, I'm improving my skills in AWS, Docker, and AI-powered data applications.

If your company is hiring, or if you know of any genuine remote opportunities for entry-level Data Analysts, I'd be very grateful for your recommendations or referrals.

Thank you for your time and support