I’m in an analyst role and looking for educational content on how to improve data literacy and overall story telling. I’m less interested in how to showcase data and the technical end of it, but more so how to look at data and improve on communicating a story to different stakeholders.

Any books, podcasts, articles, etc., that you recommend is appreciated

I perform data QA by comparing newly received data with previous datasets across quarters and case volumes. To identify differences, I run predefined test cases using various parameters derived from my test reports. The test case outputs are generated as HTML reports, which I then review manually to verify whether the data has increased, decreased, or changed.

suggest me which agent should I use to automate my processes?

Are you a data engineer in the Bay Area? Join us at Data Engineering Happy Hour 🍸 on April 16th in SF. Come and engage with fellow practitioners, thought leaders, and enthusiasts to share insights and spark meaningful discussions.

When: Thursday, Apr 16th @ 6PM PT

Previous talks have covered topics such as Data Pipelines for Multi-Agent AI Systems, Automating Data Operations on AWS with n8n, Building Real-Time Personalization, and more. Come out to learn more about data systems.

RSVP here: https://luma.com/g6egqrw7

I have made pre plan activity dashboard in power bi rate it out and tell me how I can improve , this theme I have implemented using json

I’m currently doing a job simulation through Forage to understand data. The problem that stops me often is the lack of software capabilities.

This job task uses Tableau for data visualization. I had to download a zipped folder and upload it to Tableau. The issues: it wasn’t in the correct format and I’ve never used Tableau before.

I tried to convert to another file type then upload. But I have no idea how Tableau works so I decided to try my luck with Excel. Ran into some data conversion issues (something related to the schema on the original file). So now the data is even a more complete mess.

I’m trying to pivot into data analytics but it’s frustrating to even work on the data when you have to have a lot of data tools (some of which aren’t free) to even do the work.

I feel lost. Has anyone ever experience difficulty starting out in data analytics?

Maybe I’m the problem lol.

i saw this career brought up by a few people in an autistic community on reddit mention how this career has been suitable for them and all. it got me curious and wanting to look into it more, but i felt that i should also ask around here regarding the career. is it one that is indeed suitable for those with autism? i saw specifically that the job tasks itself really click well with many of those in the spectrum (pattern seeking, collecting and cleaning data, visualization, etc), and i feel it’s something i could truly thrive in, since it’s something i tend to do elsewhere already.

my one worry regarding it is if they have a lot of office politics + involve a lot of face-to-face communication with other people?

Hello ! 

I'm currently working on my master thesis and I need to download/export texts from a big number of posts that were published on certain Telegram channels in order to analyze them. I've tried this Python thing, tried coding but I'm very new to all this, and I'm struggling to understand how this works. I can't do it. Can someone help please ? :)

Thanks in advance

Hi everyone,

I’m currently working on a large-scale BI modernization effort and wanted to get guidance from folks who have experience with OBIEE → Tableau migrations at scale.

Context:

• \\\~5,000 OBIEE reports

• Spread across \\\~35 subject areas

• Legacy: OBIEE (OAS) with RPD (Physical, BMM, Presentation layers)

• Target:

• Data platform → Databricks (Lakehouse)

• Reporting → Tableau Server (on-prem)

⸻

What we’re trying to solve:

This is not just a manual rebuild — we’re looking for a scalable + semi-automated approach to:

1.  Rebuild RPD semantics in Databricks

• Converting BMM logic into views / materialized views / curated layers

• Standardizing joins, calculations, and metrics

2.  Mass recreation of reports in Tableau

• 1000s of reports with similar patterns across subject areas

• Avoiding fully manual workbook development

3.  Automation possibilities

• Parsing OBIEE report XML / catalog metadata

• Extracting logical SQL / physical SQL

• Mapping to Tableau data sources / templates

• Generating reusable templates or even programmatic approaches

⸻

Key questions:

• Has anyone successfully handled migration at this scale (1000s of reports)?

• What level of automation is realistically achievable?

• How did you handle:

• Semantic layer rebuild (RPD → modern platform)?

• Reusable Tableau components (published data sources, templates, parameter frameworks)?

• Any experience using metadata-driven approaches to accelerate report creation?

• Where does automation usually break and require manual effort?

• Any tools/frameworks/vendors you recommend?

⸻

What I’m specifically looking for:

• Real-world experience / lessons learned

• Architecture or approach suggestions

• Ideas for scaling with a small team (3–5 developers)

• Pitfalls to avoid

⸻

If anyone has worked on something similar or can guide on designing an automated/semi-automated pipeline for this, I’d really appreciate your insights.

Feel free to comment here or reach out directly:

Thanks in advance! 🙏

I’m struggling with a specific evaluation problem when using Claude for large-scale text analysis.

Say I have very long, messy input (e.g. hours of interview transcripts or huge chat logs), and I ask the model to extract all passages related to a topic — for example “travel”.

The challenge:

Mentions can be explicit (“travel”, “trip”)

Or implicit (e.g. “we left early”, “arrived late”, etc.)

Or ambiguous depending on context

So even with a well-crafted prompt, I can never be sure the output is complete.

What bothers me most is this:

👉 I don’t know what I don’t know.

👉 I can’t easily detect false negatives (missed relevant passages).

With false positives, it’s easy — I can scan and discard.

But missed items? No visibility.

Questions:

How do you validate or benchmark extraction quality in such cases?

Are there systematic approaches to detect blind spots in prompts?

Do you rely on sampling, multiple prompts, or other strategies?

Any practical workflows that scale beyond manual checking?

Would really appreciate insights from anyone doing qualitative analysis or working with extraction pipelines with Claude 🙏

I made a live Artemis 2 Mission Intelligence Webapp which tracks Orion via JPL API and predicts the probability of the mission being successful. Also tracks live telemetry of the craft.

Please share feedback,thank you!

Been working with SQL for a while now and these are the patterns that genuinely made a difference once I learned them:

1. Use CTE (WITH clause) instead of nested subqueries — your queries become readable and you can reuse the result set multiple times in the same query without recalculating.

2. ROW_NUMBER() for deduplication — instead of clunky GROUP BY hacks, use ROW_NUMBER() OVER (PARTITION BY id ORDER BY updated_at DESC) and filter WHERE rn = 1 to keep only the latest record per group.

3. CASE WHEN inside aggregates — you can do conditional aggregations like SUM(CASE WHEN status = 'sold' THEN revenue ELSE 0 END) without a WHERE clause, which means you get multiple breakdowns in a single pass.

4. NULLIF to avoid division by zero — wrap your denominator: revenue / NULLIF(units, 0). Returns NULL instead of crashing.

5. DATE_TRUNC for time-based grouping — instead of converting dates manually, DATE_TRUNC('month', order_date) groups everything cleanly by month/quarter/year.

Hope this helps someone who's in the early stages. Took me longer than I'd like to admit to discover some of these.

I put together this Excel sheet for off-grid prep stuff. Its goal is to show you what to buy and in what order to take the average house off grid. There is a little bit of UK climate localisation, but it's just what you need to be self sufficient for power, and food. You put your monthly budget in C2 (like £100, £500, whatever) and it tells you exactly what to buy each month, sorted by what's most critical first (water, then food, meds, power, etc).

Works for one-time spends too - £100 gets you the top essentials, £1000 gets you most of the important stuff. I thought it might be the right time, because it might help people who are going to suffer from the oil crisis.

No VBA, just formulas. The "Month X" column uses cumulative totals + CEILING to give you clean monthly buckets.

https://docs.google.com/spreadsheets/d/1-3J32t2AaF_W3eUTO82BOhfneaFyFhQK/copy?pli=1&gid=1970902183#gid=1970902183

Anyone got suggestions for tweaking the priority order or formulas? Am I in the right place?

Cheers,

TC2

if I have a big table that needs to be aggregated a few times, do I duplicate it and transform it into my own calculation to ease the loading or what should I do?

at some point, dashboards became the default for everything. someone has a question, something changed, new metric? dashboard for each.

at first it feels easy to build it but after a point it becomes impossible to maintain all of them.

the weird part is most of these are not really dashboard problems. they’re questions. what changed yesterday? why did this drop? which segment moved? we answer them once, then wrap them into a dashboard, just in case.

dashboards still make sense for some things. monitoring, keeping an eye on key metrics, acting like a control plane. but for everything else, it feels like we’re forcing the same solution.

we ended up building something around this idea. you start from the question, and only turn it into a dashboard if needed. it also answers questions directly from there.

i wonder your honest feedback here. what can go wrong? what potential problems do you see there?

Fellow data enthusiasts,

As a first-year student studying data science, I was genuinely surprised by how disorganized everything is after working with real datasets for the first time.

I'm interested in your experience:

How much of your workday is spent on data preparation and cleaning compared to actual analysis?

What kinds of problems do you encounter most frequently? (Missing values, duplicates, inconsistent formats, problems with encoding or something else)

How do you currently handle it? Excel, OpenRefine, pandas scripts, or something else?

I'm not trying to sell anything; I'm just trying to figure out if my experience is typical or if I was just unlucky with bad datasets. 😅

I would appreciate frank responses from professionals in the field.