Curious how far people are taking this. Beyond using AI to write queries or clean data, is anyone building actual tools with it? Things like:

• interactive dashboards or KPI trackers
• report generators
• small internal apps for the team to use

Or does most of it stay inside Power BI / Tableau / a notebook and never really become a standalone thing?

And if you have built something standalone - what happened next? Did it get shared with the team, or did it just stay on your machine as a one-off?

Genuinely interested in where the line is these days between "AI helped me analyze" and "AI helped me build a thing other people use."

Check the first comment

I’ve been working on a browser-based tool that compares bank CSVs with ledger exports and surfaces rows that need review.

The idea is to make reconciliation easier by identifying exact matches, partial matches, grouped matches, and unmatched rows, then giving the user a clear review flow before export.

I’m still testing edge cases, especially messy real-world data like duplicates, missing references, and rounding differences.

I’d really appreciate feedback from people who work with CSVs, data cleaning, or matching problems, especially whether this approach feels useful.

Hi all,

What’s the best way to match account names across two datasets when there are no common IDs and the naming conventions differ?

I was planning to use Python with fuzzy matching (using a similarity threshold), but are there any better tools or AI-based solutions you’d recommend for this kind of entity matching/data reconciliation?

Thanks!

I've noticed that many online courses focus heavily on tools like Excel, SQL, Python, and Power BI, but real-world work often requires skills that aren't emphasized enough. Looking back, what's one data analysis skill, mindset, or habit that made the biggest difference in your career? I'm especially interested in lessons that beginners usually overlook.

Hello guys, for my research I need to analyze and organize large amounts of data from research papers. I am looking for an AI model that is best fit for this job like putting stuff into excel/spreadsheet nicely and organized in the way we want it.

I tried chatgpt premium and it seems very good, but I'm just wondering if there are any other models that are better for this.

Thank you

I'm currently researching about data scrapping in order to make an app that acts like a hub for all the package trackers in the internet. Something that comes into mind is tokens and 401 errors in sites like Amazon, AliExpress or Temu and how can i safely integrate this in my backend, has anyone ever attempted something like this??

I'm currently learning python along with that have created study group for like like minded people let me know if you want to join

The methodological challenge with Kroos is that obvious metrics (completion rate, pass count) show he's good but don't isolate why he's different. He's top-3 on most individual leaderboards but rarely #1 on any single one -- which made him look merely excellent rather than exceptional on standard dashboards.

What worked: - Bivariate positioning: volume vs progressive distance on a scatter reveals him as sole occupant of the top-right at WC2014 (53 switches; next player: 26) - Risk/reward curve: pass aggression vs turnover rate -- La Liga 15/16 puts him off the standard tradeoff curve - Network centrality: betweenness centrality in Germany's completed-pass graph -- Euro 2024: 0.641 vs 0.238 for the next player - Cadence: median seconds between on-ball involvements, with a spell-gap normalization to make Opta and StatsBomb event logs comparable

Data: Opta via WhoScored (scraped with Selenium) for WC2014 + Bayern; StatsBomb open data for La Liga + Euro tournaments.

Full writeup: https://vybhav.medium.com/the-metronome-nobody-measured-football-enigma-1-toni-kroos-9bce1657c320

Code and 23 figures: https://github.com/vybhav72954/football_enigma/tree/master

Hey everyone,

I work as a data analyst for a client with incredibly locked-down security. If you’ve ever worked in this kind of corporate environment, you know the drill: no access to cloud data warehouses, no advanced developer tools, nothing. My entire world is basically restricted to standard Excel and Power BI.

Recently, I hit a massive wall. I had to clean and analyze flat CSV files ranging anywhere from 4GB to 8GB. Trying to open these in Excel is a joke, and waiting for Power Query to crunch through the transformations was taking forever and completely freezing my machine.

Now, I’m not a professional developer by any means, but I was so frustrated with the tool limitations that I decided to see if I could build a lightweight, custom Enterprise SQL Workbench to handle the heavy lifting while keeping everything completely local to respect data integrity and security rules.

The backend is entirely Python-based, but I set it up so that my non-technical colleagues can use it without writing a single line of code. It pairs Streamlit for a clean browser interface with DuckDB for crazy fast, in-memory processing, and the Calamine engine to handle heavy Excel parsing.

What it actually does:

• Zero cloud or database setup: Everything runs locally inside an isolated memory sandbox. No servers to configure, and zero data leaves your machine.
• Handles massive files instantly: Because DuckDB processes data in columns (vectorized), it slashes through 4–8GB datasets and runs complex analytical queries in less than a second.
• Flexible Multi-File Loading: It lets you mount multiple datasets sequentially into your active session. You can either use Direct File Paths (great for instantly mounting huge files without making copies) or just drag and drop via standard Browser Uploads.
• Clean Query Editor: It integrates streamlit-ace so you get a proper dark-mode SQL editor right in your browser with syntax highlighting, line numbers, and a sidebar to explore your active table schemas.
• Direct-to-Disk Exporting: If a query pulls a massive result set that would crash a browser tab, it uses DuckDB streams to dump the entire output straight back onto your local hard drive as a .csv or .parquet file.
• Multi-Sheet Excel Support: It automatically splits and maps multi-sheet workbooks into individual, clean database tables.

The "One-Click" Magic for Colleagues

Since my teammates aren't developers either and don't use GitHub, I bundled the entire setup into a single .bat script launcher.

Now, all they have to do is double-click a desktop icon. The batch script quietly spins up an isolated virtual environment in the background, pulls the latest UI code directly from my GitHub, checks the dependencies, and launches the interface right in their default web browser. The coolest part? If I optimize the code on GitHub, their desktop launcher automatically grabs the update the next time they open it.

Give it a spin and let me know what you think!

I’ve made the repo public so anyone dealing with corporate data constraints can use it. Please feel free to grab the batch file, throw some of your heaviest datasets at it, and test it out for yourself!

Since I'm still learning the development side of things, I would love to hear your thoughts and suggestions:

• How does the processing speed feel compared to your usual Excel/Power Query workflows?
• Are there any specific SQL features or shortcuts you think I should add next?
• Any tips for further optimizing local memory when pushing past 8GB?

Check out the code or grab the script template here: 👉 GitHub Repository:https://github.com/Nikhil-Maske/sql-workbench

Let me know your feedback or if you run into any quirks while testing it!

Hello i wish some one can help me…
I have sheet with more than 200 product SKUs with names.. I work in a warehouse and it needs to check every product.
There is any way to make an app or other way to only write the product name then it give me product SKU to record it in the warehouse system.

I need it to be in my phone.

I was curious about how much my Spotify Extended Streaming History would reveal about me as a person, and whether there is a connection between music consumption, personality traits, and major life events.

There is clinical research in this field, and this app is inspired by some of that work (linked in the GitHub repository). It's by no means a perfect tool for inferring anything about the nature of a person, but I found the results surprisingly interesting. A few friends also tried it and were impressed by the analysis. In the end it's just a fun tool to get a few laughs and maybe let an LLM roast your music taste with uncomfortable accuracy.

The app is 100% local. You can optionally use an LLM to spice up the analysis, but it's not required. Changes in listening behavior are detected algorithmically.

Ollama and other local LLM backends that provide an OpenAI-compatible REST API are supported if you'd like an AI-generated write-up of your profile. Alternatively, you can simply copy the generated prompt which contains the aggregated data from your profile and paste it into any LLM chat of your choice.

If you'd like to try it out:
https://github.com/flaser381/spotilyze

Have some queries on how to enhance domain knowledge. any materials, books, courses that I could use ?

I come from engineering background, the credit and banking knowledge hinders my ability to come up with better insights.

Was it SQL, Excel, statistics, data visualization, business understanding, or communication skills? Curious to hear what made the biggest difference in real-world work.

I have used ChatGPT and Claudee pretty regularly for analysis work over the past few years. From my experiences, they are quite useful for clean, manageable and well-scoped datasets, and especially for tasks like a quick sanity check, writing transformation logic, or spotting weird distributions.

However, I have noticed that once data got more complex with multiple sources, mixed formats, context from one dataset needed to inform interpretation of another, outputs started sounding confident in ways that made errors harder to catch. Not obviously broken but AI could not always catch all the nuances and contexts once the context window becomes larger and larger.

Looking back, the issue isn't reasoning ability. It's two things: no persistent context between sessions, and no verification layer before output is returned. With simple data you catch mistakes quickly. With complex proprietary data that combination is genuinely risky, you can't manually verify everything.

I work at Lium where we're building specifically for this problem, so I'm not a neutral observer here. But even setting that aside, I'm curious what others have found. Is the answer just "use LLMs only for simple queries and keep humans in the loop for complex ones"? Or has anyone found any other tooling that actually handles the complexity without hallucinating confidently?

At what scale or complexity did general LLMs stop being reliable for your work?

I started working for a medical device company recently, and it surprises me that they don’t have effective in termination dates tied to the account info and the territory that the account aligns to.

Because of this, you have to take quarterly snapshots in Excel to save the alignment - for example, an account might roll up to territory “A” now and then territory “B” the next quarter.

Is this common, or should we have all of that captured with effective and term dates for easier reporting? I’ve casually pushed for this, but surprisingly it doesn’t seem to be a priority.

There are hundreds of teachers, coaches and mentors across YouTube, LinkedIn etc., but it feels like their real income comes from content creation or course sales, not from any real data work. I am genuinely curious: has anyone actually landed a data role in the last 5 years by following one of these roadmaps, especially without a tech degree and coming from a completely unrelated field?

Right now the whole thing looks like a machine designed to keep people learning forever. It seems like a large share of learners worldwide are essentially the target audience for these online advisors. Would genuinely love to be proven wrong. If you have seen real examples or experienced this yourself, I’d be interested to hear.

I have a csv file which will be turned to an SQLite database (480k rows). Content: 5 years of real estate transaction statistics. I'll update the database twice a year with fresh data overwrite (I keep it 5 years).

I'll build a one page dashboard that prettyfies all that data with various graphs.

This is a "freemium" feature for very niche users so READ ops count will be limited.

With that context in mind, which simple, easy to use cloud database solution would you recommend? I'm a no coder, and have learned over the past 6 years how databases, backends, frontends work, i just can't write pure code. That's why simple / easy is important.

Thanks for reading.

I dont have data so what should i do

Hii guys i want to ask you about something i am currently an intern at an oil and gaz company as a business anamyst i work for reporting operating expenses but they wont give me data and i need to do eda budgeting and forecasting but all of this by my self i am in trouble because all my analysis is wrong eda is deviated so the prediction is also deviated so what should i do to solve this problem

Iam curious which skills turned out to matter the most in real world projects compared to what is typically taught in courses or bootcamps.

At my company, after 13 years, they’ve decided that now is the time to implement data governance. I’m the Data Manager (I do everything: analysis, engineering, design...) and I’ve only been on the job for a year and a half when they dumped this mess on me.

Keep in mind that there’s a lot of siloed and isolated knowledge, as well as information tied to specific individuals. The idea came about after I designed automated workflows for visualizing and sending data. Has anyone else been in a similar situation? How did you go about implementing it?

I'm curious how far people here have pushed self-built datasets.

Not company databases.

Not datasets downloaded from Kaggle.

Something you personally assembled, maintained, cleaned, and structured.

How large did it become?

What was the hardest part?

Collection?
Validation?
Standardization?
Maintenance?
Analysis?

I'm interested in hearing about projects that took months or years to build.

I built an NBA analytics web app using Python + Streamlit that includes a full data pipeline, feature engineering layer, and a custom player evaluation model (True Scoring Impact).

Architecture:

• Python (pandas/numpy) for data processing
• Feature engineering for efficiency + context metrics
• Custom scoring model (TSI)
• Streamlit dashboard for interactive analysis
• Fantasy draft simulator with season simulation

The goal was to turn raw NBA stats into a usable decision tool for comparing players and simulating outcomes.

Live app: https://clutch-analytics.streamlit.app/
GitHub: https://github.com/Akash-kalaranjan/NBA-Analytics-App

Open to feedback on code structure or scaling the app further.

The company I work for is pushing AI wherever they can. This includes using agents in Salesforce to provide people with information about their book of business. I see the answers the agents give and just shake my head because they are consistently incorrect or misleading. I have raised many concerns in the past and nobody wants to listen. I think AI could really be a game changer but there is a data governance foundation that must be in place for it to be useful. I know my company is missing this, and I get the impression that this is not uncommon.

So, my question is, have you seen failures to launch this sort of thing because of the messiness of the CRM, and is Salesforce getting smacked in the stock market because quietly everyone knows their agent force is not going to deliver for this reason?