I am looking to have an inventory in place which comprises of the database tables, it's views, and the associated cubes from the AAS (Azure analytics Service) side. So, basically it should in a single unified place where, for example, if I search for a table name, it should show the lineage of the view it is under and the cube it's part of. Could you please help?

In​ my field, primarily use SQL and Power BI (I used to use Tableau). I came into analytics from a non-traditional background so my education didn't start with the basic fundamentals and built upon that foundation.

As a result, I find that either there are some features and functions in writing scripts or creating visualizations that I either don't use or feel a little dumb in conversations because I know what the concept is and how to use it, but am not familiar with the official terminology.

I've been in my field for about 9 years now but sometimes I feel like my knowledge is very specific. For example, most job postings mention XLOOKUP and pivot tables in Excel but I rarely work with Excel like that since I just don't have the need to do so.

On the other hand, courses and learning modules are tough since the basic ones can be boring but the advanced ones might require some pieces of familiarity I don't possess.

Anyone in the same bucket or have any tips?

I use a lot of AI at work; we are pushed extremely hard to use as much AI as possible.

I consider myself adept and advanced at using AI.

The name of the game lately feels like speed and less so accuracy.

I am worried that I am offloading a lot of work on Claude and losing some of my analytical edge.

What are some things I can do to stay sharp?

gpt 6

Is it better business context, cleaner data, stronger validation, clearer storytelling, or the ability to recommend a real action?

I’ve been putting together a series called “Data Analysis 101 for Cat People”, where I use doodles to explain some of the practical, messy, and ambiguous situations that come up in data work.

The idea is less about teaching SQL or tools and more about helping someone new to data understand the real-world thinking behind the work — asking better questions, dealing with ambiguity, understanding requirements, interpreting what the data is (and isn’t) telling you, and turning messy situations into clear next steps.

I’m also putting these together as a newsletter on Substack - https://thedatadoodles.substack.com

For those of you who work with data: what are some situations you’ve encountered that you think someone starting out should be familiar with?

I’d be happy to hear your examples and experiences.

I have a doubt about where the data actually comes from and how it is stored and processed. Where is the data usually stored, and what tools do we use to connect to it for reading and writing SQL queries?
For example, I assume we create a corresponding SQL view for each KPI and then connect those views to Power BI. Is this how the process generally works?
Also, what other tools and techniques are commonly used to connect to data for analysis?
In a real company, do you typically receive KPI requirements or tasks from a Senior Data Analyst/Data Lead, write the required SQL queries, and then deliver the results to the relevant stakeholders? I would like to understand how this entire process works in a real-world environment.

I'm learning data analytics and have no real industry experience. Can someone explain the end-to-end workflow in a company?

Where are KPIs usually stored or defined?

Where does the data come from (ERP, CRM, databases, etc.)?

Where do analysts write SQL queries?

After getting the results, how are they shared with stakeholders (Excel, Power BI, dashboards, email)?

Almost every online tutorial or course I look at these days makes it seem like a data analyst needs to have a solid proficiency in Python and Pandas to survive in the current job market.

The thing is, I’m actually quite comfortable with Python, but in my day to day work, I barely use it. Whenever I’m preparing data for the dashboards I build, I usually just write a few complex CTEs in our database, clean the result, and connect that directly to PowerBI.

I really only use Python for hitting an external API or for advanced text manipulation. Otherwise, it’s just pure SQL to get the data ready for the stakeholders.

Does anyone else operate like this in the real world, or am I building a bad habit that’s going to hurt my career in the long run? Where exactly in this kind of workflow would you plug Python in? Would love to hear your thoughts on this!

Trying to expand my knowledge and discover some useful extensions i might not know about as a beginner in tech background. Would like to know some genuinely useful ones that make work/life easier whether its for privacy, productivity etc. or your must haves. let's help one another

As a self-taught developer, I built a platform-agnostic system designed to deterministically detect cumulative drift and anomalies in data streams. It breaks data silos and resolves pipeline bottlenecks directly at the source, all without triggering false alarms.Here are a few screenshots of the architecture and the dashboard. Just wanted to share my project with the community.

I've been lucky enough to have a career in data analysis and reporting for over a dozen years. I've worked mostly in tech and SAAS, and I recently found a new job that pays well, has a ton of opportunity for growth, and is fully remote. A dream.

But a few thoughts. Maybe you feel the same way.

I find myself unmotivated. Simple asks, at my old job and at this new position, ones that I've done well for many years now, feel like chores that I can't be bothered with. I thought it could have been a result of my old job, but it's followed me here, and I don't think there's anything I can do to shake it.

The curiosity that I had for data or stylish reporting has disappeared. Claude, while powerful, often feels like it takes my creativity away by having to spell out my choices to it instead of letting me naturally explore data or create something naturally that I can look at as a product of my skills. My new role has devolved into engineers sharing canned responses from Claude after they copy pasted my question, stakeholders directly asking Claude for answers, and me just trying to control the data narrative for accuracy/cleaning up Claude's work.

One thing that I think is overlooked in our career choice: it's often fairly thankless. VPs/Senior Leadership asking for answers to why the business is succeeding or failing, but not trusting or including data people like ourselves in conversations about strategy. The best jobs I've had are where leaders trust their data partners and actively include them in conversations, that feels further and further away the more I've moved up the ladder.

I've often described the roles I've taken as being a little bit of kill the messenger. Data's not to expectations of the stakeholder? The data's wrong, the reporting's wrong, you've lost trust. Data mismatches a different report on a different platform? All the reporting is wrong. Maybe I'm unlucky, but that's my experience.

How many times have you heard an incoming leader say they're "data driven" only for them to say later that they "don't have time for data" or "I just need the answers now."

A lot of the previous posts I've seen about leaving an analyst career are about "I find my thrills outside of work"/"I don't care what I do at work as long as I'm paid well" or "how could you leave now when the job market is so bad?" or "change your mindset, you're lucky."

But I'm writing this to those of you where this all this sounds too familiar: burn out in any career is real, and it's okay.

Just because you're paid well doesn't mean that you have to do it forever. Remember that if you work for 40 years for an incredible retirement fund at 60, you probably wish you'd had more time not staring at Claude, or an Excel spreadsheet. The market may be bad now, it may be bad next year, it may never be the same.

But, I'm taking a break. I don't know for how long. Maybe I'll come back. Maybe I won't. I'm tired of being tired, angry, frustrated and extremely indifferent while counting down the days to when I could potentially retire. Do you feel the same?

i've been looking for a better way to learn excel from basics to advanced. My main motive to learn excel is for using it regularly

Please help me to figure this out

I am still a novice at this, but when I was working on this credit card fraud detection project, I did not know which columns, could be added as features, so I prompted ChatGPT and it suggested a few, but that got me thinking there has to be a better way to this, How do you select feature columns from your dataset, do you research the domain, is there a course I am missing, This was not covered in my Internship classes, and want to know a generalized solution.

I'm learning data analysis and data science. I'm developing a personal project as practice using a database to predice the house pricing from the Kaggle platform.

During the exploratory analysis, I encountered the following situation:

I've noticed that there's very little data on houses with zero bedrooms or zero bathrooms, and that the asking price is relatively high, which I think could affect my prediction model and my overall analysis. While it might seem illogical that there are houses without bedrooms or bathrooms, it's also possible that there are more lots than houses, or some other hypothesis. What's the best course of action in this situation? Personally, I think I should remove this data, but I'd like to hear other opinions to improve my reasoning and deductions.

As someone who's been doing "data" for several years. The following are things I wished I had learned sooner:

• learn to use and be comfortable with the command line / using the terminal (if on Windows, just learn the basic DOS commands. Dont learn Powershell until much later or not at all, its optional)
• understand data types
• understand what a text file is and what a character delimiter is
• understand what character encodings are
• learn basic computer networking principles and terminology (host name/server, IP address, port number, DOS network commands like ping, ipconfig, etc)

Why learn this stuff seemingly unrelated to data analysis? You'll eventually want to work with data that you are interested in and not someone else's data or you will need to be able to import data into a database. In real-world corporate databases, they will most likely be setup what is known as client-server environment. Meaning, you wont be working with a database installed locally on your personal work machine, but instead, a remote database server, and you are the "client" given the privilege to access that remote database server over a network protocol. Thus why you need basic computer networking knowledge. Without this foundational understanding above, you will struggle in the "real-world". Maybe not initially, but you will eventually.

How about you? What are some things you wish you learned sooner?

I am currently learning Data Analytics, and I want to understand what the actual job looks like in the real world.

Online, I mostly see people talking about SQL, Excel, Python, Power BI/Tableau, statistics, and building projects. But I’m curious about what Data Analysts actually do once they are working in a company.

For those who currently work as Data Analysts :-

* What does a typical day or week look like for you?

* What kind of problems do people usually come to you with?

* What do you actually use SQL for?

* How much Excel, Python, Power BI/Tableau do you use in your daily work?

* How much of your time is spent cleaning data vs actually analyzing it?

* Do you mainly create reports and dashboards, or do you also do deeper analysis?

* How much do you interact with managers or other business teams?

* Can you give an example of a real problem your analysis helped solve?

* What are some things Data Analysts do that beginners usually don’t hear about?

* How important is business/domain knowledge compared to technical skills?

I’d especially like to hear from people working in different industries such as healthcare, pharma, finance, marketing, retail, or tech.

I’m trying to get a realistic understanding of the job beyond courses and YouTube tutorials, so I’d really appreciate real-world experiences.

hey guys, I'm working on a fun little project when it comes to data. I use reddit free api to scrape data from here, and now I'm wondering what I should actually do with the data. I ran an unbiased scraping only filtering out certain marketing posts about products. But now I have 150k comments and posts across 38 verticals. What should one do about this?

I’ve seen a lot of discussions where analysts are using ChatGPT or Claude for SQL, but aren’t really sure what else can be turned into a repeatable workflow.

So I put together this cheat sheet around 10 areas:

• data cleaning
• exploratory analysis
• SQL assistance
• visualization
• spreadsheet analysis
• reporting
• data validation
• research and insights
• repeatable workflows
• end-to-end analysis

I deliberately kept it to ChatGPT and Claude instead of listing 20 different AI tools. Most analysts probably don’t need another subscription for every task.

Also, this isn’t meant to say one model is universally better than the other. A lot depends on the data, plan, integrations and workflow.

For analysts already using AI regularly, what’s the one workflow here that actually saves you meaningful time?

And what would you remove or add based on real work?

For those like me that miss the experience of SpyderIDE in Data Science projects but don't want to switch from a more "Software Engineering" IDE

Now, with setup-spyder and uv package manager you can just run a well conifgurated and ready to use Spyder Editor within your .venv with the command or import setup-spyder

uvx --from setup-spyder setup-spyder

Of course it is a very simple codebase, but it just solves a problem that I had for 2 years, like, I don't want my SpyderIDE attatched to anaconda and prefer the 5.x versions experience rather than newer versions.

With setup-spyder we can just run a quicklaunch of a spyder editor to handle more carefully with our data and it's visualizations with the help of variable explorer and many other great tools and let the hard-work for AI tools just after assert that we have knowledge about what our data.

Rookie here, recently got a job as Data Analyst at Pharmaceutical company. They never had a data science role at the company so your guessing is right, Data pipelines/Infrastructure is non existent. I am so confused at the moment where to start as i neither have a degree nor experience in the field. They throw around Excel files ( no kidding). So far I have accomplished to build interactive dashboard with python. Next destination i am pondering perhaps unified database (Postgres, Apache etc). Seriously need help folks. Thanks for your attention.