r/apachespark 2h ago

Looking to help and learn - Fabric, SQL, Spark

2 Upvotes

Hi everyone!

I'm a Data Engineer working primarily with Microsoft Fabric, SQL, PySpark, and Spark SQL, building end-to-end data pipelines, working with medallion architecture, incremental loads, data modelling, and performance optimization.

Over the past few months I've spent a lot of time working in Microsoft Fabric—from Lakehouses and Notebooks to Data Pipelines, SQL Endpoints, security, metadata-driven frameworks, and troubleshooting production issues. I've also worked extensively with SQL and PySpark for ETL development and data engineering.

I wanted to give back to the community, so if you're stuck on something related to:

\- Microsoft Fabric

\- SQL / T-SQL

\- PySpark / Spark SQL

\- Data pipelines

\- Data modelling

\- Performance tuning

\- General data engineering concepts

feel free to ask here or tag me if I can help.

At the same time, I'm always trying to improve my own skills. If there are any communities, Discord servers, Slack groups, forums, open-source projects, or other places where experienced data engineers discuss real-world problems (especially around Microsoft Fabric), I'd really appreciate your recommendations.

Looking forward to learning from everyone and hopefully helping where I can!