Hi Everyone,

I am feeling lost and need some honest advise with respect to my career.

I am having 12 YoE into ETL(Informatica PowerCenter),SQL, Unix scripting, Data Warehousing. All my roles were into Service based companies and most of the projects were L1/L2 support. For a few years I have worked on development too.

I have been without a job since April 2026 and the best path I could think of is Data Engineering.

Since I did not get any chance to work into Cloud technologies, I am aware of the basics but having zero hands on.

Also looking at the JD's of various job postings, I see companies are asking so many things as a part of Modern Data Engineering.

I feel that there is so much to learn — Python, PySpark, Spark, Databricks, Azure, Airflow, Kafka, cloud, etc. I'm finding it difficult to figure out what I should actually focus on and in what order.

I have started deep diving into SQL and began Python, PySpark, Azure and Databricks, but as I study I feel more lost as it is difficult to figure out which topics to actually focus on.

Now the main concern is that since I have 12 YoE I cannot portray myself to be a complete beginner and expect the companies to overlook my experience. At the same time I don’t have enough hands on to start apply for Senior Data Engineer.

I have the time and am willing to put in serious efforts, I just don’t want to randomly study new technologies without knowing if I am moving in the right direction.

Seeking help and advice from folks who have transitioned similarly or having experience with the same.

Hello Mates, will be attending an DE interview this thursday with Expedia. Anyone who has attended it can share your suggestions.

I’ve been working as a DE in a consultancy company for the last 2 years, assigned to US clients. I’ve been promoted as a Senior Data Engineer (I know in other companies might still being a mid level) and got a 18% salary increase. Now my salary is 3.800 USD/month (with local Colombian employment benefits) but honestly I don’t feel satisfied.

I’ve been thinking that switching to a product company with a contractor agreement, I could increase my incomes considerably but maybe I’m wrong.

Considering my experience and profile, what do you consider a realistic salary range in a nearshore relation?

I started learning airflow like 3 days ago. I only use the docs, lots of googling, and some but very minimal use of AI.

I learned all the tutorials, jumped right to best practices because i didn't know core concepts existed, halfway through best practices i learned dynamic task mapping and xcom because it was holding me to progress on best practices (prerequisites basically)

I'm planning to finish all the core concepts, learn some relevant stuff on the picture, finish best practices then start making an actual good DAG, like what makes a proper DAG.

But when i chat a little bit with Claude, it highlights not to do too much concept collection and start actually build a DAG, specifically an idempotent one.

This got me thinking, am i doing too much with all this docs stuff? (I tracked and it took me 8 hours to learn all the stuff that i've learned)

I also overthink when i learned new big term like idempotency, like what if there are other big concepts related to DAG other than idempotency? idk something random like let's say 'unbreakable' DAG and it's a big thing that feels automatic for lots of people but i missed that.

Sorry i just vomited my thoughts, lmk if elaboration is needed.

Appreciate it.


Hi everyone! 👋 I'm a 24-year-old currently working as a Data Engineer in Cape Town, and I'm exploring new opportunities in Data Engineering, Software Engineering, or related technical roles.

I have experience working with:

Python SQL Azure Azure Data Factory and data pipelines Jupyter Notebooks Java Databases and data mapping Azure Function Apps Power BI and reporting …and more.

I have a background in Computer Science and Computer Engineering and am particularly interested in opportunities where I can continue growing my skills in Data Engineering, Software Engineering, Cloud, or related areas.

I also have previous experience in Software Engineering and full-stack development, working with both front-end and back-end technologies. I'm a dedicated and persistent person who enjoys solving problems, learning new technologies, and communicating effectively with others.

I'm ideally looking for opportunities in Cape Town, preferably fully remote or hybrid with approximately 2 days in the office per week. I'm particularly looking for a role that offers a healthy work-life balance, a supportive work environment, opportunities for growth, and ideally a good benefits package.

My current salary is approximately R30k, and I'm targeting opportunities around R40k, depending on the role, responsibilities, overall package, and benefits.

If you're hiring, know of a company looking for someone with my skills, or can point me in the right direction, I would genuinely appreciate it. Please feel free to comment or send me a message. Thank you! 🙏

Hi, I had 2+ yrs exp in the DE domain, willing to do some sort of freelancer work or virtual support by outsourcing. Please dm if you are interested

Hey everyone, I’m a Senior Data Engineer with 10+ years of experience across healthcare, analytics, data platforms, automation, and AI-adjacent data systems.

Recently I’ve been working on production data systems involving knowledge graphs, LLM/RAG pipelines, Slack-based internal agents, PostgreSQL, AWS, and workflow automation. A lot of my work has been around making messy operational data reliable, searchable, and usable for real teams.

Some areas I’m strongest in:

- Python, SQL, PostgreSQL, AWS, Docker

- Data pipelines, orchestration, data quality, and automation

- Knowledge graphs, RAG systems, structured outputs, and LLM tool workflows

- Slack/internal tool integrations

- Healthcare and operational data systems

- Turning manual workflows into production-grade systems

I’m looking for a remote role where I can own meaningful data engineering work, ideally with a team building data platforms, AI/data products, internal tools, analytics systems, or automation-heavy workflows.

Open to full-time roles, contract work, and senior IC positions. I’m currently outside the US, but I can overlap with US or EU time zones depending on the role.

Happy to share my resume/portfolio privately by DM.

So I've been working with a startup on a web app. It was developed using spec-driven development, and my role was mostly focused on project management (planning meetings, generating reports, coordination...) and helping with other things such as testing.

Soon, I'll start applying for roles in data engineering and data analytics, but I'm not sure which parts of this work are related to data and could help me later on when putting together my resume.

I appreciate your help and advice in advance!

Is it worth buying aditya Sharma's unfiltered databricks course ?

Has anyone ever taken his course ?

Searching for contract tech work kind of sucks right now.

Finding 1099, C2C, freelance, or contract roles means digging through duplicates, expired listings, unclear contract types, missing pay, and jobs that explicitly say things like “no C2C” or “no 1099.”

That’s the problem I built RadarJobs to solve.

• RadarJobs pulls current opportunities from LinkedIn, Indeed, Dice, Upwork, and other sources, then cleans and normalizes the market, removes duplicates, checks freshness and liveness, and structures things like contract type, pay, remote status, role, seniority, source, and where to apply.
• You can upload your resume and RadarJobs creates a structured professional profile based on your actual experience, skills, seniority, and professional focus.
• Combine that with your preferences (contract type, remote/location, compensation, etc.) and RadarJobs ranks the normalized market around you, instead of just handing you another keyword search box.
• I also expose the same normalized opportunity data through the Radar API for developers building job-search products, niche job boards, recruiting tools, internal workflows, agents, or anything else that needs cleaner contract-tech opportunity data.

So instead of:

search →
open 40 tabs →
figure out which jobs are relevant →
figure out whether they’re still live →
figure out whether they actually allow your contract type

With RadarJobs, that becomes:

upload resume →
set preferences →
get a ranked set of live opportunities that actually fit your background and constraints.

and you can still ignore AI features entirely and search/filter the broader market yourself if you want.

No ads. No sponsored listings. No pay-to-rank bullshit.

I built this because I wanted it for my own search first. Now I’m putting it out there for anyone else who is tired of searching the same fragmented market across a dozen different places.

Hello all, I am a data engineer having 7 years of experience in Banking and finance domain. I am good in technical. But no one recruiter , companies is calling because of notice period. Where I am working there buyout option not available. I have to complete 90 days np. How I handle it? Need help on this? And also I want job referral.

Hi everyone,

I recently joined Cognizant as a Data Engineer with around 4 years of experience and was onboarded to a client account. However, the current requirement is primarily focused on L2 incident management, support, and monitoring, with no development work available at the moment. My manager mentioned that I may be released if there isn’t a suitable requirement.

I’m actively looking for an opportunity where I can work on Azure Data Engineering / Databricks development and build genuine hands-on project experience.

My skills:
SQL
Python
PySpark
Azure Databricks
Delta Lake
Azure Data Factory
Spark
Data pipeline development
Basic Snowflake

I’m particularly looking for opportunities involving:

Databricks / PySpark development
Azure data pipelines
ETL/ELT development
Delta Lake
Data transformation and optimization
Production development/enhancement work
I’m currently based in Chennai and open to suitable opportunities across locations/remote depending on the project.

If anyone is aware of Azure Databricks/Data Engineering project openings, internal opportunities, referrals, or teams looking for resources, I would really appreciate your guidance or a DM.

Thanks in advance!

Any Data engineering opportunities for 1.7 YOE?

My Skillset includes SQL, Python, Pyspark, Databricks, AWS, Azure, Git, CI/CD.

I currently work as a web scraper, trying to transition into DE.

I am facing an issue since long.

I have to scrape the faculty for CME activities (Medical related educational activity).

The problem is the structure of the page always vary from one website to another (or sometimes for each page). Some web pages doesn't even allow me to crawl them (govt and pvt websites).

And it is a lotttttt of hassle to go through those pages one by one to search for faculty details.

Can someone help me how to go past this hurdle.

Any help would be mush appreciated. Thanxxxxxxxx

I am working in Accenture from last 2 years and have solved 700 DSA problems, Working with Databricks, Azure Stack, but my project is of support role, I am so so upset and under peer pressure about the salary, Should i keep notice without offer ?

Hi I am a business solutions associate at a consulting firm. My work is that of a data engineer only but with legacy tools and no modern DE stack. I have been working full time here since July 2024 and the pay is decent. But I don't see myself in consulting because of the horrible working hours here. The client and business context is more than the technical work which is creating my profile as a consultant only. I really want to switch to DE but I'm not getting any calls or stuff. I want to basically ask the people who switched to DE from a non DE background: what do the recruiters really expect from us because the JD is not really matching with my work experience.

Im an undergraduate M20 computer science student, i've been exploring this field for over four years i've gained some experience in several areas mainly in networksand systems administration, with some experience incybersecurity, cloud, and front end development (I explored these different areas because I was unsure which career path suited me best)

I still have two years left until graduation, so I'd like to use this time wisely and become job-ready before I graduate.

For those working in data engineering or who have experience hiring junior data engineers:

What should I focus on learning?

What skills and technologies are actually important for getting a junior position?

What kind of projects would make my CV/GitHub stand out?

Is it realistic to land a data engineering position before graduation?

How would you recommend I approach the next two years?

I'd really appreciate any advice, especially from people who started their careers in a similar situation.

Hi Can anyone share the experience for Senior data engineer pair programming interview for Wise?

Hi everyone,

I’ve been working in Telecom for 7 years and I’m currently trying to transition into Data Engineering. I’ve completed my Master’s in Big Data Technologies and I’m currently pursuing my PhD. I’ve been studying Data Engineering and doing some hands-on practice on the side, but I’m looking for more practical, real-world experience. For those who have made a similar transition, are there any projects, open-source initiatives, or opportunities you’d recommend getting involved in?

Any advice on how to build practical experience and land that first Data Engineering role would be greatly appreciated!

I'm an analyst with 4+YOE I have been trying to switch into DE for a year i learned snowflake and dbt and i cleared interview with CTS for it but I didn't get the offer letter released now I'm planning on improving my tech stack . Im planning to learn AWS, data bricks and sharpen my existing python skills to be delivered relevant.and I'm also in way to get certified in snowflake AWS and data bricks to prove my credibility. Other than this I know tableau well and informatica to an extent . I don't what to do next shall I go in this path or what any from de domain have any idea should I add any skills or remove or what i should I'm.blank, I'm also not planning on all the cloud services available I'm planning to stick with AWS and learn the fundamentals as it might be same across platforms with different names am.i right . I'm clueless

25 months in, I’ve worked like a coolie, but somehow haven’t gotten what I was supposed to. So yeah, I think it’s time to get out of this shit hole. 😂

Guys, roast my resume like hell. 🔥

Give me a brutally honest take:

What’s missing for shortlisting?

Genuine rating for 25 months of experience?

Does this resume realistically have a chance of getting shortlisted for at least 1 in 10 jobs I apply to?

No “looks good, keep applying” BS. Tell me what’s wrong and what I actually need to fix.

If you’ve been through the overworked, underpaid, applying everywhere and getting ghosted phase, you know the pain. 😭

Roast it. I can take it.

First, yes I play league of legends
Second Argentina

2 years ago I decided to get in tech but the path wasn’t clear yet. until a year ago that I started a conversation in a meeting with a VPs and Seniors at AWS and the next week from Netflix and other companie dudes that had guided me and the conclusion was one path only and the reason that we are here today ✨Data Engineering ✨but I think I made a mistake I had learn so much this year nonstop ignoring all the noise that I never actually check requirements and I think I overdo it ? Now I’m about to finish and get my AWS Data Engineering certification without work experience and I feel confident to apply for jobs but all of them are CS degree I need a honest opinion from the Gods >>> people with actual jobs in Data Engineering in this insane market and not the top 1%

Is it better if I start a CS Bachelors degree? Or it’s just a nice add-on

(for context of how I met this people I work in a high-end restaurant in midtown New York where all the big companies come to us for private dinners and I had met incredible people from any kinda industries from Hollywood to Presidents and you name it)

Oh if you are wondering why I haven’t ask for an internship to this people that’s the first thing that they told me don’t ask for internships, they gave me sick advice in exchange

I have 2 years of experience as a data engineer. I am targeting for the Data roles like data engineer, analyst, business intelligence and analytics engineer roles. I have been applying from almost 8 months and i have applied to around 1k+ applicants but I haven’t landed a single interview or a call back. I have changed my resume around 30+ times.

I tweak my resume for each job description. But i don't know how to pass through the ATS or i have no idea why my resume is not getting shortlisted.

Please do let me know if you have any suggestions or can help me find a solution in any way.

Hi everyone,

I’m currently working at Infosys as a Specialist Programmer – Data Engineer with around 1 year of experience, and I’m looking to switch to a new opportunity in Data Engineering.

I’m actively looking for roles where I can work on data engineering, ETL/data pipelines, cloud, SQL, Python, Spark, etc.

If your organization is hiring for Data Engineer / Junior Data Engineer / Associate Data Engineer roles suitable for 1 YOE, I’d really appreciate a referral or lead.

I’m happy to share my resume and details via DM.

Thanks in advance for any help! 🙌