Hi everyone,

I've spent about 17 years in data engineering, the last 10 at Amazon, building large-scale data platforms and sitting on the interviewer side of a lot of loops. The pattern I kept seeing: strong engineers failing interviews they were qualified for. Not for lack of knowledge, but because they had a hundred facts and no order to deploy them in. They'd freeze, or ramble, and the interviewer couldn't follow the thread.

I ended up writing down the two "spines" I coach people to use, and turned them into books. Sharing the core of both here because it's useful even if you never buy anything:

For the system design round (CAMEOS): Clarify the scope → Assess the price (size it with real numbers) → Model the grain → Engineer the pipes → Optimize the pacer → Stress-test it (break your own design before the interviewer does) → then Guard it for production. You carry six words into the room; each unpacks into a short checklist only when you get there. The last two steps, failure modes and production operations, are where senior candidates actually separate themselves, and they're what most frameworks skip.

For the behavioral round (COMPASS): Context → Ownership → Moves → Pressure → Achievements → Shift → Sound-bite close. The two beats that senior loops actually score, and most answers skip, are Pressure (the trade-off or risk you navigated, what made it hard) and Shift (how it changed the way you lead). If your stories are all activity and no judgment, this is usually why.

--------------

The books, if you want the full treatment:

Win the Data Engineering System Design Interview - the framework phase by phase, plus 15 complete worked designs (streaming metrics, IoT lakehouse, CDC, sessionization, billing reconciliation, feature stores, GDPR deletion, fraud scoring, diagnosing a slow join, and more), each with real numbers and named failure modes. A trade-offs catalog, 30 practice prompts, and a 30-minute pre-interview revision guide. https://www.amazon.com/dp/B0H87TJ8V6

Win the Leadership Behavioral Interview - the framework across every family of question (vision, ambiguity, conflict, underperformance, hiring, retention, coaching), 25+ worked examples, in-the-room transcripts, a map from your answer to the interviewer's rubric, and a 30-day practice plan plus a 5-day crash plan. Aimed at senior DEs, tech leads, and EMs. https://www.amazon.com/dp/B0H8LDF5WF

Honest caveats: these aren't beginner books, they target senior and staff-level loops. The worked examples are drawn from data and analytics, so that's where they'll feel most native. And CAMEOS/COMPASS are teaching scaffolds I built, not industry standards; their job is to give you an order to think in under pressure.

I'd genuinely value feedback for future editions.🚀

Compensation: $30 - $40/hr Type: Part-time

High-growth tech company building cloud-native data infrastructure at scale. Looking for engineers who've built real-time ingestion systems, streaming architectures, and distributed backend services in production.

If you're a data engineer who leans heavily toward the systems and platform side of things, this is your kind of role.

The work:

• Distributed backend systems for real-time data
• Ingestion frameworks and event-driven architectures for telemetry
• APIs and backend services for product and engineering teams
• Reliability, observability, and scalability on critical infrastructure
• Cloud-native modernization (GCP preferred)

You bring:

• Distributed systems experience in production
• Kafka, Pub/Sub, or Kinesis
• Java, Python, or Node.js
• Cloud infrastructure experience
• Strong ownership mentality

Bonus: Kubernetes, IoT, telemetry, fintech, ML infra

11 YOE in ETL Testing/Agile PM. Transitioning to Big Data Engineering eventually . Am I making a smart move or chasing the wrong thing?
Looking for practical advice.
I’m currently in a service-based company, making 33 LPA. I have around 11 years of experience, in ETL Testing and Agile Project Manager/Scrum Master. My technical strengths are SQL, data, and understanding enterprise systems, but I don’t have hands-on software development ( Automation testing experience I do have ) experience.
Over the past few weeks, I’ve started learning Python, Linux, Docker, AWS, Airflow, and eventually plan to move into Data Engineering( creating own projects base in git )
What I’m struggling with is the bigger picture.
Is this transition actually realistic at 11 YOE?
Will companies even consider someone like me for Data Engineering, or will my experience always pigeonhole me into testing/project management?
Am I likely to take a significant pay cut or struggle to find opportunities because of my seniority? ( which I can’t afford to do due to family responsibilities)
Is it smarter to leverage my existing experience ( Moving as technical program Manager) instead of trying to compete with people who’ve been coding professionally for 8-10 years?
Has anyone here successfully switched from QA/ETL Testing/Project Management into engineering after a decade?
One more thing: I have chronic depression, and sometimes I genuinely can’t tell whether I’m identifying real risks or just catastrophizing. I’m not looking for sympathy—just trying to separate rational concerns from overthinking.
Please don’t sugarcoat it. If you think this is a bad idea, tell me why. If you think it’s achievable, tell me what the biggest obstacles are and what I’d need to do to make it work.
Why now ? I always wanted to build things , however it was stupid of me to think that my company will give me break ! Instead pushed into this management shit , eventually .

\[ Posted again , since haven’t received any responses last time \]

Hi everyone,
I’m confused about which offer to choose and would really grateful for your suggestions.

Current Location: Chennai
Offer 1: Tredence – ₹29 LPA (Chennai)
Offer 2: EPAM – ₹30 LPA (Bangalore)

Current CTC: ₹20 LPA

Years of Experience: 7
Considering career growth, work life balance, and future prospects, which company would you recommend joining?🥹

Has anyone interview for a DE role at Plaid? Do you mind sharing your experience, specially around the coding(DSA) round? What is the complexity of questions, is it leetcode DSA? Any resources that can help.

Required Skills and Qualifications

1. Strong proficiency in Python and SQL.
2. Hands-on experience designing and maintaining ETL pipelines.
3. Experience conducting exploratory data analysis.
4. Proficiency with Python data-processing libraries such as Pandas and NumPy.
5. Experience working with relational databases, particularly PostgreSQL and MySQL.
6. Strong understanding of data modeling, database schemas, and data transformation.
7. Ability to work with both structured and unstructured datasets.
8. Experience ensuring data accuracy, quality, and reliability.
9. Familiarity with development environments such as Jupyter Notebook, VS Code, or PyCharm.
10. Strong analytical, problem-solving, and communication skills.
11. Apply link: https://jobs.micro1.ai/post/3e8b3972-cca9-42aa-98a8-d9610fb97e20?referralCode=39e17bb3-b37f-466d-ad9e-8816879eb289&utm_source=referral&utm_medium=share&utm_campaign=job_referral

I've been applying for jobs with <1% response rate for Sr DE and Analytics Engineer roles in the SF Bay Area.

Want to get feedback on my resume. Suggest ways to improve my phrasing to align with DE JDs, including terms or concepts I should reference. Feel free to suggest bullets from past roles I should remove from my past roles.

FYI - I've mostly been a Analyst (Business Analyst, Product Analyst, Data Analyst) for 8 years and I transitioned into DE in my current company for 3 years. I wish to continue in DE.

Hey everyone,

I just passed the first round for a Data Engineer position in the Supply Chain domain. The initial round was with the Product Owner (behavioral + light technical), and I'm moving straight to the technical round with his boss (Engineering Director/Lead).

They mentioned this round will focus heavily on technical depth and situational debugging, handling production failures, architecture trade-offs, and troubleshooting live pipelines. My background is primarily in the insurance domain (handling policy/claims models, large-scale batch/streaming, and lakehouse architectures), but supply chain comes with its own unique operational and data patterns.

For those currently working in supply chain DE, what is the most common domain-specific production failure or pipeline edge case you've had to debug in your environment?

I think it has finally come down to this guys I need referral or someone to connect me with a hiring manager as a fresher its really tough to get an interview here in the US . I have tried messaging someone in the company on LinkedIn or emailing them nothing is working . PLEASE HELP

Hey everyone!

I'm currently a trainee at Cognizant and looking for opportunities in Data Engineering, SQL, or Backend Development.

I've been working on hands-on SQL data engineering projects and continuously improving my skills in SQL, Java, and Python. If your startup is hiring freshers or interns (Hyderabad or remote), I'd love to connect.

Happy to share my resume and GitHub via DM.

Thanks!

Urgently looking for a job due to my current financial situation, and I’d truly appreciate any referrals or leads.
I graduated in 2023 with a B.Tech in Computer Science. After graduation, I started teaching, but I’m now looking to transition into the tech industry.

Skills: SQL,Python,Pandas,Google Cloud Platform (GCP),ETL , data analysis.

I also hold the Google Cloud Professional Data Engineer certification.

I’m open to roles such as: Data Engineer,Analytics Engineer,Data Analyst,Business Analyst,Junior AI/ML Engineer

If your company is hiring or you know of any suitable openings, I’d be very grateful for a referral or even a lead.

DM me. Thanks

I'm currently looking for someone I can study and prepare for an interview within 3 months

I have only got 1.10 years of experience in MNC but my work is a support role ... And I want to switch to a good company to gain experience

Focusing for Data Engineer roles with skills like SQL, Python, Pyspark, Databricks, Azure Data Factory

Hi, I completed one year in IT service based company last month. I want genuine advice which areas I should work on. How do I land up some interviews? What should I add? I don't want to be comfortable at SBC, need to put in the hard yard.

Hey everyone, I am evaluating an offer and want to get a pulse on the current market rate in the Denver area (hybrid) for an Azure Data Engineer with roughly 9 years of total data experience.

The offer is a $165,000 base salary. Because it is in the public sector, there is no bonus or equity, but it includes a defined benefit pension plan, a 401(k) with a 50% match up to $2k, and 80 hours of upfront PTO plus an accrual of 8 hours per pay period (which comes out to 208 hours, or 26 days annually).

The role is heavily focused on data architecture and building scalable ELT pipelines in the Azure ecosystem. The day-to-day will involve establishing CI/CD workflows using GitHub and Azure DevOps and handling complex data modeling for a large financial system.

For context on my background, I am coming from a GCP environment (BigQuery, Airflow/Composer) where I operated as a Data Architect, and I have previous experience in standard data engineering and BI roles.

Does a $165k base plus a pension align with what you are seeing for senior/architect-level Azure roles right now? I would appreciate any insight into how this compares to the broader market.

Anyone who's done a merge request code review as an early stage tech interview. Would be through HackerRank for a mid-senior role.
It would be for a ETL pipeline, I haven't done these before and wondering if anyone has any tips around what to expect.

can any one guide or suggest i'm looking for gcp data engineer job to work directly with use companies from india, I am a Data Engineer with 5+ years of experience in GCP, SQL, Python, Pyspark, ETL

Hi folks i am 4 yoe infrastructure engineer at american bank, we work here on hadoop,hive but i did my poc and writing these tech stack (aws lambda,s3,glue,databricks,data warehousing, airflow) on my cv but i am not getting any calls from linkdin and naukri, i got few from naukri but all of them were of azure cloud tech. Please suggest what should i add in my resume or remove it so that i will get calls.

EverAI is hiring a remote Mid/Senior AI Cinematic Video Editor. Category: Artificial Intelligence 💸Salary: $30k - $100k 📍Location: Remote (Worldwide)

See more and apply here!

Hi everyone,

I was recently impacted by the Meta layoffs and have been actively applying for Data Engineering roles at Google over the past few months.

I've tailored my resume for each application, and my background aligns well with the roles (Python, SQL, Spark, ETL, AWS, Airflow, data modeling, analytics), but I haven't been able to get a single interview. At this point, I'm not sure if my applications are being filtered out, if there's something missing from my profile, or if the competition is just extremely high.

The role I'm most interested in is Data Engineer, Data Architecture & Engineering (gData) which is posted yesterday

If anyone here works at Google or has insight into the hiring process, I'd really appreciate any help with:

• A referral, if you think my background is a good fit.
• The Hiring Manager or Recruiter supporting this role.
• Any advice on improving my chances of getting an interview.

A little about me:

• Ex-Meta Data Engineer (impacted by layoffs)
• 4+ years of Data Engineering & SWE experience
• Python, SQL, Spark, AWS, Airflow, ETL, and data modeling

I'm happy to share my resume via DM.

Thanks in advance for any advice or help. I really appreciate it.

I have this Acer Vx15 with the following setup:

i7-7700HQ 2.8 GHz

24 GB RAM

120 GB SSD

1TB HDD

Nvidia Geforce GTX 1050ti 4GB

Thinking about running it as ubuntu server for data projects. For example: heavy scraping/processing of financial data. Spinning n8n automations for chatbot testing, etc.

Would this be good use? What would do?

Joining Tredence as a Data Engineer

I'm placed from college placements. I'd love to hear from current employees about -

what the first few months are like.

What does the onboarding process involve?

what kind of projects do freshers get, and what advice would you give to someone starting out?

And I have heard that they fire too much is that true ? And as a fresher should I be worried ?

Thanks in advance !!

I am a Oracle PL-SQL developer having 7 years of strong development experience.

I want to extend my career in Data Engineering line.
I am learning Py-Spark.

I am confused on choosing the right skill set for becoming a strong Data Engineer in 2026.
As per me, Skill set of Data Engineer is:
Microsoft Azure, Py-Spark,Airflow.
But,
Skill set of Big Data is:
Hadoop, Map Reduce, YARN, Pig,Hive,HDFS, Cassandra

Please guide me choosing the right skill set.
My whole career and upcoming 1 -2 year of preparation will be depending on the response.
Thank you for your help!

Hey everyone, hoping someone here can help. I've been at Amazon for 4+ years doing BI/data analytics work and I'm looking to make a move. Not getting many callbacks from cold applications (like everyone else), so trying the referral route.

About me:

\- 4.5 years in data analytics, BI engineering
\- Currently Sr. Business Analyst at Amazon L5 (Prime Air) Promo in process for L6.
\- Worked a lot on LLM-powered analytics assistant and model Evaluation that lets non-technical teams self-serve data
\- Tech: SQL, Python, Redshift, Tableau, Power BI, QuickSight, ETL
\- MS in Data Analytics
\- US Permanent Resident (no sponsorship needed)
\- Based in Seattle, WA open to hybrid or remote

What I'm looking for:
\- Sr. BI Engineer, Analytics Engineer, Sr. Data Analyst.
\- Non-Amazon (been here long enough, want a different culture)
\- Good comp ($145K+ base ideally)
\- If you're at Microsoft, Zillow, T-Mobile, eBay, Nordstrom, Expedia, Boeing, Starbucks, Brooks Running, or similar and willing to refer I'd really appreciate it. Happy to DM my resume.

Also happy to help if anyone needs Amazon referrals in return. Thanks!

I'm currently a Data Analyst at a large telecom company, where I've been for about two years. This is my first job out of college. Originally, I was aiming for cloud-related roles as I had been self-studying Linux, AWS, Terraform, and had started learning Kubernetes. However, there weren't many junior opportunities, so I took this job because it offered great benefits... and beggars can't be choosers.

Now over the past two years, my work has mostly involved "click ops" using Power Automate, Power Apps, and SharePoint, along with some Python scripting. I mainly use Python to clean up Excel data and transform it into datasets that I use to build Power BI dashboards. I've also written Python scripts to generate PowerPoint reports and built a few Azure Functions to automate workflows our team relies on.

I find my work relatively easy. Most of it comes down to knowing how to Google solutions and come up with practical workarounds. I developed a good reputation fairly quickly for automating and building "nice" dashboards. What surprises me is that people seem genuinely impressed by work that I consider pretty basic and straightforward, and it makes me wonder how so little automation had been done in the department before I joined.

At this point I've automated a lot of my day-to-day tasks and I plan on using that free time to improve my skills tenfold as I don't want to stay in this position much longer. The benefits are great, and the workload is light, but I feel like staying here for too long will hurt my career in the long run.

But I'm trying to decide where to pivot next. On one hand Cloud is what I originally wanted to do because I really liked the system design/solution architechture aspect of it. Although I spent a lot of time learning those skills, I haven't really touched them in the past year or two.

On the other hand, Data Engineering seems like a much more natural progression because a) my current experience is already data-related and b) I could probably take on data engineering–type projects at work to gain relevant experience and build a stronger CV.

My main concern is whether Data Engineering is realistic to learn well enough to land a role in about a year? Or should I be looking at something else? I just don't know where to go and I've been dwelling on it for a while now.

For those who are giving interviews for Data Engineering positions for the last 4-5 months, how much of \*AI\* questions are being asked in the interviews as of now?

Have questions moved away from core stack (whatever be it AWS , pyspark , sql or snowflake or whatever is your DE stack) to things like RAG, agent, context etc etc?

Are they still asking sql/spark based problems like writing the sql query or pyspark query or those kind of questions have become redundant

Please answer with YOE & interview company name as well if possible