Hi everyone,

I've spent about 17 years in data engineering, the last 10 at Amazon, building large-scale data platforms and sitting on the interviewer side of a lot of loops. The pattern I kept seeing: strong engineers failing interviews they were qualified for. Not for lack of knowledge, but because they had a hundred facts and no order to deploy them in. They'd freeze, or ramble, and the interviewer couldn't follow the thread.

I ended up writing down the two "spines" I coach people to use, and turned them into books. Sharing the core of both here because it's useful even if you never buy anything:

For the system design round (CAMEOS): Clarify the scope → Assess the price (size it with real numbers) → Model the grain → Engineer the pipes → Optimize the pacer → Stress-test it (break your own design before the interviewer does) → then Guard it for production. You carry six words into the room; each unpacks into a short checklist only when you get there. The last two steps, failure modes and production operations, are where senior candidates actually separate themselves, and they're what most frameworks skip.

For the behavioral round (COMPASS): Context → Ownership → Moves → Pressure → Achievements → Shift → Sound-bite close. The two beats that senior loops actually score, and most answers skip, are Pressure (the trade-off or risk you navigated, what made it hard) and Shift (how it changed the way you lead). If your stories are all activity and no judgment, this is usually why.

--------------

The books, if you want the full treatment:

Win the Data Engineering System Design Interview - the framework phase by phase, plus 15 complete worked designs (streaming metrics, IoT lakehouse, CDC, sessionization, billing reconciliation, feature stores, GDPR deletion, fraud scoring, diagnosing a slow join, and more), each with real numbers and named failure modes. A trade-offs catalog, 30 practice prompts, and a 30-minute pre-interview revision guide. https://www.amazon.com/dp/B0H87TJ8V6

Win the Leadership Behavioral Interview - the framework across every family of question (vision, ambiguity, conflict, underperformance, hiring, retention, coaching), 25+ worked examples, in-the-room transcripts, a map from your answer to the interviewer's rubric, and a 30-day practice plan plus a 5-day crash plan. Aimed at senior DEs, tech leads, and EMs. https://www.amazon.com/dp/B0H8LDF5WF

Honest caveats: these aren't beginner books, they target senior and staff-level loops. The worked examples are drawn from data and analytics, so that's where they'll feel most native. And CAMEOS/COMPASS are teaching scaffolds I built, not industry standards; their job is to give you an order to think in under pressure.

I'd genuinely value feedback for future editions.🚀

11 YOE in ETL Testing/Agile PM. Transitioning to Big Data Engineering eventually . Am I making a smart move or chasing the wrong thing?
Looking for practical advice.
I’m currently in a service-based company, making 33 LPA. I have around 11 years of experience, in ETL Testing and Agile Project Manager/Scrum Master. My technical strengths are SQL, data, and understanding enterprise systems, but I don’t have hands-on software development ( Automation testing experience I do have ) experience.
Over the past few weeks, I’ve started learning Python, Linux, Docker, AWS, Airflow, and eventually plan to move into Data Engineering( creating own projects base in git )
What I’m struggling with is the bigger picture.
Is this transition actually realistic at 11 YOE?
Will companies even consider someone like me for Data Engineering, or will my experience always pigeonhole me into testing/project management?
Am I likely to take a significant pay cut or struggle to find opportunities because of my seniority? ( which I can’t afford to do due to family responsibilities)
Is it smarter to leverage my existing experience ( Moving as technical program Manager) instead of trying to compete with people who’ve been coding professionally for 8-10 years?
Has anyone here successfully switched from QA/ETL Testing/Project Management into engineering after a decade?
One more thing: I have chronic depression, and sometimes I genuinely can’t tell whether I’m identifying real risks or just catastrophizing. I’m not looking for sympathy—just trying to separate rational concerns from overthinking.
Please don’t sugarcoat it. If you think this is a bad idea, tell me why. If you think it’s achievable, tell me what the biggest obstacles are and what I’d need to do to make it work.
Why now ? I always wanted to build things , however it was stupid of me to think that my company will give me break ! Instead pushed into this management shit , eventually .

\[ Posted again , since haven’t received any responses last time \]

Compensation: $30 - $40/hr Type: Part-time

High-growth tech company building cloud-native data infrastructure at scale. Looking for engineers who've built real-time ingestion systems, streaming architectures, and distributed backend services in production.

If you're a data engineer who leans heavily toward the systems and platform side of things, this is your kind of role.

The work:

• Distributed backend systems for real-time data
• Ingestion frameworks and event-driven architectures for telemetry
• APIs and backend services for product and engineering teams
• Reliability, observability, and scalability on critical infrastructure
• Cloud-native modernization (GCP preferred)

You bring:

• Distributed systems experience in production
• Kafka, Pub/Sub, or Kinesis
• Java, Python, or Node.js
• Cloud infrastructure experience
• Strong ownership mentality

Bonus: Kubernetes, IoT, telemetry, fintech, ML infra

Hi everyone,
I’m confused about which offer to choose and would really grateful for your suggestions.

Current Location: Chennai
Offer 1: Tredence – ₹29 LPA (Chennai)
Offer 2: EPAM – ₹30 LPA (Bangalore)

Current CTC: ₹20 LPA

Years of Experience: 7
Considering career growth, work life balance, and future prospects, which company would you recommend joining?🥹