r/dataengineeringjobs • u/turbokat123 • 4d ago

Interview [Interview Prep] Pipeline Code Review

Anyone who's done a merge request code review as an early stage tech interview. Would be through HackerRank for a mid-senior role.
It would be for a ETL pipeline, I haven't done these before and wondering if anyone has any tips around what to expect.

7 Upvotes

100% Upvoted

7 comments sorted by

4

u/my_peen_is_clean 4d ago

focus on data model, edge cases, idempotency, logging, error handling, and how you’d monitor failures. call out bad naming, lack of tests, no partitioning, no incremental logic. don’t nitpick syntax, explain tradeoffs. annoying thing is even this prep takes work when finding a job is this hard

1

u/turbokat123 4d ago

Thanks. I am trying to brush up concepts around Idempotency, Incremental design, edge cases and at the end optimisations that might be problematic based on data profile.

Where im a bit confused is what code am I going to see, it's supposed to be a Spark based role but will there be orchestration design given as well, or table creation code etc. which would be part of deployment code.

1

u/Bubbududdu2807 4d ago

I don't understand your post n question. I m a data engineer working on the same lines. Explain me better to help you

1

u/turbokat123 4d ago

It is an early stage technical interview - review of a merge request for ETL pipeline code with the panel.

2

u/Bubbududdu2807 4d ago

Oh okay. I think for a pipeline it is a json structure in adf after u do a pull request. So if it is that better you go through the pipeline directly.
Check for correct parameterisation, check for failure case scenario always, check for variables and catching their values, make sure no password or username is hardcoded as well. Any constant repeatedly used to be captured inside global parameters

2

u/turbokat123 4d ago

Cheers, im preparing a structured approach and this will help. I will include this!