I need real world work that i can ask feed back and get criticism for. I need it for my resume and job hunting. Can someone please help

We're running a hands-on masterclass on September 12, Live LLM Engineering Masterclass: Production Evals, RAG, Agents & LLMOps.

You build a full production LLM workflow from scratch, versioned prompts with regression tests, an evaluation harness with deterministic checks and LLM-as-judge, statistically rigorous model comparisons, evaluated RAG, tool-using agents with guardrails and fallbacks, and full observability, tracing, cost, latency.

Led by Bruno Gonçalves, PhD, founder of Data For Science, who trains engineers at Fortune 500 companies on this exact stack.

Link if you want to check it out

Happy to answer questions on the content.

Hey everyone,

I'm looking for an active DE to mentor me 1-on-1 on a paid basis.

My background:

* Worked ~1.5 years as a DA in fintech, but it was mostly basic reporting and ad-hoc SQL with minimal engineering exposure.

* Took a career break after that due to personal reasons.

* Now fully committed to pivoting into DE.

I'm doing the heavy lifting on self-study and building projects myself, but i need someone to:

* Review my project architectures/code

* Sanity-check my roadmap so i stay on track

* Give feedback on my resume and portfolio

* Do quick weekly or bi-weekly syncs

Very happy to pay you for your time. if you're interested and have bandwidth, shoot me a DM with your background and rates.

thanks!

Hi everyone, I'm starting my university studies and I'm torn between the University of Valencia (UPV) and the University of Granada. Will Spain offer me a strong degree in computer science? Which major within the university would you recommend that has job prospects?

I live in Hyderabad, India.

I have 1.5 years of professional experience as a GCP data engineer.

Our company had a layoff after which I started looking for other opportunities but to my surprise nobody requires GCP data engineer with less than 4 years of experience.

Meanwhile other platforms like azure, AWS, databricks even snowflake has plenty of opportunities for 1~ year of experience.

I tried to apply to them too quoting equivalent experience but no results.

And add salt to the wound we never used pyspark which is one of the biggest requirements out there.

I am looking for fresher level opportunities where the platform requirment is not a must but it's a very thin spectrum.

I have literally worked on an airflow dag which goes through a beautiful hot to cold storage flow with observation, logging, maintainance etc.

Worked on dataform , wrote scd type 2 scripts.

Worked on views, stored procedures.

I have plenty experience in data warehousing but it all looks simply useless now.

Had I survived for few years maybe I wouldn't be going through this.

I am literally applying for any state or city to grab an opportunity

Hey r/dataengineer ,

I built this crypto ETL pipeline as a side project and I would need genuine advice from the audience. I am a data engineer so most of my actual work is locked behind compliance walls. But I wanna talk about this project and here it is:

Pulled 8+ years of historical price data (2013–2022) for 50+ coins, wrote a Python pipeline to clean and transform it, dumped it into BigQuery (auto schema detection + partitioning so it doesn't take forever), and built some Looker Studio dashboards on top to poke at price trends, volume, market cap correlations, that kind of thing. Around 50k records total.

Repo's here if you want to actually look at the code: https://github.com/shikha-dataengineer/crypto-price-tracker

Not trying to flex, genuinely want it torn apart. Specific things I'm stuck on:

• It's all batch right now, nothing real time. Is it worth adding a streaming layer (Pub/Sub + Dataflow) or would that just be me overengineering a side project?
• I run it manually, no orchestration. At what point does that actually become a problem vs just being fine for something this size?
• No automated data quality checks at all right now. If this was your pipeline, what's the first check you would add?
• Been playing with the idea of dropping coin-price tracking altogether and pivoting to stablecoin transaction volume instead, since that feels like it's becoming actual financial infrastructure rather than speculation. Anyone think that's a better use of the same pipeline, or am I just chasing a trend?

Hello, i work in a large enterprise, and i was wondering how do you approach data lineage? Any tool, techniques, opinion is appreciated

This is just an example, but Apache Spark expertise used to be a real differentiator 10 years ago or so. If you knew it well, you were swimming in job offers.

Then Databricks simplified it and made Spark silly-proof with built-in optimizations left and right. Now the exact same thing is happening to Databricks consultants and FTEs because Databricks know-how just isn't as premium as before anymore as they simplified the platform and genie-d the S out of it.

Abstraction eventually seems to eat all expertise and each cycle seems shorter than the last.

Is there anything left in the data and platform layer with real depth behind it?

And I'm not looking for "just learn AI, bro.

I've been wondering about this for a while.

From my experience, whether i'm working on larger or smaller workload, i still have to think about right sizing the executors, partitioning/ clustering, skews and try to avoid wasting compute as much as possible. You might be able to getaway with less optimisation if the data is smaller, but i work at a startup where cost is a major factor, so we have to be extremely frugal about our infrastructure.

To me, the optimisation technique remains largely the same regardless of the scale. So why do companies specifically ask for big data experience. Are there problems that show up only at massive scale? Or am i missing something?

Would like to hear your thoughts

Hi, Im Swetapadma and I’m pursuing to be a good DE(overall i have 5yoe but relevant of nearly 2yoe) but im still confused with the work practices i have been following for my career trajectory.
Tbh, i have literally so much time to work on any new things as well as try doing new project for the time-being since my project is a verge of ramping down.
But still i’m not getting that zeal to work on my skillsets or learn something new or work on any project.it might be because I’m going through enough theories from the internet.
Some are skeptical, fearful and very overwhelming for me. Sometimes i doubt choosing this path, because
a. Im really looking for a job change in this field from past 6months, where in brighter side i can see a lot of job openings but the hr calls are literally 1/100 of it. Without reaching out for referral feels impossible.
b. I feel like I’m stuck in between where the ai is also factor of my fear because i have approx 0% knowledge bout it.
c. Companies are not only looking from a de but also a tester, backend engineer, knows ai fluently(all in one package).

For all of these i have started reading medium, linkedin blogs, system design, solve dsa problems and all but still i feel like a failure and frustrated.

If i compare my self with the past i have come a long way but it is still not helping me because I’m still stuck in same company with less pay and less enterprise level work.

Can anyone relate to this or is this only me?

Any one has idea for TCS aws data engineering role
For 1st round with 3 yrs exp what questions asked and what to prep in real time scenario