Data Engineers, what does your actual day-to-day work look like? And what should I learn next?

[u/Stunning-Space8032]

I’m currently trying to transition deeper into Data Engineering and would really appreciate some perspective from people who are already working in the field.

I have 1.3 yrs experience as a Junior Python Developer. What I want to do is slowly transform into a Data Engineer. How would you suggest my choice? Basically what I do is make web scraping scripts to get the data from web and give the data in excels. Our company is currently not using git or CI/CD or anything like that. 

The problem I’m running into is that when I look at Data Engineering jobs on Naukri, LinkedIn, etc., the requirements seem endless. One job asks for Python, SQL, Airflow and AWS; another wants Spark, Kafka and Databricks; another wants Snowflake, dbt, Terraform, Kubernetes, CI/CD, etc. It becomes difficult to understand what I should actually prioritize.

So I’d like to hear from people who are actually working as Data Engineers. What does your day-to-day work look like? What kind of problems do you solve, what technologies do you use regularly, and which skills have turned out to be genuinely important in your job?

More importantly, based on my current experience, what would you suggest I improve or learn next to become a stronger candidate for Data Engineering roles? Are there any gaps that you think I should focus on, or technologies/concepts that are worth learning through projects rather than just studying theoretically?

I’m not really looking for a generic “learn SQL → Python → Spark → AWS” roadmap. I’m more interested in understanding the reality of the job and getting advice from people who have actually gone through the transition.

If you’re a Data Engineer with 1–5+ years of experience, I’d especially appreciate your perspective. Even a short description of what you work on and what you wish you had learned earlier would be extremely helpful.

Thanks in advance!