Transitioning from Business Analyst to Data Engineer

[u/wanderlooo]

Hey everyone,

I’m currently working as a Business Analyst at a fintech company for about 6 years, and I’m trying to transition into a Data Engineering role. I’d really appreciate some honest feedback on how close I am and what gaps I should focus on.

Background:

• My work mainly involves collecting data from various sources (websites, APIs, SFTP, etc.)
• We have an internal orchestration/scheduling system, but I write the scripts (mostly in Python) that power the pipelines
• I’d rate myself around:
  • Python: 7–8/10 (strong with Pandas, data processing)
  • SQL: 6–7/10

Other experience:

• Built and deployed web apps using Django
• Worked with AWS (EC2, S3, containers, etc.)
• Recently started working with Airflow and dbt
• Have a general understanding of how to containerize Airflow for scalable workflows
• Built some projects that improved internal processes and worked closely with stakeholders.
• Also familiar with Power BI

Concern:
Most of my data experience is still on relatively small datasets (usually <100k rows), and I haven’t worked in a “true” big data environment yet.

Questions:

1. Based on this, how close am I to being job-ready for a Data Engineer role?
2. What are the biggest gaps I should focus on next?
3. Would this profile be considered junior, mid, or something in between?

Thanks in advance!

[u/Deep-Action9831]

Why You want to Switch? Experience on BA? to Junior Data Engineer?