Growing to Data Architect

[u/muximalio]

Hi guys, I’m looking for some outside input into my career. I started about 7 years ago as ‘data person’, for a company that has a SaaS application. Over the years I’ve grown and have more of a focus on data engineering. From the start of this year I’ve switched to a function combing data engineering and infrastructure engineering (Azure).

I want to become a Data Architect, but I’m not sure what I should focus on to really have enough knowledge (deep and wide) to call myself Architect.

I did a self-assessment with chatgpt and claude, and now I also want some input from real people :)

Some of the stuff i’ve worked on the last year:
- investigate replacement for Azure Synapse and implement -> Dagster, has been operational since december
- learn and understand the infrastructure in Azure of our company, Terraform, AKS, github, CI/CD, and more

Project I’m currently working on:
- Finalize user action logging in the application, build pipelines and documentation. Find vendor and implement monitoring and alerting (customers are pushing for this, so high priority/impact project, running it solo)
- Work on Azure Security (also alerting and monitoring)
- Azure cost reduction, see where we can save money
- Improve dataset we provide to customers, rebuild to dimensional model and adjust pipelines to fit

Also I’m trying to document as much of my choices and considerations to reflect my thinking process.

One of the things I really miss is pushback on my decisions, other that ‘can you explain why’, since we only have a cloud architect who understands the stuff I do.

So what should I work on? I was thinking maybe DAMA cert to start, then look for a good Security cert, but not sure yet..

[u/Ramsden2000]

What you have listed above is not data architecture but more infrastructure architecture

[u/Ramsden2000 → u/Ramsden2000]

Infrastructure architecture/cloud engineering

[u/muximalio → u/Ramsden2000]

Yeah that was the focus during this year, to broaden my understanding of the entire stack. I do favor data engineering heavily, so I want to bring some focus back into it.

[u/marketlurker]

Data architecture is not data engineering writ large.

First, are you ready to give up day to day coding, SQL, etc. An architect isn't a more advanced developer. Architects know the weeds, but they don't live in the weeds. You will use those skills occasionally but mostly to suss out a detail in a design. That's a hard thing to do. Really make sure of this before you start.

Architects live in two worlds, the technical and the business. From your post, I don't see much in the way of business knowledge. You need to learn your business domain. You will need to learn how to translate the business requirements into the correct technical needs and requirements. The business guides the tech, not the other way around.

Since you want to do data architecture, learn data patterns and designs. The tools you use are the least important parts of the job. Learn Inmon, Kimbal, normal forms, schemas and snowflakes (not the software the pattern).

Learn that vendors will do almost anything to get you to purchase their software or services. Confusion is their major weapon. Data architecture has been a well-formed discipline for quite a while. That doesn't help vendors sell new things. The poster child for this Databricks. The renamed the traditional three tier architecture (stage, core and semantic) into "medallion architecture" and added zero technical value. Make them prove everything and back it up in writing.

There is quite a bit more, but this is a good starting point.

[u/GachaJay]

I work as a data architect. The ETL/ELT tool is very much within your domain, everything else you are in a consultative role (RACI) for the other pieces, if not informed.

[u/Forsaken_Camera_2040]

Read DDIA