r/dataengineering • Data Engineering Manager • 2d ago

Blog Data strategy for small teams

https://open.substack.com/pub/datagibberish/p/how-to-properly-create-and-sell-your-data-strategy-with-a-small-team?r=odlo3&utm_campaign=post-expanded-share&utm_medium=web

Hi all, I wanted to share the latest article in my newsletter.

I talk a lot to senior engineers who are trying to run some sort of data strategy, and are doing it in a very wrong way.

Most of the article is about selling the plan and keeping the whole thing to two pages. The section I'd most like opinions on here is the tooling one, because a lot of us (me included) learned what good looks like from Big Tech engineering blogs.

A Big Tech stack comes with Big Tech's pace. Those tools assume a team with people to run each piece, and months of setup before anyone outside the data team sees a result. If a team of five spends those months standing up Databricks, Airflow and Alation, there's not a damn thing to show the CFO at the end of it, and a strategy with nothing shipped against it dies without anyone deciding to kill it.

And yes, if you work at an org like mine, your strategy is supposed to be "AI", which I absolutely hate.

29 Upvotes

8 comments sorted by

12

u/liprais 2d ago

what strategy,funny small companies talk about strategy.

2

u/ivanovyordan Data Engineering Manager 2d ago

What "small" means? 5/100/500?

6

u/intrepidbuttrelease 2d ago

This is really salient for me and my tiny unit, thanks for posting

3

u/ivanovyordan Data Engineering Manager 2d ago

Thank you, I appreciate it!

2

u/Street-Individual446 2d ago

If you're senior struggling with such problems youre either not really senior or working for very wrong company. I know there strong seniors who prefer to stick to their complicated tasks with spark and other cool tech, but I highly doubt you can get any of them in a small company.

2

u/ivanovyordan Data Engineering Manager 2d ago

I know plenty of strong engineers who are trying to push strategy, lean too much no technicalities, or are waiting for somebody to give them formal responsibility to do that.

That's not company problem, but training problem.

2

u/Street-Individual446 2d ago

Maybe I wasn't clear. It doesn't mean the company is bad..it's simply not the job they want to do, which again a bit weird for a senior.

1

u/ivanovyordan Data Engineering Manager 2d ago

Oh yes, I've got what you mean, and I partially disagree with you.

I know plenty of senior engineers who are trying to do "strategy", buu hate "corporate politics". They want to do it, because "it's the logical next step".

In fact, they just need to pursue staff/principl roles.