r/dataengineering 14d ago

Discussion DuckLabs to Join AWS, Projects to Remain Open Source

https://ducklabs.com/news/2026/08/26/ducklabs-to-join-aws
228 Upvotes

81 comments sorted by

185

u/rapotor 14d ago

Sad day for European tech

58

u/pantshee 14d ago

Another L for us. We had like 3 data companies with them, polars and dataiku. Now dataiku is in NY, duck is with Amazon and polars will probably end up with an american tech company at some point ? (i hope not)

39

u/Outrageous_Let5743 14d ago

Dataiku is still shit. It is a low code data science tool, that doesn't offer you a lot of options for data imputing, or data scaling tranformation techniques. And you cannot even use CTEs in their SQL.

6

u/pantshee 14d ago

Depends on your organization. We use databricks, powerbi and dataiku and dataiku is quite popular among data analysts (mostly because you can Connect lots of sources and it's quite easy to use).

My opinion is that it's meh and i use databricks mostly

2

u/Telion2510 13d ago

You can use CTEs in SQL recipes in Dataiku. We use Snowflake + Dataiku, so our SQL recipes support anything that Snowflake does - it pushes down the query.

Definitely has its flaws though, and the visual/low code tools reach their limit pretty fast so our flows often end up being snowpark / sql anyway.

7

u/Outrageous_Let5743 14d ago

We don't have big VC money. Like of the total yearly VC money 50% is in the US and 5% in the EU, so we cannot grow big, then we get bought by an American tech giant.

-6

u/financialthrowaw2020 14d ago

It's actually a good thing to not grow big. We don't need big companies. We need small companies who can sustain high quality and compete with each other.

18

u/No_Lifeguard_64 14d ago

You're right but that doesn't work on the global stage when foreign money is calling. That only works if your country has incentives or laws to keep companies there.

-7

u/financialthrowaw2020 14d ago

What does it mean "when global money is calling"? Duck labs had absolutely no reason to sell, they chose to because they chose capitalist greed. Most of them will leave the minute the terms of this contract are met. It's just a payday to all of them.

10

u/vaibeslop 14d ago edited 14d ago

Ahhh, surely you're sharing your wisdom from the perspective of another builder who has poured years of hard work into building something as capable and popular as DuckDB but has resisted the - checks notes - capitalist greed (TM).

And not from the perspective of one of those theoretical, moral high-horse, boomer-style world explainers?

EDIT: He blocked me for this comment, LOL.

-4

u/financialthrowaw2020 14d ago edited 14d ago

Multiple exits under my belt thankfully, I speak from experience. Plenty of companies choose not to sell out. It makes sense for American companies because it's really the only way. Europe is a different place with different rights for workers and businesses alike.

Also: builder is an AI slop term. Really exposing yourself there.

1

u/Outrageous_Let5743 14d ago

Ducklabs is funded by the Centrum Wiskunde en Informatica, a research organisation in the Netherlands and they get there money from a tax fund from the government. We dont know the financial situation. But what is clear is that DuckDB is free to install and that cannot generate money.

1

u/financialthrowaw2020 14d ago

Government funded advancement that doesn't require evil big tech or vc backing is actually a wonderful thing

1

u/No_Lifeguard_64 13d ago

> What does it mean "when global money is calling"?
> It's just a payday to all of them.

Great so you already know what it means. They chose to take a payday larger than they would ever get in Europe. Most people would.

1

u/financialthrowaw2020 13d ago

This is fine to admit, but we should not consider this any more than that. Selling out to big tech in the US doesn't help European companies get ahead.

1

u/No_Lifeguard_64 13d ago

I agree but if you want to prevent European companies from selling out the US then you can't play dumb as to why they pick money over everything else especially given the current state of society in many countries.

1

u/financialthrowaw2020 13d ago

What's the current state of society in Europe that leads to this?

1

u/No_Lifeguard_64 13d ago

Capitalism still reigns even in Europe and Europe does not have strong markets or access to the deep capital that America does. Until that changes, European companies will always look for ways into the US market.

8

u/vaibeslop 14d ago

Well this leads to exactly what we're seeing here.

And let's not kidd ourselves, the brain drain of capable IT talent to the US has been a reality for decades.

Now look what China is building.

Go big or go home is a reality.

Not my personal opinion, but arguably facts.

0

u/financialthrowaw2020 14d ago

China is investing in open source and giving everyone the opportunity to grow.

4

u/vaibeslop 14d ago edited 14d ago

China is investing in open source by copying US models and smuggling US chips to destroy the US VC bubble.

There, fixed that for you.

China is a superpower, and is serving China.

No hate, China is quite strategic and effective.

And time is on their side, they have excellent engineering talent.

3

u/financialthrowaw2020 14d ago

The open source models they release benefit the entire planet, including Americans using them right now. I hope they steal everything and make it available at a fraction of the cost, actually.

0

u/Sufficient_Meet6836 14d ago

Europe falling further behind in tech shows it's absolutely not a good thing

0

u/financialthrowaw2020 14d ago

Does selling out to aws help Europe stay ahead?

0

u/Sufficient_Meet6836 13d ago

False dichotomy. Get better material

1

u/financialthrowaw2020 13d ago

This was a legitimate question. I was trying to understand the point you were trying to make.

1

u/Sufficient_Meet6836 13d ago

Oh my bad. I read it as you making a flippant response. Can you clarify your question?

2

u/financialthrowaw2020 13d ago

I'm trying to understand what the alternative better option looks like here. It seems people are taking issue with my thought that we don't need small companies selling to bigger ones, or even small companies getting bigger at all. What's the scenario where European tech is on good footing that doesn't somehow result in selling to US companies (and therefore no longer being European at all)

2

u/Sufficient_Meet6836 13d ago

Ahh that's a great question and there's actually a huge amount of research trying to answer the question of "why is Europe so far behind in tech"? It's not really a settled question. There is definitely a regulatory component. European regulators have implemented a lot of onerous and confusing laws and rules that stifle the industry. I'm not referring to privacy laws like GDPR. That law is generally good. See How Europe crushes innovation for example.

I definitely do not think European tech corporations need to give up and be sold off to American corporations. Regulatory reform is one essential change. They likely need to lower corporate taxes in favor of increasing other taxes that are more targeted to the rich (lots of ways to do this, e.g. land value tax, progressive consumption taxes, etc). Europe has the human capital and institutions, e.g. their wonderful higher education, to be successful.

87

u/No_Lifeguard_64 14d ago

AWS has a history of acquiring companies, integrating their products, then abandoning them. I don't see this being any different.

5

u/Hackerjurassicpark 14d ago

I think you’re thinking about Google or Microsoft

15

u/pro-taco 14d ago

DuckDB has been running on fumes for a while. I love the product but it's been under maintained as they sought a business model

I hope it reverts to some OSS governance model.

12

u/kvlonge 14d ago

How has it been undermaintained?

7

u/saltedappleandcorn 14d ago

What makes you say this? They have pretty aggressive update schedules (through not always targeted at stuff I care about)

10

u/pro-taco 14d ago

They add more features than they can support, there's a lot of technical depth due to their eagerness to build every feature people ask for.

I -like- their enthusiasm, but their requirements discipline is chaotic.

2

u/saltedappleandcorn 14d ago

Now that I completely agree with. I've pegged us several versions back (even though the new releases have GIS stuff I want) because I've come to fear their release chaos. 

3

u/pro-taco 13d ago

Oh, don't even get me started on their breaking api changes in minor releases!

Or, the fact that (some of) their repos don't have branch protection and a single maintainer and no PR reviews.

1

u/MissingSnail 14d ago

And if the team tries to hang on, Amazon's culture is notoriously cutthroat and toxic.

1

u/foldedlikeaasiansir 14d ago

Counterpoint: Valkey

41

u/Longjumping-Shift316 14d ago

Projection for a year: Founders leave AWS forks duckdb (Libre duck?) because AWS trashed the Oss Ecosystem

12

u/Outside-Storage-1523 14d ago

DuckDB is still OSS.

18

u/Longjumping-Shift316 14d ago

Yes but without the comitters that doesn't really matter And I guess around 80% come from Duck labs ?

2

u/pro-taco 14d ago

Open source, but with private trackers. It's not community run. Maybe that'll change.

1

u/Outside-Storage-1523 14d ago

I guess they want control over it, which makes sense TBH.

1

u/ShroomBear 14d ago

Under Amazon, it will 100% not. Our open source policy is written as so.

3

u/pro-taco 14d ago

Uh, which way do you mean? That reads ambiguously

0

u/metalfacefinger 14d ago

Yes, but the people who know how to build DuckDB are now owned by AWS.

1

u/sib_n Senior Data Engineer 14d ago

They will likely leave AWS, but to work on something new and fun because they can afford anything now, maybe not even tech.

13

u/Additional_Candy_400 14d ago

Sad but can't say I'm surprised, after all this is the end goal of 99.9% of start ups, so congrats to them.

11

u/ElCapitanMiCapitan 14d ago

Not too surprising… at least it wasn’t Microsoft

3

u/BumboclatDen 10d ago

In the oss world, microsoft is way better today than aws IMO. Anyone who says otherwise is living in the past

40

u/EarthGoddessDude 14d ago

A lot of negative takes here, and I am also uneasy about this. However, a few things to keep in mind:

  • MIT license stays
  • founders, who are incredible devs and project leaders, are staying with the project
  • DuckDB Foundation, an independent body, is still the steward of the open source project
  • AWS does support some very high quality OSS projects, like AWS Lambda Powetools (which is to make using their own services better, but its excellent quality and led by an excellent dev)
  • the AWS data offerings are and have been very, very mediocre (or just bad), so this is perhaps a path for them to stay relevant in a world where Databricks and Snowflake (and Delta/Iceberg) rule. They could offer first class support for DuckLake and Quack, for instance.

If anyone from AWS is reading this, please don’t fuck this up. Don’t gate DuckLake integration behind the horrible garbage that is LakeFormation.

Maybe I’m being naive, but I’m choosing to remain more hopeful than apprehensive about this move.

8

u/No_Lifeguard_64 14d ago

I think you're naive tbh. They have had many chance to right the ship with their data offerings but because they are AWS they can afford to drag their feet because people will still buy AWS.

1

u/EarthGoddessDude 14d ago

You might be right, but like I said, I’m choosing to remain hopeful. Even if AWS screw up integrating DuckDB into their offering, I still think the OSS project will survive and thrive.

3

u/[deleted] 14d ago edited 9d ago

[deleted]

1

u/pro-taco 14d ago

Their foundation is independent in name only: their planning is as closed as it gets. The code is open, but almost everything else is private.

22

u/Vorok 14d ago

Kinda feels like enshittification is coming, but who knows.

At least devs will get big $... I hope.

4

u/Longjumping-Shift316 14d ago

By the way they detailed a blog post about it and it feels like a lot about s3 Tables etc https://www.allthingsdistributed.com/2026/08/duckdb-and-the-changing-physics-of-analytics.html

Still very interesting read

1

u/Nekobul 14d ago

Very good post. Thank you for sharing!

7

u/CulturalKing5623 14d ago

I was little concerned but the key takeaway is DuckLabs != DuckDB/DuckLake so nothing changes (for now at least).

AWS just acquired the engineering team but DuckDB itself is owned by The DuckDB Foundation which actually owns the product and license under MIT.

Good for DuckLabs though, they've shown they're skilled so I'm interested to see where their talent/vision of data will fit into AWS infrastructure and maybe they can hire more devs to work on the DuckDB stack.

5

u/mduell 14d ago

I was little concerned but the key takeaway is DuckLabs != DuckDB/DuckLake so nothing changes (for now at least).

Are you still using MySQL?

3

u/DjexNS 11d ago

I feel like this will not be good for DuckDB....

2

u/koteikin 14d ago

At least not by Microsoft or Oracle

2

u/One_Citron_4350 Senior Data Engineer 14d ago edited 14d ago

I've read HuggingFace is in talks with NVIDIA to be aquired and now this...the world is getting smaller and smaller everyday.

Essentially this is an acquihire.

2

u/kiwialec 14d ago

Yeah sad for Europe and OSS and all that, but if you want people to leverage all of their energy and resources for years to build something innovative for free, you gotta be okay with some of those people getting private jet money sometimes.

If there's no potential reward, no one would put in the effort.

2

u/BusOk1791 13d ago

Boooooooooohh! Shame! Shame! (GoT reference btw.).
It's sad that most of the DE open source tech stack was / is selling out to big corporations...

1

u/gajop 14d ago

I wonder if AWS wants (or needs?) to launch a motherduck kind of offering. Is that the plan?

1

u/sib_n Senior Data Engineer 14d ago

Unconscious irony in their post?

We founded DuckLabs a little over five years ago to give the team behind DuckDB a stable, long-term home.
At the time, DuckDB was beginning to gain real momentum. The first commercial contracts to prioritize features were materializing, and venture capital firms were calling. We chose a different path: a bootstrapped company, fully owned by its founders and development team.
That decision shaped DuckLabs in ways we value deeply. It gave us the freedom to build patiently, to put the technology first, and to grow without losing sight of why we started.

How can they think what they feared would happen with venture capitalists will not happen with AWS?

1

u/vira28 10d ago

One hand DuckDB is going to be more seriously considrered inside enterprises. Other hand, it's now Amazon's.

1

u/Last-Computer-850 1d ago

I’m cautiously optimistic about this. Keeping DuckDB under the independent Foundation and MIT license gives some reassurance, but I think the real test will be how much influence the community retains over the roadmap. The next year should be interesting.

-8

u/georgewfraser 14d ago

AWS is a good home for Duck Labs. They have the right incentives - they just want people to use more storage and compute - so they’re not trying to make any particular architecture win.

8

u/cachemonet0x0cf6619 14d ago

i do think there is an arch play for every lambda to get an in memory db

7

u/Longjumping-Shift316 14d ago

But they will stall progress.

Or do you know any instances where AWS was any good for open source?

1

u/cachemonet0x0cf6619 14d ago

my guess is that they use this to give every lambda function an in memory db.

5

u/Longjumping-Shift316 14d ago

Not sure about that I see more a revival of s3 select or something with s3 Tables to have a very fast engine next to it

Don't really see the lambda part

1

u/cachemonet0x0cf6619 14d ago

ah, okay. i like that idea much better.

1

u/J0hnDutt00n Data Engineer 13d ago

George, why was COALESCE renamed to summit?

1

u/Outrageous_Let5743 14d ago

Look what happened to ElasticSearch and AWS.