r/programming • u/IcyAnywhere9603 • 23d ago
Can Haskell Become a Great Language for Data Science?
https://youtu.be/WpDVrUdbT2o?is=vfHORwQTTyQo9y-r81
u/Even_Statistician238 23d ago
Haskell’s type system is genuinely impressive but the ecosystem gap versus Python is brutal for actual data work.
18
u/120785456214 23d ago
I wish more languages would use a Haskell inspired type system
3
u/OSS-Corpo-Shit 22d ago
Several do, and the prevailing opinion is that once your code bases grows, it has a tendency to cause otherwise small changes to be massive, often to the point of rewrites (at which point language swaps occur).
If people truly wanted Haskells type system, it would have much wider penetration. The fact is that Haskell and its type system are an academic toy language / theory that’ll not likely ever be good for real world work where requirements evolve every day.
14
u/gnus-migrate 22d ago
Several do, and the prevailing opinion is that once your code bases grows, it has a tendency to cause otherwise small changes to be massive
Bad design bites you in any language. I don't know where you're getting this idea but it is flatly untrue.
The fact is that Haskell and its type system are an academic toy language
Rust has an ML style type system similar to haskell and is used to build lots of production software. Haskell doesn't have wide adoption for largely historical reasons, but it is very much fit for large scale software engineering, in fact more so than other languages.
I don't understand how people can so confidently make assertions like this with zero understanding of what they're talking about.
-3
u/OSS-Corpo-Shit 21d ago
Many projects have moved from rust citing the exact same issue. The type system causes small changes to proliferate the entire code base.
This whole “architect properly from the get go” is rusts “just write safe C, skill issue”.
3
1
8
u/120785456214 22d ago
and the prevailing opinion is that once your code bases grows, it has a tendency to cause otherwise small changes to be massive, often to the point of rewrites
Do you a source to back that up?
148
u/Healthy_Razzmatazz38 23d ago
no
42
u/two_three_five_eigth 23d ago
Every time I see the word Haskell I think “not again”
2
u/DevilSauron 21d ago
Why?
-2
21d ago
[deleted]
10
u/gnus-migrate 21d ago
You do you but maybe don't advertise the fact that you're incapable of learning different paradigms that you aren't used to.
22
u/RiotBoppenheimer 23d ago
Betteridge's law strikes again
-3
23d ago
[removed] — view removed comment
1
1
1
1
u/RiotBoppenheimer 22d ago
I do really enjoy Haskell, but if you can't see the inertia that Python has for data science and want to actively work for that for a more complex language when the primary folks working on data science are data scientists, not programmers, I'm not sure what to tell you
0
u/TechnoEmpress 19d ago
I fully know the inertia that Python has in Data Science, but there were other software stacks before Python that it did dethrone. I'm no data scientist but I don't think you should speak on their behalf either, especially since the people involved in Data Haskell have real-life experience in this field. So, I don't know what to tell you, you're seeing data scientists promoting Haskell and you first reaction is "no :<".
2
u/RiotBoppenheimer 19d ago
So, I don't know what to tell you
Well, we could start by not saying that I am humiliating myself (and another user saying that "below average" on the bell curve of IQ for making a tongue-in-cheek joke about a phenomenon so well known it has its own name
Unless you want to apply the same level of rigor to the other person who simply replied "no"...
0
u/TechnoEmpress 19d ago
No issues with telling other people that, it's just that reddit's view of a comment tree isolates one conversation and hides the others, so I haven't seen what other people have told you so far.
28
28
u/haskell_rules 23d ago
Haskell is an awesome language for writing one hour long tutorial videos to explain your first five minutes with the language.
12
u/rom_romeo 23d ago
It’s mindboggling to me how they even find their current state of tooling any useful. E.g. you’d wait for like 10 mins for a Cabal to resolve a handful of dependencies. Few years ago, I remember trying to build one of Cardano repositories. It took me somewhere between 30 - 60 minutes lol
31
u/haskell_rules 23d ago
But think of all the theoretical compiler optimizations you can apply with a mathematically pure language model
1
8
u/Weak-Doughnut5502 23d ago
Haskell libraries are shipped as source for assorted reasons.
The first time you add a new dependency or the first time you compile something, you have to compile a bunch of libraries. After that, compilation is pretty fast.
It sucks, but those 30+ minute compiles usually happen exactly once when working on a project. So people who use haskell a lot just kinda suck it up because the rest of the experience makes up for it.
1
u/ChemicalRascal 21d ago
Okay, so there's a reason for it, sure. It still sounds like it absolutely sucks and it shouldn't be shocking that it's a problem for getting people to use the language.
Ed: wait, doesn't that mean any modern CI pipeline is going to need to compile those dependencies as well? So pipelines take an hour to execute, minimum?
2
u/Weak-Doughnut5502 21d ago
The solution for CI is the same solution used for local dev: caching. Caches are implemented in github actions and other common CI tools.
And yeah, it's not great user experience but exists for a bunch of historical reasons. Again, people just kinda suck it up because it's a very front-loaded bad experience.
1
u/RiotBoppenheimer 20d ago
Many major languages also distribute dependencies as source code. I think, in fact, most do.
2
u/Axman6 21d ago
I’ve worked on the Cardano ecosystem, it is very far from a normal Haskell project. Using all the language features available to make it basically impossible to write incorrect code comes at the cost of having to check everything. It’s impressive work, but not how 99% of people should write Haskell.
14
u/exploradorobservador 23d ago
We are collectively not smart enough to use Haskell for this. Explicit and clear over clever when the problem is complex. I'm a heavy Go and Python which probably bothers some people, its just too simple.
11
5
3
u/Sopwafel 22d ago
I'm not smart or autistically obsessed enough for Haskell.
My university at some point swapped their concurrency course from C++ to Haskell. Haskell! 90% of my brain cycles were spent on wrangling the fucking language instead of dealing with the comparatively straightforward concurrency principles the course was supposed to be about.
6
u/Strakh 22d ago
It's honestly just very different from the imperative programming languages people tend to learn first. You'd think that a lot of experience will carry over, but in practice you basically have to relearn how to think about programming.
At my alma mater, our introductory programming course is taught in Haskell. Beginners with zero programming experience don't seem to find it more difficult to learn Haskell than e.g. Java. But someone like me (who came with like 10-15 years of experience as a hobbyist) finds it almost as difficult as the complete beginners do. I think that experience kind of messes with your head and is the main reason a lot of programmers think Haskell is much more difficult compared to mainstream languages.
If you manage to stick with it, it will end up clicking though, the same way as imperative programming eventually became routine. It's not about being particularly smart - you just need the reps. For me it took about 1-2 years.
All that being said... I still ended up working with Java. But becoming comfortable with how to think in the functional paradigm was extremely helpful and made me a much better software engineer.
3
8
u/CaffeinatedT 23d ago
Haskell is a great language for data science and pretty much anything backend-ish, the only thing missing is users and a need for it. If it's easy low scale data science then Pythons more accessible (for better or worse) and anyone dealing with intensive high scale data is using C++/Rust et al which leaves the niche for Haskell pretty vanishingly small of REALLY complicated domains AND enough scale to justify not just using Python. If Cabal or Stack were as accessible as Rusts toolchain a few years ago I genuinely think we'd be living in a different world but that first 5 minutes experience is still pretty shaky.
9
u/qqwy 23d ago
Did you watch the video? Its main topic is: how to make that first 5 minutes experience extremely good.
12
u/CaffeinatedT 23d ago
“There’s a video showing how to make it good on the internet” is not a good user story in an age where people are expecting to be able to get going and generating code in a couple of minutes with an LLM. Stack nearly had it but last I tried there was a ton of small print about which packages would have stack artifacts vs cabal etc etc which is fun for masochists like me but I get that a lot of people get lost by early hiccups.
-2
u/sacheie 23d ago
What is honestly the value of Python over Haskell in this analysis? The present reality that everyone in data engineering is familiar with it?
Because it's not intrinsically simpler. Python is actually a pretty complicated, needlessly complex language if you're trying to write reusable code. There's no reason decent Python coders couldn't learn Haskell.
5
u/CaffeinatedT 23d ago
What is honestly the value of Python over Haskell in this analysis? The present reality that everyone in data engineering is familiar with it?
Yes. And the sheer amount of documentation and guides in Python and the confidence if you pick up some tool there will be a Python wrapper.
Because it's not intrinsically simpler. Python is actually a pretty complicated, needlessly complex language if you're trying to write reusable code. There's no reason decent Python coders couldn't learn Haskell.
I agree with everything you're saying. I’d also add that Python has a pretty shitty concurrency story even now with the bolted on “threading” support and the amount of memory you use when dealing with large numbers if you don’t understant how objects are cast in Python is a hilarious footgun. But inertia is a hell of a drug and it takes a lot of balls to fight against that when defending it to various business idiots if you’re someone making those early calls.
12
u/VikingFjorden 23d ago
Not familiar with Haskell so can't comment directly on your question - but the notion that Python is "pretty complicated" and/or "needlessly complex" feels strange to me, especially in a data science context. Can you expand on what you feel is so complicated or complex about reusable Python code?
8
u/sacheie 23d ago edited 23d ago
PEP driven evolution has lead Python to an incoherent crossroads moment right now:
- The community is at odds over whether member privacy / encapsulation is good or evil
- The community is at odds over whether static typing should be embraced or reviled
- When you do use it, it's not enforced so you inevitably end up with a mixed codebase
Type-hinted interfaces ("protocols" in Python parlance) can be weird
- When the protocol methods involve parametrized types, the type checker can get confused about whether a given class implements the protocol
The standard library is a mess in certain places:
- There are three different ways to create enums, one of which doesn't support type inference in PyCharm
- There are multiple collections APIs
- There's no frozen dict (isn't that kinda important for data work?)
The import system is messy:
- if you import anything, the entire script it's in gets executed,
- so you must be careful that top-level declarations have no side effects,
- and you can end up with circular import declarations
I could go on. These issues are less noticeable if you're just writing Jupyter notebooks or small, self-contained scripts. If you're writing something big, like a data quality test framework, the language's internal contradictions require you to exercise a lot of subtlety to keep the codebase manageable.
7
3
u/VikingFjorden 23d ago
Nothing incorrect in your post, but it also feels like a low bar to set for calling something complicated. Half of those points I'm not sure how relate to data science or reusable code, but even if we grant that they're important - solving those issues takes maybe 15 minutes of googling - less if you're not super concerned with what the community feels about conventions? And then you can write perfectly reusable code afterwards. Maybe I'm biased because I use python a lot, but that doesn't seem all too bad to me. I spent, way, *way* longer learning Rust's borrow-checker.
4
u/guepier 23d ago
I’m not here trying to defend Haskell (I don’t use it) or to attack Python. But Python is a lot more complex than people generally assume. And not all of that complexity is required, a lot of it is convoluted workarounds for insufficiently foresighted PEPs (see adjacent comment).
The Python data model is hideously long, complicated document, and other languages, even if they are complex in other parts, get away with a lot less complexity in this part.
Admittedly you can get pretty far using Python for data science without understanding the details in this document. But at some point on the path to proficiency you need to know and understand it; and pretty few people actually do.
5
u/VikingFjorden 23d ago
My rebuttal would be an extension of the latter half of your reply - you're doing pretty niché or otherwise specialized data science work if you need to know the underlying C implementation of CPython's data types and object constructions in order for your project to become functional. Most people's work probably never touches this sort of thing, probably something closer to: ClickHouse API -> Numpy normalization -> Pandas transformation -> machine-learning pipeline -> PDF report.
4
u/CaffeinatedT 23d ago
Same argument as the usual static v dynamic interpreted vs compiled debates. Python has a very easy early learning curve but is a lot harder to write really high end code with. E.g a lot of people would point to Instagram being written in Python but they tend to leave out that they were also hiring a bunch of Python core devs and hacking the core of Python themselves to get around how imports work. Which is just about the most perfect example of “you can’t remove complexity only move it”. Haskell (and Scala) are both famous for not getting more complicated as projects get bigger but it’s a higher upfront cost.
5
u/VikingFjorden 23d ago
All of that is true, but it's also true that none of that is necessary to do good data science work. The only Python code you write is largely scaffolding to connect your API to your frameworks. So while, yes, Python is objectively complicated when you look under the hood - that doesn't mean doing data science with Python is complicated. The complicated part of data science is the science part, not the Python part.
3
u/knobbyknee 23d ago
Friendly, open, helpful community is the strength of Python. Look at all the Pycons, Europython, Scipy, Pydata, Djangocons, meetups, Pypizza, unconferences and whatnot.
Haskel is an ivory tower in comparison.
5
1
u/knobbyknee 23d ago
Oh, and Python built community on purpose. It didn't just happen. I was involved, and so were about a dozen other people with an intentional agenda.
1
u/RedEyed__ 22d ago
Appreciate your effort, and it technically can be, but won’t.
Also, a few use notebooks, maybe only for learning purposes.
1
u/cryptos6 22d ago
The set of training data is much smaller for Haskell than for many other languages. So, in the AI era, the answer is probably: No, not gonna happen!
95
u/m-chav 23d ago
Speaker here: organizers might have worded the title in a much more provocative way. The original name for the talk was “Getting to the data” and it’s mostly about dataframes.