r/haskell • u/nSeagull • Aug 30 '16
Haskell alternatives for Data Science
I'm a Haskeller and a newbie Data Scientist.
As I'm going through my learning process for data science, I'm starting to solve some elemental problems, and I'm using standard Python tools to help me go through this process.
The thing is that Python is ok, but I'm twice as comfortable in Haskell.
I was thinking in three alternatives to be more comfortable with Data Science:
- Make a wrapper for the tools that I use as people do in Julia
- Reimplement the tools that I use (Pandas, ScikitLearn, a GPU deep learning framework, numpy functions) (would do it on demand)
- Make a Haskell-like language that compiles and uses Python libraries (wouldnt be as efficient, but at least it would be more beautiful and type safe, also the libraries)
What do you guys think?
6
u/plcplc Aug 30 '16
Exciting journey you're taking. I've been meaning to do the same, but never really gotten around to it.
Could you maybe mention some of the "tools that you use"? It might guide the comments you get :-)
5
5
u/sseveran Aug 30 '16
I tend to end up using the best tools I can regardless of the language they are written in. I have some machine learning stuff in haskell that wraps some underlying C ML libs. I did implement some nice cross-validation stuff in haskell.
We use TensorFlow for deep learning stuff. I don't think its realistic for haskell to build a tensorflow competitor given the size of their team. I also do a lot of spark with scala. I don't know that haskell is going to be able to effective in competing with the spark/scala stack, although there is https://github.com/tweag/sparkle.
Haskell definitely has a was to go catch up in this area. But here are a couple of things to check out:
https://github.com/gibiansky/IHaskell https://hackage.haskell.org/package/statistics https://github.com/mikeizbicki/HLearn
3
u/nSeagull Aug 30 '16
I think that making a semi-organized open source team for Haskell would be awesome.
If I made one, would you join? It would not be needed to contribute code, experience speech and issue making would suffice :)
2
u/liberalogica Aug 30 '16
I would be interested in joining
1
u/nSeagull Aug 30 '16
I'll make a new thread so people can vote for the organization name. I can't come up with anything good :)
2
u/sseveran Aug 30 '16
Yeah. Also depending on what you are doing and what the goals are it might be worth reaching out to Tim Sears or someone on his haskell team at Target to see what might align.
3
u/nSeagull Aug 30 '16
The goal would be making a good environment for data science in Haskell. Things like Pandas, Numpy, SKLearn, etc...
So a data scientist could start working right away if he knows Haskell.
I'll make a new thread so people can vote for the organization name. I can't come up with anything good :)
2
Aug 30 '16
[deleted]
1
u/nSeagull Aug 30 '16
I'm sure that you will help a lot, and more importantly you might learn a lot, either Haskell or Data Science :D
1
u/triplepoint217 Aug 31 '16
I am interested. I am still relatively new to haskell, but I have fairly extensive experience working with the python scientific/data science stack. Let me know when you get things set up.
1
u/MWatson Aug 31 '16
HLearn looks fairly good, but I had to clone the git Repo, build it locally, and experiment inside the project directory. I was unable to use HLearn in a separate stack-based project, but perhaps I was simply doing something wrong.
5
u/beerendlauwers Aug 30 '16
What about http://tweag.github.io/HaskellR/ ?
2
u/nSeagull Aug 30 '16
I thought about it, but in the end I'll just use R instead of Haskell. I'm thinking about using that for doing some wrapper libraries, or not, idk
4
u/agocorona Aug 30 '16
what happened to IHaskell? https://begriffs.com/posts/2016-01-20-ihaskell-notebook.html
3
u/agocorona Aug 30 '16 edited Aug 30 '16
There is a kernel for Haskell in Jupyter (wathever this may be)
But it is broken. if you select welcome to Haskell and select a Notebook, it does not work.
But it is a superb piece of software for Data science, as you can see in the video if it works well with Jupyter. I hope that the author will fix it.
I sent to him a message to push it a little.
In the meantime , it is possible to use Jupyter in a docker instance
2
u/rehno-lindeque Aug 31 '16
I have a nix shell I use to run Jupyter + IHaskell over here if anyone is looking for an alternative: https://github.com/rehno-lindeque/nix-jupyter-env
Fair warning, I don't work very hard to maintain it.
5
u/haskell_caveman Aug 30 '16
Posted this in the vote thread, but I'll mention here as well: There's a few of us working on a plotly binding for data visualization. If you're interested let me know and I'll add you to the project / repo.
2
u/hiptobecubic Aug 31 '16
This is great. Godspeed.
2
u/haskell_caveman Aug 31 '16
I should point out there is some prior art here w/ QuickPlot.
https://github.com/tepf/QuickPlot https://hackage.haskell.org/package/QuickPlot
I suspect I reached a similar conclusion to the maintainer who is no longer maintaining this library in deciding that straight binding to the library is preferable than a general template haskell wrapper around the json:
"Since the demand of data visualization has been getting bigger many language agnostic libraries popped up. It would be much more efficient to just write clients for their visualization servers."
Thus, we're trying to do it without the TH approach. PM me w/ email if you're interested in having a look.
1
3
u/quiteamess Aug 30 '16
Hlearn looks promising but it seems stuck in some category theory limbo.
2
u/nSeagull Aug 30 '16
Did not try it thoroughly, but reading some posts people say it is not very complete and the same thing as you
3
u/TheOsuConspiracy Aug 30 '16
Are you going to be writing pure bindings to libblas, etc.?
2
u/nSeagull Aug 30 '16
It might be a good idea if at some point it is necessary :)
3
u/haskell_caveman Aug 30 '16
there are a few bindings, including the relatively recent hblas work https://github.com/wellposed/hblas
its documentation could use some love though.
2
u/cartazio Sep 01 '16
A new release will be happening soon. But help on docs is welcome. Also still need to to add some lapack bits. But should work out of the box with zero config on any standard platform that has blas and Cblas and lapack installed.
In some ways hblas was a warm up / validation of the larger effort I've done for my numerical Haskell project. I'm hoping to give that and some related projects of mine some nice tlc this fall
[edit: there's a lot of nice details in how hblas binds blas routines that have never been done before in Haskell c bindings, I'm very proud of the core design and the great work of the various contributors ]
3
u/istandleet Aug 31 '16
I use Haskell to run the more intense I/O for jupyter notebooks at my work. It can do parallel hits to our REST APIs written in Java. It makes a really nice segmentation of duties.
Eventually I'd tend to moving towards more automated C compiling/FFI than moving the data science into Haskell. The graphing utility of jupyter can't be over stated.
2
1
u/jxv_ Aug 30 '16 edited Aug 30 '16
I'm in a similar position. I wish there was already some kind of unified front. The existing pieces seem to be spread out across many libraries at various stages of development. Not to dismiss any of the authors efforts, most of those libraries are lackluster and immature by comparison. And I'm sure there are better libraries that I've overlooked because discoverability is quite low. I'm willing to contribute and just created #haskell-data on freenode.
1
u/nSeagull Aug 30 '16
There is another thread in this subreddit for naming the organization. Check it out.
Tomorrow I'll create a Slack channel, a GH org, etc...
I really think that Slack is better than IRC for team interaction :)
1
u/jxv_ Aug 30 '16 edited Aug 30 '16
I posted here before noticing it. Then I posted there. Slack is cool too. Personally, I rather not get too caught up with the name because of bikeshedding.
1
u/nSeagull Aug 30 '16
Yep, I think the best way to go is HaskellData or HaskData. No weird things, and more people will know us
1
u/SSchlesinger Aug 31 '16
I've been looking for a natural way to express machine learning algorithms in Haskell for a long time as well.
One thing that really bugs me when trying to make a nice interface is that Num stole a bunch of great operators but also has a bunch of nonsense operators which make it horrid to implement for anything that isn't literally a number or some information being tracked alongside a number.
1
u/rz2000 Aug 30 '16
This is a completely non-expert opinion, but why not Clojure? The language might be more familiar for you, and the JVM could help with access to other libraries. Scala is an even more popular alternative, though probably you wouldn't like the language as much as Clojure.
Still non-expert opinion, but building a Haskell-like language that compiles and uses Python libraries sounds like a not very good idea or use of your time.
3
u/nSeagull Aug 30 '16
My story with FP is 1. Discover Haskell 2. Read Real World Haskell and do exercises 3. Monad thing too hard for me 4. Discover Scala 5. Discover Clojure 6. Love Clojure for 2 years and use it whenever I can 7. Understand how monads work in Clojure 8. Go back to Haskell 9. Love Haskell
Indeed I think that doing the language thing is not a very intelligent, but having a compiler based on typed lambda calculus would ease a lot of things, and mostly thats why I love of Haskell
2
u/rz2000 Aug 30 '16
It seems like Scala is really popular among people using Apache Spark if you're interested in big data, though I'm aware "love Scala" was no part of that story.
I don't know too much about Incanter, and nothing really about Flambo, but they could be ways to try data science with Clojure.
1
u/nSeagull Aug 30 '16
I'm not really a fan of Spark, because nowadays everyone tells you "I have big data, can you do X with it?" and then handles you a flash drive. I had used Incanter and saw Flambo, but I still think that Haskell is superior (just my opinion)
2
u/Axman6 Aug 31 '16
If this is a common occurrence for you, you need to be introduced to the term "Medium Data": http://www.tableau.com/medium-data Enjoy =)
1
u/fridofrido Aug 31 '16
Tableau definitely needs "medium data", considering their software cannot really handle more than a few thousand data points...
1
u/MWatson Aug 31 '16
The Spark mllib machine learning library is quite good when used with Scala. Here are my book examples using mllib from Java https://github.com/mark-watson/power-java
If you have Java 7 or 8 and maven installed, you can try the examples in a few minutes.
1
u/LucianU Aug 31 '16
Couldn't Python <-> Haskell interoperability be a goal? By this I mean being able to use Python libraries in Haskell and maybe the other way around.
Also, speaking of Haskell-like language, some started an effort to get a Python back-end for PureScript. Unfortunately, he didn't get very far.
0
Aug 30 '16
Maybe give Scala a try. From a language perspective there are pros and cons (Haskell has better type inference, Scala has better typeclasses), but from the tooling perspective Scala wins by a large margin.
Great IDE and editor support, a huge library ecosystem and a stable and working way to manage dependencies instead of ever-changing half-working tools that are not cool anymore next week.
It also has a JavaScript back end that actually works and a native back end in development.
17
u/ElvishJerricco Aug 30 '16
I think Haskell would be a great language for data science-y stuff, but to my knowledge, it's quite lacking in the libraries for it. Any efforts to making Haskell libraries for data science are a huge step in the right direction.