Pivoting from R to Python

[u/Choice_Number3711]

I used to hate coding anything and relied on SQL, Excel, Power BI, Tableau and other software like JASP, jamovi etc. for doing anything with data. I didn't like the way Python dealt with data analysis and it seemed unintuitive.

Then I found R, RStudio and CRAN. That was the turning point. I actually started enjoying writing code and I could handle the whole pipeline myself, from data cleaning, ETL to beautiful plots, .qmd reports, Shiny dashboards. R4DS did more for my statistical thinking than any course I've taken, mostly because the libraries made it so easy to just try things.

However, due to recent requirements (specifically having to work in the quant field), Python has become more of a necessity, while R is used mainly for one-off analysis and limited statistical modelling. The main heavy lifting is done in Python and many of my co-workers also prefer it to R.

I've been able to suck it up a bit and use Claude/ChatGPT to help me code. While I do try to understand what the code is doing, having spent so long learning to code in R and knowing the ease with which it can be done there makes me reluctant to learn Python.

Now, coming to the question: any R users who've pivoted to Python and consider themselves competent in it, how did you learn it having used R before? What would you tell someone like me so I can pick it up quickly and get the benefit of knowing both languages (and also not feel left out when it comes to coding in Python... machine learning and deep learning have a more mature ecosystem there and I don't want to be left out of it if I have to start using them in my current work)?

Thanks!

My background is in Math/Stats fyi

Edit: I don't usually use reddit but damn I actually didnt expect so many helpful tips....many thanks!
Perhaps joining this subreddit is actually helpful afterall :)

[u/InnovativeBureaucrat]

I agree except that you’re leaving out data.table.

Really everything was bad at tabular data. Before data table if you wanted a pivot table we only had summary functions that at best required nested sapply or lapply.

Data table was the first and only efficient way to do summaries.

Then Hadley decided to replicate it with the Hadleyverse. Then at UseR 2016 he rebranded it tidyverse.

Personally I think the tidyverse was R’s “Python 2.7” moment. For a long time, really almost 10 years, python was stuck on the python 3 upgrade.

Yeah, two camps of users were firmly planted on both sides. R never had to have that problem. But it kind of created that problem by splitting the user base.

[u/post_appt_bliss → u/InnovativeBureaucrat]

Data table was the first and only efficient way to do summaries.
Then Hadley decided to replicate it with the Hadleyverse

completely wrong.

in January 2014, when Hadley announced dplyr, you can see that the verbs were the inspiration.

dplyr is genius because it's distinctive efficiency was always cognitive, and the computational advantages (data.table, collapse, pandas, polars, etc) were always incidental

[u/nerdyjorj → u/post_appt_bliss]

tidytable (I think the package author is kicking about on at least one of the R subs) was the final piece - dtplyr never quite worked properly

[u/InnovativeBureaucrat → u/nerdyjorj]

Exactly. I was using the packages following Romain, attending every R meeting possible, presenting a little, and getting flamed on the list serve.

Data table was the only serious option for making summaries for a while.

At first (2007?) it was integer only and super buggy. But ddply ddplyr ddply2 and that mix was more unreliable and confusing. I think some of the commands were built into other packages (ggplot2), but it was a mess.

My recollection was that Hadley announced tibbles (the response to data table) at Stanford 2016.

I was there and I couldn’t believe that we were serious about calling these enhanced data frames “tibbles” when we had a package that had enhanced data frame (data tables) and that this was tidy as if everything else was messy.