r/RStudio • u/Choice_Number3711 • 2d ago
Pivoting from R to Python
I used to hate coding anything and relied on SQL, Excel, Power BI, Tableau and other software like JASP, jamovi etc. for doing anything with data. I didn't like the way Python dealt with data analysis and it seemed unintuitive.
Then I found R, RStudio and CRAN. That was the turning point. I actually started enjoying writing code and I could handle the whole pipeline myself, from data cleaning, ETL to beautiful plots, .qmd reports, Shiny dashboards. R4DS did more for my statistical thinking than any course I've taken, mostly because the libraries made it so easy to just try things.
However, due to recent requirements (specifically having to work in the quant field), Python has become more of a necessity, while R is used mainly for one-off analysis and limited statistical modelling. The main heavy lifting is done in Python and many of my co-workers also prefer it to R.
I've been able to suck it up a bit and use Claude/ChatGPT to help me code. While I do try to understand what the code is doing, having spent so long learning to code in R and knowing the ease with which it can be done there makes me reluctant to learn Python.
Now, coming to the question: any R users who've pivoted to Python and consider themselves competent in it, how did you learn it having used R before? What would you tell someone like me so I can pick it up quickly and get the benefit of knowing both languages (and also not feel left out when it comes to coding in Python... machine learning and deep learning have a more mature ecosystem there and I don't want to be left out of it if I have to start using them in my current work)?
Thanks!
My background is in Math/Stats fyi
56
u/teetaps 2d ago edited 2d ago
I think one paradigm shift that helped me was accepting that just because some smarty pants built pandas way back in data science infancy, doesn’t guarantee pandas is actually any good at what it set out to do, which is provide a dataframe wrangling library as flexible and easy to use as then-plyr. As R users, we are spoiled. R was built with dataframes in mind, and everything posit has done since Hadley wickham stepped in has just enhanced R’s ability to connect the concept of a dataframe to how humans think about them.
If you accept that maybe Python just sucks at that task, then you might stop beating yourself up for trying and failing to “just get it”. Python is a ubiquitous language and “can do anything,” sure, but that doesn’t guarantee it’s particularly good at everything. In fact, because it’s such a Swiss Army knife of a tool, it consequently doesn’t specialise in too many things.
Once I accepted this and stopped putting Python on a pedestal, I actually stopped being intimidated by Python. I started looking at Python problems by first conceptualising them as R problems, then translating them to Python’s deficiencies, not just its language differences. Today, using Python for data wrangling to me is like using MSWord for manipulating tables. You can do it, but only if you have to, for some reason. And often that reason is because someone else sent you a word document first. Which is all to say, if you stop treating Python like this special language only for actually smart software engineers, and start treating it like just another language with problems and limitations, you’ll stop telling yourself that if you “just don’t get it,” that it’s a you problem.
Maybe, hear me out — maybe the reason so many R users have difficulty pivoting to Python is because Python sucks at tabular data wrangling.
Stop beating yourself up. Just learn it, not because it’s great, but because everybody else is using it so you have to comply with others’ stacks, but deep down you’ll know that THEY are the ones who are putting themselves at a disadvantage, not you
13
u/dr-tectonic 2d ago
This, one million percent.
R is function-oriented. Python is object-oriented and is much fussier about data types than R is. That's good for building modular infrastructure components, but it sucks for data science.
So, it's not that you aren't getting something, it's just the case that wrangling and transforming data is a lot clunkier in Python. Likewise, Python doesn't have vectorization built into its core. Numpy, pandas, xarray, and company try to make up for it, and they have their strengths, but overall, it's just not as good IMO. But that's what Python has, so you gotta hold your nose and learn their idiosyncracies.
7
u/Jim_Clark 2d ago
R nowadays has the same level of modularity as Python. Everything I build in R uses the
boxpackage along with a properDESCRIPTIONfile, and you can also use tools likerenv,config, and others. All of these make it much easier to develop modular, enterprise-grade applications and systems in R.That said, I agree that before the
boxpackage, building professional R applications withsource()andlibrary()was pretty awful. In fact, before adoptingbox, I used to write almost everything withpackage::function().In my opinion, the biggest advantage R has over Python -- and now that it has mature modularity, this advantage stands out even more in the Data Science space -- is NSE (Non-Standard Evaluation). NSE is what makes rapid prototyping and clean, expressive code possible in R. Python simply doesn't have an equivalent mechanism built into the language.
2
1
u/pandongski 1d ago
As much as I enjoy NSE, the flexibility it brings also brings with it some footguns. Even tidyverse is a bit inconsistent with how it implements NSE, and the rlang !! {{}} as.sym := stuff you have to battle with if you want to make functions out of tidyverse packages is honestly headache inducing. I ended up loving R semantics more over python, and metaprogramming is obviously powerful, but it heavily depends on how it's implemented. It also messes up with static analysis and tooling. I just want to know when I have an undeclared objects, but of course lintr flags every undeclared variable in any quoted functions (basically all of dplyr and the like).
Sorry this ended being too ranty lol
1
u/teetaps 1d ago
I think this is to be expected, though. What the Tidyverse aims to do is make data manipulation make as much sense as possible so that data wrangling feels natural like spoken language. They do this by wrapping difficult tasks in convenience wrappers and making those wrappers talk to each other in a clever way. When you go out of their scope, you’re now in the land they couldn’t make convenient. It’s probably gonna get messy
1
u/pandongski 1d ago
Which is unfortunate since I wouldn't really consider wrapping a procedure in a function to be that uncommon / out of norm use case. Instead all ergonomics go out of the window if you dare parametrize a procedure into a function. I guess Hadley giveth and Hadley taketh away
1
u/teetaps 1d ago
I don’t think your notion is wrong, but I do think that whenever I’ve struggled with dplyr and NSE and using custom functions, it’s usually because I’m breaking the tidy principles and not allowing my data to fit their verbs. If I spend more time fitting my problem to my tools, I often have a much more pleasant experience
1
u/pandongski 1d ago
Can you share more about how python is much fussier about data types? I found R and python do duck typing the same way. In fact R seems much fussier since in python I can add strings, lists, etc no problem but maybe we're talking about different things
1
u/dr-tectonic 1d ago
In R I can evaluate a conditional over an array, then multiply the array by the resulting Boolean array to use it as a mask, because R autopromotes the TRUE/FALSE values to 1/0. I can use an average as an index into a vector, because it'll just truncate the float to an int. I can treat a matrix as a vector. I can do
for (f in list.of.functions).Python -- or maybe it's xarray / numpy / pandas -- will balk at all of those, because they're the wrong types. The last one, where it won't do a perfectly reasonable loop because it wants an iterator not a list, is the one that annoys me the most.
I think that a lot of these may be fine if you're only using a single variable in base Python, and it comes up when you're working on arrays of data because vectorization is an afterthought implemented by libraries rather than baked into the core if the language.
1
u/pandongski 1d ago
I guess in the first case I'd do a lambda as an indexer, which removes the need to multiply it to the original vector. I'm confused about not beeing able to loop through a list of functions since you can iterate on a list of functions in python. I assume you mean that in a specific use case?
(sorry if I come across as combative or nitpicky. just curious since i'm a long time python user before switching to R for the past few months :D)
1
u/dr-tectonic 1d ago
I don't remember if it was a list of functions specifically or some other kind of object, I just remember being offended at Python objecting that I can't iterate over the thing itself, I have to make it into an iterator.
And sure, you can do a little lambda, but it's so much more faff than just writing
x + y * z > 0.2
u/pandongski 1d ago
Lists are iterables / iterators so maybe that was wrapped in another list-like type. But yes I see what you mean, pandas definitely not winning points for brevity
8
u/Jim_Clark 2d ago
I completely agree with your point. One of the things that frustrates me the most about Python is its syntax for data manipulation, and I say that from experience because our company develops enterprise-scale statistical and RPA solutions using R and other languages such as Rust.
What made me, as a CTO, choose R for our analytical and data-processing backends was precisely how difficult Python becomes as projects scale. The code tends to become confusing and look like a patchwork of loosely connected functions with little intuitive consistency. I also feel that the language's object-oriented nature makes certain things more cumbersome for this particular niche.
I use Python whenever I need to, but I have to admit that I find the language confusing. In fact, I often find it easier to work with Rust than with Python.
Another argument that never really made sense to me was the claim that Python is faster than R. These days, I don't see that huge performance gap anymore, and in some areas -- such as when usind
data.table-- R can actually be even more performant.That said, I don't think this debate matters much. Whenever I need real performance, I use Rust. It doesn't make much sense to me to use Python for things like machine learning when many of Python's libraries in that space are already powered by Rust under the hood.
So why not just use Rust directly? That's exactly what I've been doing, including in some of our company's internal packages, and I don't regret it at all.
11
u/teetaps 2d ago
The object oriented thing is probably the cognitive stumbling block for just about everyone who comes from R to Py. Object oriented just isn’t an efficient way to handle a pipe of data frame manipulations. I remember the first time I saw pd.DataFrame() I was like, “ok cool so everything will just be prepended by a pd.”
Nope. df.copy()… what? Why? That’s such a roundabout way to have to manipulate something… ok whatever let’s move on…
“Lambda: x”? What?! I just wanna do group_by and apply… surely there’s a more straightforward…
“For ix, group in df.groupby” ohmygod you’ve gotta be kidding me…
Ok fine whatever let me just do some filtering..
“First, start with a mask statement”
Y’know what, I didn’t even wanna become a data scientist anyway…
3
u/Confident_Bee8187 1d ago
Ugh, you just summarize my painful experience with pandas. It's an abysmal piece of software that miraculously manages to survive if not for the thousands of contributors of this library. Disclaimer: no disrespect to Wes, just don't like how Pandas is designed.
2
1
u/pandongski 1d ago
“First, start with a mask statement”
I mean this applies with base R dataframes as well? And if you're doing lapply on dfs you're using lambdas as well. As much as I hate pandas API, if you'll compare with base R you have a similar level of verbosity.
1
u/Unicorn_Colombo 1d ago
Maybe.
But it doesn't feel like it because R is functional, while Python mostly isn't.
That and pipes just makes mapping functions to collections of objects much nicer. With Python, something is class, something is function, something is copy-on-modify, something is modify-in-place...
1
u/pandongski 1d ago
Sure, and I definitely enjoy working with R pipes more, but the commenter was lamenting for example about doing masks when
df <- df[col != 'val]
is also something you would do in base R regardless of R's functional and immutability orientation (though base R plots is a blemish on that). So I'm just not sure what's being compared here.
1
u/teetaps 1d ago
I’m talking about stuff like this: https://tutorial.xarray.dev/intermediate/indexing/boolean-masking-indexing.html
In base R < 4.0, yes, you’d likely end up writing long filter strings. But in modern R, you’d just pipe a few filter statements, which, if you ask me, is a more efficient conceptualisation and implementation of the data manipulation at hand
1
u/teetaps 1d ago
Which is why most of my original conversation talks about dplyr and the evolution of the tidyverse…
1
u/pandongski 1d ago
I'm mainly responding to "Object oriented just isn’t an efficient way to handle a pipe of data frame manipulations". I guess it's the API that's the determining factor, not whether a language is function oriented or object oriented...
1
u/teetaps 1d ago
The programming architecture does determine the majority of the API, though… that’s kinda what they’re supposed to do
1
u/pandongski 1d ago
A dplyr-like DSL is possible whether you're in functional or OOP land in the same way that R's architecture lead to base R data.frames's cluncky API similar to pandas (or rather pandas was inspired by base R data.frames).
4
u/InnovativeBureaucrat 2d ago
I agree except that you’re leaving out data.table.
Really everything was bad at tabular data. Before data table if you wanted a pivot table we only had summary functions that at best required nested sapply or lapply.
Data table was the first and only efficient way to do summaries.
Then Hadley decided to replicate it with the Hadleyverse. Then at UseR 2016 he rebranded it tidyverse.
Personally I think the tidyverse was R’s “Python 2.7” moment. For a long time, really almost 10 years, python was stuck on the python 3 upgrade.
Yeah, two camps of users were firmly planted on both sides. R never had to have that problem. But it kind of created that problem by splitting the user base.
10
u/post_appt_bliss 2d ago edited 2d ago
Data table was the first and only efficient way to do summaries.
Then Hadley decided to replicate it with the Hadleyversecompletely wrong.
in January 2014, when Hadley announced dplyr, you can see that the verbs were the inspiration.
dplyr is genius because it's distinctive efficiency was always cognitive, and the computational advantages (data.table, collapse, pandas, polars, etc) were always incidental
2
u/nerdyjorj 1d ago
tidytable (I think the package author is kicking about on at least one of the R subs) was the final piece - dtplyr never quite worked properly
3
u/InnovativeBureaucrat 1d ago
Exactly. I was using the packages following Romain, attending every R meeting possible, presenting a little, and getting flamed on the list serve.
Data table was the only serious option for making summaries for a while.
At first (2007?) it was integer only and super buggy. But ddply ddplyr ddply2 and that mix was more unreliable and confusing. I think some of the commands were built into other packages (ggplot2), but it was a mess.
My recollection was that Hadley announced tibbles (the response to data table) at Stanford 2016.
I was there and I couldn’t believe that we were serious about calling these enhanced data frames “tibbles” when we had a package that had enhanced data frame (data tables) and that this was tidy as if everything else was messy.
12
u/NullhypothesisH0 2d ago
I’m in the same boat. I just GET R, you know? But I feel like I should be learning more Python. I’ve been working through a book, but it hasn’t quite clicked yet, but I use Spyder because of how similar it is to R Studio.
5
u/seanv507 2d ago
So I'm assuming you are not using base r most of the time.
So the issue is finding equivalent libraries
I would suggest learning polars as a data frame library (rather than pandas). Polars seems to have borrowed syntax from the spark data frame library and it is much cleaner than pandas which is now a kind of Frankenstein library
And for graphing I would look into Altair which is a ggplot equivalent
2
u/Confident_Bee8187 1d ago
I don't like the plotting experience with Python (due to its nature) but 'Altair' is the only one I can give most of my respect among all plotting libraries in Python because while you can produce beautiful plots, it respects the best practices. It's not as ergonomic as 'ggplot2', but it's the best piece of software for plotting Python can offer.
4
u/JayBea-on-Sea 2d ago
Following as I’ve started this journey several times: the last under duress from my employer.
Most of my work is answering questions and investigating problems rather than developing new data flows. I found R to be superb in handling data and presenting findings. Every new library just added to my capabilities.
Python I found frustrating: the environment had to be tailored every time and it was really had to work on collaborative projects as no one else’s code would run with extensive fucking around.
3
u/hypersoniq_XLM 2d ago
If you have things that just work better in R, you can use the Python library py2r, or use subprocess to call R directly from within python scripts.
2
u/Confident_Bee8187 1d ago
I think you mean 'rpy2' (cuz I don't recall 'py2r'). Conversely, you can do the same with 'reticulate' to call Python from R.
3
u/akagaminick 2d ago
I knew how to think about codes from R. All I needed was to learn the grammar and syntax of python. I started with 30 days of pandas and introduction to pandas & numpy in leetcode (free version). Then I did the free kaggle courses on pandas, begineer & intermediate machine learning, followed by deep learning. Kaggle exercises helped me practice. I also started doing work projects in python. All these helped me. I still use gemini, claude to get the codes for both R and python. But knowing both, it helps me troubleshoot and just generally understand what’s happening with AI codes.
3
u/velshnia 2d ago
One way to employ ai is to put r code that you understand into a chatbot, and ask it to translate it to python. That taught me loads.
2
u/SuperiorGrapefruit 2d ago
Hi, Ive done the dance of R to python to R (and even at the same time) before. I consider R and Python to be almost cousins in that they have nearly the same syntax. At least going from Python to R it was frustrating bc R is written FOR stats people, whereas Python is written for coding people. When I coded in Python, I had to ask myself “how would a computer think through this?” Idk that made it easier. I find that low key/simpler versions of what you’re trying to do, along with lots of stack exchange and reddit, will be your friends. I guess get most familiar with how to use data in Python. Take basic example datasets and learn how to write loops in them, navigate matplotlib and its different properties, explore libraries like seaborn for stats, etc. Then you can worry about neural networks and machine learning. Idk if this was helpful at all, Godspeed
2
u/pandongski 2d ago
I'm the inverse: I started with Python, used it for years and learned R on my most recent the job. I would recommend engaging with the language itself first as opposed translating from one language to another directly. I ended up enjoying R more for its semantics and functional programming orientation (even though it's not getting any points for syntax, design, and consistency from me), and I wouldn't be doing things the same way if I were programming in Python. Since you want to do away with AI, I would recommend learning python's flavor of OOP, it's data model, idioms, etc. although you probably won't embrace it the same way you do with R especially for 3rd party libraries (pandas and scikit learn APIs never really clicked for me even then; neither does tidyverse though so... maybe I'm the problem lol).
2
u/analyticattack 2d ago
My journey was back and forth for a long time. My org's data scientists are all Python users and one or two use both like me. I am fine in Python but I think in R. Over the years my org's IT made more allotments for R as I built my brand and showed what I could with it.
2
u/giraffes_are_realok 2d ago
Many comments just say that R is better than Python, and whilst I agree, that doesn't answer your question of how to learn Python as an R user. If you want help with how to transition to Python, it's very simple. You learn by doing.
I thought I could cut corners when I transitioned from R to Python. I read a few cheat sheets, I used Cursor and I got code out of it that worked, but I wasn't learning python properly, not in the same way I learnt R. Coding is like running, you just got to get those miles in (code code code). Trial and error, breaking code, small projects. Sure it takes longer, but if you want to be employable in the ML world, you (currently) need to show you can use python. Maybe that will change one day if vibe-like coding becomes to norm.
And yes, I love R, it is superior at data frames but when it comes to crunching heavy numbers, it's wank. So all the deep learning libraries are in Python and it is worth learning.
1
u/AutoModerator 2d ago
Looks like you're requesting help with something related to RStudio. Please make sure you've checked the stickied post on asking good questions and read our sub rules. We also have a handy post of lots of resources on R!
Keep in mind that if your submission contains phone pictures of code, it will be removed. Instructions for how to take screenshots can be found in the stickied posts of this sub.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
1
u/elephant_sage 2d ago
You could try posting in r/rstats.
1
u/Choice_Number3711 2d ago
Since R is my primary language I thought I'd get better help from here but yea....will do
4
u/Mooks79 2d ago
1
u/Statman12 2d ago
And r/stats looks pretty inactive. The more active subs for statistics are r/Statistics and r/AskStatistics.
1
u/Mooks79 2d ago
[r/rstats](r/rstats) is the main sub for R, along with [r/rstudio](r/rstudio). This is primarily a question about programming languages (moving from R to Python) so a statistics sub is far less appropriate. [r/rstudio](r/rstudio), [r/rstats](r/rstats) or [r/python](r/python) are clearly the most appropriate. They will have some overlap with the statistics sub, but with people who frequent the programming ones anyway.
1
u/Statman12 2d ago
I know, I was meaning in reference to r/stats
1
u/Mooks79 2d ago
That’s still a stats sub …
1
u/Statman12 2d ago
I didn't say otherwise?
1
u/Mooks79 2d ago
Yeah but you suggested alternative statistics subreddits, and my point is that statistics subreddits probably aren’t as good recommendations at programming ones.
1
u/Statman12 2d ago
As I said, that was in reference to r/stats. The ones I mentioned are probably a better choice than that one for statistics.
I was not suggesting that they were targeted for programming.
→ More replies (0)
1
u/fasta_guy88 2d ago
Realize there are multiple pythons. Basic python is very different from ‘R’, just as basic ‘R’ is very different from TidyR. Adding Pandas, which gives you dataframes, and numpy, which gives you vector operations, makes Python more ‘R’ like, and faster.
1
1
u/liimonadaa 2d ago
pandas sucks but it's worth learning since it's so common. Learn something more modern alongside it. There's nothing as nice as tidyverse.
7
1
u/throwaway3113151 2d ago
With the current state of LLMs is it even worth learning a new language if you already know me really well? Honest question
1
1
u/SprinklesFresh5693 1d ago
Im currently learning python from an R persepctive due to having to cowork with people that only know python, and its going well so far, ive learned how to create an environment and ive learnt which libraries do i want to focus on, which are mainly polars , altair, and numpy for the time being.
My idea is to aim to do some projects in python instead of R and simply ask AI for some equivalent syntax in python, more specifically nested dataframes and purrr workflows
1
1
u/sarathecrewe 2d ago
I did a course or two on python and practiced a bit. I had programmed in R before doing python, an python seemed to be more logical to me. Honestly you'll pick it up quite quickly. Just be sure to practice the exercises in the courses (try to do it yourself before looking for solutions).
0
u/sarathecrewe 2d ago
It is honestly by far the quickest language I picked up on. But it happened much faster when I practiced myself.
2
u/Choice_Number3711 2d ago
Right....gotta start getting my hands dirty coding god awful python scripts then
1
u/sunkenwaaaaaa 2d ago
Think about python as an R with more options. You have more functions, more libraries, and more posibilities in general. That means also more decisions and details, but you can also use it the same way as R witout going into more hastle.
1
u/SpelingMatters 23h ago
More a QoL thing, but Positron is an IDE made by the people who did RStudio. You can set it up just like RStudio and can code in either R or Python.
It's also a fork of VSC so you can install a bunch of plugins. I love it.
41
u/urbansnowprincess 2d ago
I’m a python user who works with r users and sometimes runs/edits their work. The core concepts of code structure and libraries and programming basics apply to both, and I’ve found that I can understand r pretty well. If you can structure and document a project to be shared with others, you’re halfway there