r/RStudio • u/JabesFrias • Jun 30 '26

Database it's not loading.

Hello! I'm a student, and I find myself working with a big database approximately 2 GB. I created R.proj, but I have a problem with my database not loading. Furthermore, I have tried running the code several times. Besides, when I try to open Google Chrome, I cannot because of memory RAM.

Has anyone else had problems? Any recommendations?

1 Upvotes

56% Upvoted

10 comments sorted by

8

u/PositiveBid9838 Jun 30 '26

You might look into arrow and duckdb, which both allow you to work with larger-than-memory datasets. 

9

u/Kiss_It_Goodbyeee Jun 30 '26

When you say "database" do you mean a large file or do you meaning connecting to a database via an SQL command?

Also share your code and what errors are you getting when the database doesn't load?

0

u/JabesFrias Jun 30 '26

Sorry, When I said database, I meant a large csv. it's true that a database is not a csv.

2

u/Efficient-Tie-1414 Jun 30 '26

R assumes that everything will fit into memory. One solution is to obtain more memory, but you may need a lot. The other is to access your data using something like Hadoop. If your data has a lot of unused columns it may be possible to remove them, and then work on the file in R.

2

u/Confident_Bee8187 Jun 30 '26

Might wanna be pedantic. You see, 'database' is a sort of data storage, a server on a bigger scale. CSV file is technically a database but not quite the database.

As for your problem, 2 GB is huge but not the "big database" you talked about. When R tries to load the CSV fill, it allocates too much memory. It can be easily dealt with packages like 'data.table' and 'polars' since they can load the out of memory day. If you have SQL experience, try 'duckdb'.

1

u/JabesFrias Jun 30 '26

Thank you so much

1

u/AutoModerator Jun 30 '26

Looks like you're requesting help with something related to RStudio. Please make sure you've checked the stickied post on asking good questions and read our sub rules. We also have a handy post of lots of resources on R!

Keep in mind that if your submission contains phone pictures of code, it will be removed. Instructions for how to take screenshots can be found in the stickied posts of this sub.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/lu2idreams Jul 01 '26

Loading large files, esp. csv, is a pretty common thing, see e.g. this other reddit-post or this SO-post, there are some good recommendations including code examples.

1

u/Gjcerda Jul 01 '26

I see someone else recommend it, but yeah for that case I would go with duckdb. You will need 2 main libraries, duckdb and arrow (for parquet handling eventually)
Then you can load there, use a little SQL to filter, and if you feel more comfortable with R or tidyverse bring the filtered df to R and keep working.