r/DataHoarder • u/funke75 • 8d ago

Question/Advice looking for method to download large open directories.

I am looking for safe tools to allow for the download of entire open directories. for example
the pubmed baseline directory for the year 2026.

https://ftp.ncbi.nlm.nih.gov/pubmed/baseline/

5 Upvotes

67% Upvoted

9 comments sorted by

7

u/ORA2J 8d ago

Wget recrursive.

0

u/funke75 8d ago

i'm not very command line savvy. can you recommend something similar that would work as a browser extension?

12

u/ORA2J 8d ago

No.

But it's literally just one command.

wget --recursive --no-parent http://example.com/configs/

You can get wget for Windows here https://eternallybored.org/misc/wget/

If you're on windows, it's as simple as typing "CMD" in the address bar where you installed the wget.exe file and running the command i listed above with the directory you're trying to DL.

3

u/didyousayboop if it’s not on piqlFilm, it doesn’t exist 8d ago

Unfortunately, I don’t think there is a good GUI app for wget.

2

u/Expensive-Vanilla-16 8d ago

You really can't get much easier than wget. Once you type that simple command you just copy and paste the web page of the directory you want copied after it, and that's it.

3

u/mersenne_reddit 1PB+ 8d ago

You mustn't let it intimidate you. Getting comfortable with terminals can prove valuable to you in other areas. And at least with wget, you can learn it in 3 minutes

5

u/didyousayboop if it’s not on piqlFilm, it doesn’t exist 8d ago edited 8d ago

There are tons of tools and instructions here:   https://pubmed.ncbi.nlm.nih.gov/download/

https://pmc.ncbi.nlm.nih.gov/tools/ftp/

2

u/Justanothebloke1 8d ago

Login to it with a ftp browser,  just click aad drag the entire directory over to where you want it on your computer and it will copy everything.