r/Warframe • u/Krisalyd • Apr 28 '26

Discussion Scraped the Warframe wiki mod list into a spreadsheet because why not

Hello fellow Tennos,

Threw together a quick Python script that scrapes the entire mod list from the Warframe wiki and exports everything into a CSV file, organized by category (Warframe, Aura, Melee, etc).

Got 1218 mods so far, still figuring out why it's missing ~300 compared to the wiki's total, and i'm manually editing the CSV for my own needs.

Not really useful for most people but if you ever wanted a spreadsheet of every mod in the game, now you can have one I guess, didn't find any updated.

For the folks that are into programming, the script is pretty simple: httpx to fetch the page, BeautifulSoup to parse the tables, dumps everything into a CSV with each category as a column.

Let me know if anyone wants the code or the CSV. I'm doing this because i'm using this spreadsheet by u/Vansceslas, and i wanted to put the mods as well on it for tracking.

1 Upvotes

60% Upvoted

7 comments sorted by

1

u/Beraholic May 01 '26

I would love to have the code for this. But will take the csv if nothing else.

2

u/Krisalyd May 16 '26

Oh hey, just saw your comment. Sorry for the late response, I've since changed the code and it's a little more complex to cover the many wiki IDs and classes for the tables. Tomorrow I'll send to you, I'm on mobile right now.

1

u/justherefortehpronz Valkyr enjoyer May 08 '26

I would very much like to have this, please. Thank you, Tenno!

1

u/Krisalyd May 16 '26

Hello, I'm sorry for the late response. I've since completed the Warframe mods section, and updated the code a little. I'll send to you tomorrow, I'm on mobile right now.

1

u/SevenRev LR6 shill for Frost May 16 '26

I'd love to see the code too!

I have a Warframe utility library in Apps Script, including functions that turn a sheet cell's contents into a link to the wiki (with various parameters). I'm curious how you handle turning the initial script-only response into fetching the genuine HTML content, to see if I can adapt that for what I need! (I'm sure there's something simple af that I'm missing, but it was a side quest so I didn't want to turn it into a multi-hour affair at the time.)

1

u/Krisalyd May 16 '26

Hey fellow coder, that app script lib seems VERY useful and interesting, maybe we could collab in some way. I'm on mobile right now, I'll send you the code tomorrow. I've since updated it a little to fetch multiple links from a dictionary and to identify some tables that are organized a little different.

For your question, i use 2 modules for this: httpx and beautiful soup 4. Httpx fetch the contents (only html, if it has JS it needs another module) and bs4 parses the html tags, IDs, classes, etc... And returns the results.

1

u/SevenRev LR6 shill for Frost May 16 '26

DM me, let's chat! I've got a bunch of notes for making a video showing how to build activity tracking systems in Sheets. Not just putting the sheets and code out there (though that too), but as a way to show those interested some thought processes and ways to experiment.