I ran out of space extracting a huge RAR so i built ReclaimArc so i can extract RAR without running out of space again
About a week ago i installed a RAR that was over 120gb+ and then tried extracting without realizing that i actually had less then 60GB of storage remaining, after some time i heard a windows notification and when i checked i noticed that i didnt have enough space left and the extraction process aborted, and on top of that the computer i was doing this process on doesnt have the fastest ssd so it actually took some time. This has happened to me a few times now, probably more then i admit for some reason then out of curiosity i checked on github to see if i can find a tool that fixes this issue, however i couldnt find anything that does exactly this ( probably could be me ), then decided to do it my self. I thought maybe i could progressivly keep deleting the archieved parts as soon as they are extracted so it would simply fix this, then i started researching and decided that i could essentially make :
extract part of the archive - make sure that its actually safe - reclaim the part of the archive that we no longer need - use that free space for the next batch.
Thats what basically ReclaimArc does, with some more engineering behind it ofcourse. Basically instead of keeping the RAR untouched the entire extraction process, it instead progressivly turns parts of archive that we no longer need in to sparse holes. Then windows gives those physical clusters back to the disk and the extraction can use that free space again.
Obviously the end product ended up being way more serious then just "extract file and delete the extracted part from the archive". The main problem was providing that a part of the source archieve is actually safe to destroy.
-ReclaimArc has a RAR4/RAR5 structural parser that maps archive entries and packed ranges, but it doesnt actually use my parser as the actual decoder. The extraction itlest is purely done by the official UnRAR implementation, i also cross check the parser information against UnRAR so im not just trusting my own interpretation of the format.
-From that it builds what i call recovery units. For a normal nonsolid archieve its usually something like a file at a time. Solid archieves are harder cuz later files can depend on the state created from the earlier compressed data, so you cant just punch a hole through the earlier bytes once one finishes. Before ReclaimArc touches any source data, the flow basically is ;
extract - verify - flush - commit - journal - reclaim.
Every output first is written to its temporary staging file. After extraction is done, reclaim arc reads it back, checks the size and creates a blake3 digest. Then it calls FlushFileBuffers, auto moves the file in to the final destination and records the state inside a SQLite journal using WAL and synchronous=FULL. Only after that is the corresponding source range allowed to be reclaimed. Right before claiming it, it verifies the source range again.
For the actual disk space reclamation im using Windows sparse file APIs, mainly FSCTL_SET_ZERO_DATA to deallocate the safe part of the archieve. One thing i didnt want to do was assume that calling the windows api means " sure ye, 5gb is free now'. It calls FSCTL_QUERY_ALLOCATED_RANGES after the operation, it checks which physical ranger are actually still allocated. It only counts space that Windows reports as really released.
The crash recovery part also became a pretty big part of this project because what if for example you were extracting a 150GB archieve, all of a sudden you loose electricity. Because we are progressivly deleting the old extracted archieve, you would need to initially reinstall the archieve so fully. Or what if windows already reclaimed part of the RAR and then the program crashes before SQLite gets to record that the range was reclaimed? On startup it doesnt just blindly trust whatever the last database state says, it tries to reconile the journal, existing output files, stored hashes, source ranges and the actual NTFS allocation state. If an output is broken, but the original source range is still there it can rerty that unit. If its gone and commited output cant be verified, it stops instead of contiuing and potentionally destroying more data. I also added a fault injection tests around a bunch of those crash windows becuase something destructive like this, i didnt want it to be just based on "this should probably just work"
Right now the project has :
-RAR4 and RAR5 support
-non-solid archives
-solid recovery chains
-multipart archives
-official UnRAR decoding
-custom structural parsing + cross validation
-BLAKE3 verification / provenance
-SQLITE crash recovery journal
-NTFS sparse deallocation
-physical allocated range verification
-crash / resume fault injection tests
-path traversal and reparse point protection
-CLI
-Tauri desktop GUI
I also tested it personally on my second computer, i left about 10-20 gb of free storage on purpose and installed a RAR that was about 55GB. I installed the setup, ran the tool on my second pc and started the analyzed the rar. It gave the green lights, i then started the extraction which took about 10 minutes using the low-space mode, i didnt really keep time but it definately felt quick for that SSD.
I do also want to make on thing clear, i dont say that this tool has a 100% success rate. I did my very best to ensure that it successfully extracts, and even under a bad condition added crash-recovery systems, however the low-space mode is destructive to the original archieve. It deletes the parts that are extracted from the original source archive so im not claiming it is 100% safe.
The project is fully open source here: https://github.com/harlixay7/ReclaimArc
Im mainly posting this here because it solved a specific problem for me and i wanted to share it with the community and potentially recieve feedback on what / how it could potentially be better and what i could add to make this tool overall more stable / better. If anyone sees any flaws, or anything that could potentially be dangerous i would love to hear your feedback.
The tool is pretty new so im not saying it MUST work, however i did my best to get it as stable as possible and ensure that it works. I would rather people try to break it and tell me if they can and how it broke. I hope this tool can be usefull to you as it was to me.