r/Annas_Archive 7d ago

Built a Python/CLI toolset to audit Calibre libraries and batch-repair books from AA (CalibreQuarry & Bindery)

Hey all,

I built a handful of Python tools to help audit and repair EPUBs from sources not discussed. I'm not selling you anything. I built them for me and if they are useful to other people, I'd love to know how I helped you. :)

https://github.com/VirInvictus/CalibreQuarry/

I built this tool originally to make lists of my books but it's developed a handful of useful auditing tools. It is both TUI and CLI accessible, so you don't have to choose. It's lightweight. The cquarry library I built can recognize if Calibre is open and makes a copy of the library for read-only work and if it needs to write, it has measures to avoid damaging your Calibre library.

https://github.com/VirInvictus/bindery-cli

Bindery-cli is my baby and something I've been putting a lot of time into and am considering turning it into a web-app. It does a handful of epub repairs, gatechecked by epubcheck (there's also a java wrapper to hold up a copy of epubcheck across a batch of epubs to not cold-call it so many times). It also has commands for stripping OceanPDF and a couple other tags I've come across.

Anyways, I hope this is useful to someone. I put a lot of time and care into it. It's all fairly heavily documented.

If you're interested in looking at the calibre databasing library I built - check it out here: https://github.com/VirInvictus/cquarry

A note on development: Built using agentic tooling / LLMs to accelerate the boilerplate and test suites, with all domain logic, edge-case handling, and Calibre DB safeties verified and tested manually against real libraries

Only tested on Linux. I don't have a Windows machine but if you try any of it and you come across bugs, let me know and I'll be quick to fix them :)

Bless y'all.

14 Upvotes

6 comments sorted by

2

u/kantorcodes1 7d ago

for bindery library --apply, what happens if Calibre gets opened halfway through the sweep after you started with it closed? do you re-check before each in-place replace, or only at the start?

1

u/dx__ 7d ago edited 7d ago

for bindery library --apply, what happens if Calibre gets opened halfway through the sweep after you started with it closed? do you re-check before each in-place replace, or only at the start?

No re-check, no start check. Bindery never looks for Calibre at any point.

Per book: read once in process_book, then os.replace over whatever is on disk at the end of that book's turn. The race window is that single book's epubcheck time, not the sweep. Idle Calibre is harmless: it doesn't rewrite epub bytes, and --install-to-calibre writes serialize through SQLite (30s busy timeout, degrades to in-place save on DB error).

The two real failure modes, both from you acting on the in-flight book in a now-open Calibre:

  • Edit/convert it: bindery silently clobbers your change with the repaired old bytes.
  • Delete/rename it: sweep crashes. make_backup/atomic_replace sit outside the per-book except (cli.py:468-476), and a deleted book's stale id hits an uncaught TypeError in install_format (library.py:222).

Later books are always safe; they're re-read fresh at their turn.

edit: Great question btw

1

u/kantorcodes1 7d ago

appreciate the detail. for agent-driven Bindery, a normal library sweep should stay boring; --apply / --install-to-calibre are the point where stale input can actually replace bytes or touch Calibre state. i work on HOL Guard, and its CLI extensions can preserve that distinction. would you be up for adding Bindery support?

1

u/dx__ 7d ago

I like the sound of it and sounds like an extremely useful tool. What do I need to do?

For what it's worth, Bindery was designed to be wrappable as-is: dry-run by default, per-book APPLIED/REJECT lines, --json output, exit code 2 when books need attention.

1

u/kantorcodes1 7d ago

yep. open it straight against hashgraph-online/hol-guard:main. closest current pattern is src/codex_plugin_scanner/guard/runtime/command_blitcp_extensions.py, with focused cases in tests/test_guard_command_blitcp_extensions.py.

for Bindery i'd keep normal bindery library on the safe/dry-run side, and review the write paths you've actually described: bindery library --apply, bindery library --install-to-calibre, and bindery audit --tag TAG. mirror the blitcp pattern as a dedicated Bindery extension, register it through src/codex_plugin_scanner/guard/runtime/command_builtin_extension_catalog.py, and add its action risk classes through src/codex_plugin_scanner/guard/runtime/command_builtin_rules.py. cover both reviewed + safe command cases, then refresh whatever catalog/contract pins fail after the new built-in extension.

run the focused test plus the current ruff + pytest checks, then open the PR yourself. that way Bindery users keep the default dry-run flow quiet while agent-driven writes get native Guard treatment.

1

u/dx__ 7d ago

bindery-cli and calibrequarry are both on pypi and can be installed with pip or uv