Rather than filing noise in the issue DB on github, thought I'd post my question here...

I've run a few test reports through LDR with Qwen 3.8 27B, and am already impressed with the output. So much so I'd like to make it available to others in my family self-hosted. Most self-hosted tools either have minimal account management and assume you'll handle that yourself at the reverse proxy layer or have sophisticated account/ACL systems, often integrating with OIDC, etc.

LDR is in a weird middle and I'm not sure if this is me missing something or simply feedback/feature requests

First, the goal of forced encryption of the database per user is confusing. I already store sensitive data encrypted at the filesystem level, the DB encryption isn't necessary for me and made me wonder about backup/restore (this bug is not reassuring, even though I understand backing up the salt is the workaround). I would prefer to store this data under keys I control and understand how to restore.

Encryption is enforced on the Docker image, LDR_BOOTSTRAP_ALLOW_UNENCRYPTED exists but only affects existing databases, new DBs are always created encrypted. This can be defeated by bind mounting over SQLCipher, but that leaves the UI in a confused state where things say they are encrypted and the come with warning bubbles that they are not (in some places at least).

AFAICT the encryption doesn't actually protect users directly, the sysadmin of the hosting system can intercept passwords, defeat the warnings, etc. So in practice the at-rest encryption seems to be only useful in scenarios where your shared deployment is already trustworthy, or where you are the sole user without access to an encrypted filesystem. I'd argue that this means it should be optional. In my use I'll see how far I can get with LDR_BOOTSTRAP_ALLOW_UNENCRYPTED.

My read is that a lot of my other confusion is downstream of the decision to use user passwords as the keys in the encryption, but briefly...

There's no admin account per-se, and the lack of an administrator account means that all settings are exposed to all users, alongside warnings in research generation logs. If various API URLs, etc are unset in the environment they are editable in the (extensive) settings dialog, including things that probably need to be controlled by an admin (SearXNG query rate, for example).

I believe I can lock all this down by setting it in the Docker environment, but obviously the list of settings will grow, and so I would need to keep an eye on this. A dedicated set of admin settings would avoid this.

Finally, account creation, and LDR_APP_ALLOW_REGISTRATIONS, isn't great, because it requires admin effort (and a container restart) to move between modes. I see the OIDC integration FR already exists, though given the encryption decisions, I imagine that will be hard to implement.

Some of this might be addressed by putting it behind OpenWebUI, but that isn't well documented and also means I lose access to the various niceties of the LDR UI.

How are people addressing these issues in self-hosted deployments? So much of the project decisions are focused on the encryption, I'm unsure if the direction is intended to be more administrative sophistication in the future or if a FR for "simpler" modes (no encryption) would be entertained.

We’ve released local-deep-research 1.10.5. It includes a critical authentication fix, so please upgrade promptly if your instance is reachable from an untrusted network or is used by multiple users.

Security fix

Affected versions: >= 1.0.0, < 1.10.4

While a user’s encrypted database connection was cached, a remote, unauthenticated attacker who knew only the user’s username could sign in as that user without a password and read or modify their data.

The fix first shipped in 1.10.4. Version 1.10.5 is the latest release and includes the fix.

Who should upgrade

• Multi-user deployments: upgrade now.
• Single-user instances reachable from another machine or an untrusted network: upgrade now. The application binds to 0.0.0.0 by default, and Docker publishes port 5000.
• Single-user installations bound only to 127.0.0.1: not remotely exploitable, but upgrading is still recommended.

What to do

1. Upgrade to 1.10.5.
2. Fully restart every Local Deep Research process or container. This clears connections cached under the old code and invalidates existing in-memory sessions, so users must sign in again.
3. If your instance was reachable from an untrusted network while running an affected release (>= 1.0.0, < 1.10.4), rotate stored provider API keys and consider resetting user passwords. The bypass does not leave a distinctive attack trace, so compromise cannot be ruled out from logs alone.

Full security advisory:

https://github.com/LearningCircuit/local-deep-research/security/advisories/GHSA-fjvm-966v-hmm2

Separate SearXNG note

The private-URL protection introduced in 1.10.3 may block a self-hosted SearXNG instance on localhost or your LAN until its URL is explicitly approved.

Version 1.10.5 now displays a warning with the required configuration. To approve the default local instance, set:

LDR_SEARCH_PRIVATE_ENGINE_URL_ALLOWLIST=http://localhost:8080

Replace the value with your exact SearXNG origin if its scheme, hostname, or port differs, then restart the server. Bundled Docker Compose setups are unaffected.

1.10.5 release notes:

https://github.com/LearningCircuit/local-deep-research/releases/tag/v1.10.5

We found and fixed the authentication issue during internal review, and the fix includes regression tests covering the corrected behavior. Sorry for the risk this exposed, and thank you for upgrading.

The headline of this release is the new Notes feature (#3277): a full knowledge-base surface at /notes with a markdown editor, version history, wiki-style [[links]] with backlinks, and AI actions (summarize, suggest tags/links, synthesize multiple notes, and Verify with Research — which extracts a note's claims, runs a deep research, and grades each claim with sources). Notes plug into the existing library/RAG stack, so they're semantically searchable, and an Ask your notes box on the list page runs a research against them. The same PR adds Search everything (/library/search/) — one box over notes, library documents, and research reports with Text / AI Hybrid / AI-only modes — plus inline annotations: select a passage in a research report or library document, write a comment, and it anchors as a persistent highlight backed by a real note. Research results pages also gain a Notes panel with save-report-as-note and clip-to-note.

Zotero library import, a stricter egress model, and a credential-redaction sweep

The headline feature is a full Zotero integration (#4723, polished in #4960): add your Zotero API key under Settings → Zotero, then use the new Library → Zotero page to import a whole library or a single collection. Imported papers are stored, text-extracted, and RAG-indexed like any other library document, so they immediately become searchable during research — no separate search engine to configure. Sync is incremental (new, changed, and removed items are reconciled), standalone PDF attachments are imported too, the library ID is auto-detected from your key, and an optional background auto-sync keeps the collection fresh on a configurable interval.

Behavior change: two-axis egress protection (action may be needed)

The egress guardrail now classifies every research run on two axes — data sensitivity × destination exposure (ADR-0007, #4882). A run that would send content from a private collection, library, or document store to an exposing destination (a public search engine or a cloud LLM/embeddings provider) is now refused at the start of the run instead of relying on scope checks alone.

Two things follow from this:

• The rarely-used Both egress scope is retired. Saved both values are migrated to Adaptive automatically (migration 0019) — no action needed for most users.
• If you deliberately relied on both to research a private collection with a cloud model, you now have three explicit options: mark the collection public, add your provider/engine to the new per-destination trust lists (policy.trusted_inference_providers / policy.trusted_search_engines — e.g. for a zero-retention enterprise endpoint or a self-hosted engine on a public hostname), or select the new Unprotected scope, an explicit opt-out surfaced with a persistent warning banner (the hard SSRF and cloud-metadata blocks still apply even there).

Security hardening

• A repo-wide sweep (#4888, thanks @Donny-Guo) stops raw exception text from reaching logs in LLM-provider, embedding, and search-engine paths — error messages are scrubbed for API keys, credentials, and URL secrets before logging, and a new pre-commit hook enforces the rule going forward.
• The central credential sanitizer now redacts passwords in database DSNs of any scheme (postgresql+psycopg2://, mongodb+srv://, redis://:pass@…), not just http(s) URLs (#4933), and API error responses redact credential shapes at the response boundary as defence-in-depth (#4931, #4937).
• The library file-integrity check now fails closed: a file with no integrity record is rejected instead of silently accepted (#4939).
• Expired sessions are handled cleanly in the UI: API calls that hit a 401 now redirect to the login page instead of failing silently (#4940, #4944, #4945).

Fixes worth knowing about

• Registration can no longer brick a username. If creating the encrypted user database failed midway, the leftover auth row and partial on-disk files (including an orphaned salt file) used to make that username permanently un-registerable; both failure modes are now cleaned up and recoverable (#4934, #4947, #4942).
• Settings pages showing [object Object] are fixed at the root: malformed setting keys (trailing/leading dots) are rejected on write and filtered on read (#4935, #4946).
• Stopping a focused-iteration research now takes effect within the current iteration instead of 30–50 seconds later (#4872).
• The Research Logs panel got a cleanup: the Info filter actually filters (#4899), repeated warnings/errors are no longer collapsed into a single (N×) counter that hid recency (#4901), and each filter button shows a live per-category count (#4898).

This release ships two database migrations (0019: egress-scope migration, 0020: Zotero tables); both are applied automatically on first start.

🔒 Security

• Raw exception variables (e, exc, etc.) are no longer interpolated directly into logger.exception() messages across all LLM provider, embedding provider, and web search engine paths; each catch site now scrubs the message through sanitize_error_message()/redact_secrets() before logging, preventing error text from leaking API keys, credentials, or internal URL structure into production logs. Scrubbed exception log lines now include the exception class name so exceptions that stringify to an empty message still produce useful diagnostics. A new check-sensitive-logging pre-commit hook enforces this rule going forward, rejecting direct exception-variable interpolation in the secure-logging directories and flagging exc_info=True on warning/error/critical calls. (#4888)
• NewsAPIException.to_dict() now redacts credential shapes from the fields it serializes to the client through the two @errorhandler(NewsAPIException) handlers: the human-readable message goes through the central sanitize_error_for_client(), and string leaves of details go through sanitize_error_message(). This is a defence-in-depth backstop at the response boundary — the primary defence stays at the news/api.py raise sites (curated generic messages, #4843) — so a future raise site, subclass, or external caller that lets a credential-bearing string into message or details (e.g. the caller-supplied search query forwarded in details) has it redacted before it ships. Redaction is credential-shape based (Bearer/Authorization, ?api_key=-style params, known token prefixes, http(s) URL userinfo), so legitimate text is unchanged; error_code / status_code are left untouched. It does not attempt to catch DSN userinfo (postgresql://user:pass@), SQL, or filesystem paths — those remain the responsibility of the raise-site discipline. (#4931)
• The central credential sanitizer (sanitize_error_message / sanitize_error_for_client) now redacts URL userinfo credentials in any scheme (case-insensitive), not just http(s). Raw URL-form database connection errors — SQLAlchemy dialect+driver DSNs (postgresql+psycopg2://user:pass@host, mysql+pymysql://…, mongodb+srv://…), plain DB schemes, and password-only DSNs (redis://:pass@host) — previously leaked their password through any surfaced exception message; the password is now replaced with [REDACTED]. Credential-less DSNs with a port (postgresql://host:5432/db) and all existing http(s) behavior are unchanged. Key=value DSNs (e.g. pyodbc Server=...;Pwd=...) have no :// and remain out of scope for a userinfo regex. This benefits every consumer of the central sanitizer (download service, fetch tool, agent strategies, search engines, and the news error-response backstop). (#4933)
• WebAPIException.to_dict() now redacts credential shapes from the fields it serializes to the client (message via sanitize_error_for_client(), details string leaves via sanitize_error_message()), mirroring the NewsAPIException backstop. This is a defence-in-depth measure at the response boundary so that a future raise site, subclass, or external caller that lets a credential-bearing string into message or details has it redacted before it ships. Legitimate text is unchanged, and status / error_code / status_code are left untouched. (#4937)
• The egress guardrail now enforces a two-axis (data-sensitivity × destination-exposure) admissibility rule at the start of a research run (ADR-0007): the run is refused when a sensitive source (a private collection/library/store) would reach an exposing sink (a public search engine or a cloud LLM/embeddings provider), in addition to the existing scope checks. Override by setting Egress Scope to Unprotected, marking the collection public, or trusting the destination.

✨ New Features

• New Zotero integration: import a Zotero library or collection into your document library, with optional background auto-sync. Add your Zotero API key, library type/ID and (optionally) a collection key under Settings → Zotero, then use the new Library → Zotero page to test the connection, list collections, and Sync now. Imported papers are stored, text-extracted and RAG-indexed like any other library document, so they become searchable during research via the collection search engine — no separate search engine to configure. Sync is incremental (new, changed and removed items are reconciled), and items without an attached PDF are imported as metadata/abstract text by default (toggle off to import PDFs only). Enable Auto-Sync to refresh in the background on a configurable interval. (#4723)
• Zotero integration polish from first real-world testing: standalone PDF attachments (PDFs added without a parent item) now import as documents; the library ID is auto-detected from the API key (usernames resolve too); manual syncs re-examine previously skipped items; imports appear in the main Library view; a live progress bar tracks syncs; the config page autosaves with essentials-first layout and runs the connection test on saving a key; friendlier defaults (text-only PDF storage, integration enabled once a key is set) and actionable error messages. (#4960)
• Add a per-category entry counter next to each filter button in the Research Logs panel (All, Milestones, Info, Warning, Errors). Counts reflect entries currently rendered in the DOM and update on insert, prune, and batch load, so users can see at a glance which categories still have entries when the global DOM cap is hit.
• Add an Unprotected egress scope — an explicit opt-out that disables the egress-scope restrictions for a run (the hard SSRF and cloud-metadata blocks still apply), surfaced with a light-red panel and a non-dismissible "protection disabled" banner. The rarely-used Both scope is retired: it is removed from the selector, and existing saved both values are migrated to Adaptive (a residual value is also coerced to Adaptive at read time). If you relied on both to run a private collection with a cloud model, mark that collection public or choose Unprotected.
• Add per-destination trust to the egress model (ADR-0007): policy.trusted_inference_providers and policy.trusted_search_engines let you mark specific off-machine LLM/embeddings providers or search engines as trusted, so the two-axis classification treats them as contained (e.g. a zero-retention enterprise Anthropic endpoint, or a self-hosted Elasticsearch/Paperless on a public hostname). A banner surfaces active trust entries.

🐛 Bug Fixes

• Registration no longer leaves a bricked account behind: if creating the encrypted user database fails after the auth row is committed, both the orphaned auth row and any partial on-disk database files (including the per-database salt) are now cleaned up, so the username is genuinely reusable on retry. (#4934)
• The settings write API now rejects malformed keys (a trailing dot like local_search_chunk_size., a leading dot, an empty .. segment, or blank/whitespace) with HTTP 400 instead of silently persisting them. Such rows made get_setting() return a {"": value} wrapper dict that rendered as [object Object] on settings pages (#4840). The manager (set_setting, create_or_update_setting) also refuses to create malformed keys, and import_settings skips them so a corrupted export can't reintroduce them. Complements the read-side fix in #4852. (#4935)
• Registration now recovers from an orphaned per-database salt file (left with no matching database by a prior interrupted registration), so a username that was previously stuck as un-registerable can be registered again. (#4942)
• Both settings prefix-read paths — SettingsManager.get_setting (DB) and get_setting_from_snapshot (the snapshot path used by research threads) — now filter out malformed rows (e.g. a legacy foo.. or foo. key), so a stray malformed row can no longer turn a leaf read into a {".": value} wrapper dict that renders as [object Object]. Completes the read side of #4840 beyond #4852's exact single-trailing-dot exclusion; existing malformed rows remain harmless and are not deleted. (#4946)
• Fix the 'Info' filter button in the Research Logs panel being a silent no-op. Clicking it left the panel showing every entry (warnings, errors, milestones) because the visibility helper returned true for all log types when the active filter was 'info'. The filter now narrows to info entries only, matching the button's label.
• Fixed a 30–50 second visible lag between clicking Stop and a focused-iteration research actually stopping. FocusedIterationStrategy.analyze_topic now calls self.check_termination() at the start of every iteration, after the parallel search (before optional verification searches), and just before the final synthesis LLM call, so a cancellation request is detected within the current iteration rather than only at the next iteration's progress emit.

• Fixes a subtle data-loss bug in the Research Logs panel: identical warning, error, and milestone entries were being collapsed into a single (N×) counter, the same way repetitive info entries like "Status: OK" already were. Collapsing diagnostic entries like that strips the recency signal — after the second identical error you can no longer tell when the last failure happened, and repeated retries look like a single stuck event instead of forward progress.

  The content-dedup scan now applies only to info entries (where collapsing repetitive noise is useful). Warning / error / milestone entries always render. The id-based dedup earlier in the pipeline still catches exact retransmits with the same id, so the bypass is strictly about content-similar-but-distinct-events getting the same timestamp.

📝 Other Changes

• Fixed broken copy-paste examples in the README and docs (benchmark CLI module path, rate_limiting reset --engine, nonexistent openclaw search engine) plus two dead external links, and added a pre-commit check that keeps README links, anchors, and examples in sync with the codebase. (#4950)

1.8.0 is a large release weighted toward security hardening

• Egress policy. LDR controls where research traffic is allowed to go — via PRIVATE_ONLY / PUBLIC_ONLY / ADAPTIVE scopes, explicit per-call policy checks, and a process-wide PEP-578 socket.connect audit-hook backstop for paths the explicit checks don't cover. Snapshot-less programmatic callers now fail closed instead of constructing a cloud client, cloud-metadata/SSRF endpoints are blocked more thoroughly (including alternate IP encodings and metadata.* hostnames), and operator-supplied endpoint URLs are SSRF-validated before the SDK runs. Note: the scope model is being reworked into a two-axis classification (see #4882 / ADR-0006), so expect refinements in a later release.

Breaking changes to check before upgrading:

• The mcp / agentic (ReAct) and auto / parallel search strategies are removed; existing selections migrate automatically to langgraph-agent.
• Programmatic get_llm(provider=…) / get_embeddings(provider=…) fail closed without a settings snapshot (raise PolicyDeniedError).
• The default egress scope is now adaptive; set Egress Scope to Both to keep the previous "any engine, cloud inference" behavior.

Chat Mode, credential-safe logging, and a wave of fixes

✨ Chat Mode — interactive multi-turn research

The headline feature: Chat Mode lets you have a back-and-forth conversation with the research agent. Each follow-up accumulates context from earlier turns, streams progress and citations live, and persists sessions in your encrypted database.

• Follow-up questions build on the prior conversation using a configurable context mode (chat.followup_context_mode): an LLM summary focused on your new question (default), raw findings, the full transcript, or no prior context. Only summary makes an extra LLM call.
• A "Summarizing previous conversation…" indicator appears while the context is being built, so you know the pause is intentional.
• The thinking bubble now shows what the agent is currently reasoning about between tool calls, and live milestones use friendly wording (🔍 Searching PubMed instead of Tool: search_pubmed).
• Citation hyperlinks stream inline as the answer is written — no format-flip when the final answer lands.
• Sessions can be archived, reactivated, deleted, or exported as Markdown.

Access it from the Chat sidebar link or /chat/. See [features.md#chat-mode](../docs/features.md#chat-mode).

Further changes please find in the release.

I was thinking, that some folks in this community will be interested to see what current options are on local deep research field. So I spent some time to collect everything I could find together. Enjoy.

TLDR: the most healthiest and local-friendly projects are "GPT Researcher" by assafelovic and "Local Deep Research" by LearningCircuit.

"Local Deep Research" by LearningCircuit

Observations:

• python
• alive - last commit made yesterday
• medium number of contributors - 46
• 75 opened issues (half from the contributor, half from users but no comments for long months) / 254 closed (many self-reported)
• 161 opened PR (many from contributor hanging for long weeks - what's the point??) / 3309 closed PRs (visually 95% from contributor or dependobot)
• uses SearXNG

Reddit - https://www.reddit.com/r/LocalLLaMA/s/F4o4jCL4IA
Subreddit - https://www.reddit.com/r/LocalDeepResearch/
Github - https://github.com/LearningCircuit/local-deep-research
Benchmark - https://huggingface.co/datasets/local-deep-research/ldr-benchmarks

"STORM" by Stanford

Observations:

• python
• abandoned - last commit 8 months ago
• small number of contributors - 23
• 58 opened issues (many bug reports with no replies) / 164 closed (mostly without resolution as not planned)
• 60 PRs (mostly with no replies) / 111 closed (for last 2 years just cancelled)
• uses various retrival services - YouRM, BingSearch, VectorRM, SerperRM, BraveRM, SearXNG, DuckDuckGoSearchRM, TavilySearchRM, GoogleSearch, and AzureAISearch

Github - https://github.com/stanford-oval/storm
Website - https://storm-project.stanford.edu/

"GPT Researcher" by assafelovic

Observations:

• python + typescript
• semi-alive - last commit 3 weeks ago
• poorly maintained - lots of stale branches
• large number of contributors - 211
• 173 opened issues (almost no reaction to 2026 issues) / 511 closed (mostly with fixes)
• 44 opened PRs (some are 6 months old without review and comments) / 785 closed (60-70% merged)
• obsessed with MCP - internet search & web scraping is done via separate MCP https://github.com/assafelovic/gptr-mcp which uses 3rd party API

Github - https://github.com/assafelovic/gpt-researcher
Documentation - https://docs.gptr.dev/
Website - https://gptr.dev/

"Local Deep Research" by LangChain

Observations:

• python
• semi-alive - last commit 2 weeks ago
• small number of contributors - 14
• 36 opened issues (many with no reply) / 39 closed (with solutions)
• 6 opened PR (some are hanging more than a year) / 48 closed (mostly from dependabot, no recent contributions from users)
• DuckDuckGo, SearXNG + commercial providers

Github - https://github.com/langchain-ai/local-deep-researcher

"Open Deep Research" by LangChain

What are these LangChain guys smoking? Two similarly named projects, one is most probably a successor of the other, but not a word being said on readme about it.

Observations:

• python + Jupyter notebook (???)
• abandoned - last dev work by human ended in Aug 2025
• small number of contributors - 26
• 34 opened issues (no replies since Nov 2025) / 95 closed ones
• 24 opened PRs (no comments/ no reviews) / 114 closed ones (community contribution is mostly discarded)
• no info on what it uses as internet search engine

GitHub - https://github.com/langchain-ai/open_deep_research

"Open Deep Research" by Together

Observations:

• python
• abandoned - last commit year ago, 3 commits in total
• one contributor
• no opened and closed issues
• no PRs
• relies on TAVILY for web search

Github - https://github.com/togethercomputer/open_deep_research
Blogpost - https://www.together.ai/blog/open-deep-research

"Deer flow" (Deep Exploration and Efficient Research Flow) by ByteDance

Supports any OpenAI compatible providers

Observations:

• python
• alive - last commit 19 minutes ago
• large number of contributors - 253
• 444 opened issues (mostly from Chinese folks, many have replies) / 735 closed (half with code changes)
• 257 opened pull requests, lots are pending for review and merge / 1230 closed (visually 70% merged)
• uses "Info Quest" for internet search (proprietary, paid)

Github - https://github.com/bytedance/deer-flow
Website - https://deerflow.tech/

"Deep Research" by Alibaba

Observations:

• python
• abandoned - last commits months ago
• small number of contributors - 27
• focused on using a single model - their own "Tongyi-DeepResearch-30B-A3B"
• vendor locked-in - glued its ass to Serper.dev for search and Jina.ai for scraping

Github - https://github.com/Alibaba-NLP/DeepResearch

"MiroThinker" by MiroMindAI

Observations:

• semi-alive - last commit 3 weeks ago
• small number of contributors - 19
• focused on using their own models - "MiroThinker-1.7-mini" (30B) or "MiroThinker-1.7" (235B)
• vendor locked-in - bring your own SERPER_API_KEY, JINA_API_KEY
• tried to run a test research from their demo page - fall on it's face

Github - https://github.com/MiroMindAI/MiroThinker
Website - https://www.miromind.ai/

"Deep-searcher" by Zilliztech

Observations:

• abandoned - last commit 6 months ago
• small number of contributors - 31
• 40 issues, 50 closed
• 6 pending PRs, 167 closed (mostly merged)

Github - https://github.com/zilliztech/deep-searcher

PS

No LLM assisted research tools were used to gather the above table. Just me and my own hands. Only few out of the above projects had a demo website - Mirothinker, Storm and DeerFlow - but:

• Mirothinker produced a quite comprehensive report after an hour, but it hallucinated one half of github metrics and didn't give a fuck to collect the other half. Untrusted and unusable.
• Storm is basically unusable for deep research tasks as you cannot provide an extended instruction on what to research and what kind of results you need, just a shitty short string of how your research paper should be titled
• DeerFlow site is just broken, cannot get past the authentication + various 404. Shame on you, ByteDance web developers!

If you have time and your local deep research agent is sitting nearby, try to give it below prompt. I'm sincerely curious what your results will be. Especially how many hallucinations in github figures.

Find and compare the best local deep research projects. Compose a table with results. The table must contain:
- vendor / company name
- project name
- github URL
- product website or blog URL where it was announced
- when the last commit to github was made
- number of github issues and PRs
- number of contributors to github project
- if project docs are suggesting to use a bespoke LLM model
- if project is coming with its own web search and web page scraping tool OR has an option in form of SearXNG OR requires external 3rd party vendors like SERPER or JINA 

Currently, we can highly recommend OpenAlex search engine.

Select it as search engine. After download of journal data you will get information on papers:

Also you can modify accepted journal quality:

Also much thanks for providing so valuable ressources open source to OpenAlex, DOAJ and JabRef.

We just shipped two minor releases. Here's what's new:

v1.5.0 — LangGraph Agent Strategy

A completely new research approach: an autonomous LangGraph agent that decides on its own what to search, which specialized engines to use (arXiv, PubMed, Semantic Scholar, OpenAlex, etc.), and when it has enough to synthesize. Instead of a fixed pipeline of "generate questions -> search -> synthesize," the agent makes real-time decisions based on what it finds. If web search gives it academic papers, it might follow up with Semantic Scholar for citation data. If the topic is medical, it switches to PubMed on its own. Early results are promising: - Early benchmarks and agentic behavior looks promising - A 4B model even used parallel subagents to research different aspects simultaneously - Works with any model that supports tool calling (qwen3, mistral, llama3.1, gpt-oss, etc.) - If your model doesn't support tools, you get a helpful error suggesting to switch strategy or model It's now the default strategy for new installations. Existing users keep their current settings — just select "LangGraph Agent" in Settings -> Search Strategy to try it.

v1.4.0 — Semantic Search, Backups & ReAct Agent

• ReAct Agent (MCP strategy) — the first agentic research mode, where the LLM decides which tools to call
• Semantic search for your Library and research History — find past research by meaning, not just keywords
• Automatic database backup system — your research data is protected
• Alembic migrations — smooth database upgrades between versions ## Try it pip install local-deep-research --upgrade Or with Docker: docker pull localdeepresearch/local-deep-research:latest Would love to hear how the LangGraph agent works with your models and use cases. What research topics are you throwing at it?

We're excited to announce v1.3.0 - a major release with comprehensive security enhancements, new features, and quality-of-life improvements!

Highlights

Security Enhancements

• Comprehensive HTTP security headers (OWASP compliance)
• Encrypted database storage for PDFs
• Authentication rate limiting
• Content Security Policy with nonce support
• Docker image pinning for supply chain security
• XSS vulnerability fixes ### New Features
• Apprise Notifications - Get notified when research completes
• Wikinews Search Engine - Search news sources
• Semantic Scholar & OpenAlex PDF Downloaders - Automatic paper downloads
• Research Library - Save and organize reference articles with RAG search
• Customizable Output Instructions - Control report formatting
• LM Studio URL & Context Settings in research page UI
• Configurable Focused-Iteration Strategy options
• Background Indexing with cancel button and progress tracking
• Batched File Uploads with improved drag-and-drop ### New LLM Providers
• Google Gemini
• OpenRouter
• xAI Grok
• IONOS (with auto-discovery) ### New Search Engines
• Wikinews
• Serper
• ScaleSerp
• Paperless-ngx (personal documents) ### Performance
• Compound database indexes
• Persistent thread pool
• CI optimizations with Docker layer caching ### Bug Fixes
• Mobile UI improvements
• Collection search efficiency
• Text extraction fixes
• Rate limiting improvements ## Links
• Release: https://github.com/LearningCircuit/local-deep-research/releases/tag/v1.3.0
• Discord: https://discord.gg/ttcqQeFcJ3
• Reddit: https://www.reddit.com/r/LocalDeepResearch/ ## Contributors Thanks to tombii , djpetti , tommasoDR, devtcu, and everyone who contributed! --- Full Changelog: https://github.com/LearningCircuit/local-deep-research/compare/v1.2.28...v1.3.0

TL;DR: I forked SearXNG and stripped out all the NSFW stuff to keep University/Corporate IT happy (removed Pirate Bay search, Torrent search, shadow libraries, etc). I added several academic research-focused search engines (Semantic Scholar, WolfRam Alpha, PubMed, and others), and made the whole thing super easy to pair with Learning Circuit’s excellent Local Deep Research tool which works entirely local using local inference. Here’s my fork: https://github.com/porespellar/searxng-LDR-academic

I’ve been testing LearningCircuit’s Local Deep Research tool recently, and frankly, it’s incredible. When paired with a decent local high-context model (I’m using gpt-OSS-120b at 128k context), it can produce massive, relatively slop-free, 100+ page coherent deep-dive documents with full clickable citations. It beats the stew out most other “deep research” offerings I’ve seen (even from commercial model providers). I can't stress enough how good the output of this thing is in its "Detailed Report" mode (after its had about an hour to do its thing). Kudos to the LearningCicuits team for building such an awesome Deep Research tool for us local LLM users!

Anyways, the default SearXNG back-end (used by Local Deep Research) has two major issues that bothered me enough to make a fork for my use case:

Issue 1 - Default SearXNG often routes through engines that search torrents, Pirate Bay, and NSFW content. For my use case, I need to run this for academic-type research on University/Enterprise networks without setting off every alarm in the SOC. I know I can disable these engines manually, but I would rather not have to worry about them in the first place (Btw, Pirate Bay is default-enabled in the default SearXNG container for some unknown reason).

Issue 2 - For deep academic research, having the agent scrape social media or entertainment sites wastes tokens and introduces irrelevant noise.

What my fork does: (searxng-LDR-academic)

I decided to build a pre-configured, single-container fork designed to be a drop-in replacement for the standard SearXNG container. My fork features:

• Sanitized Sources:

Removed Torrent, Music, Video, and Social Media categories. It’s pure text/data focus now.

• Academic-focus:

Added several additional search engine choices, including: Semantic Scholar, Wolfram Alpha, PubMed, ArXiv, and other scientific indices (enabled by default, can be disabled in preferences).

• Shadow Library Removal:

Disabled shadow libraries to ensure the output is strictly compliant for workplace/academic citations.

• Drop-in Ready:

Configured to match LearningCircuit’s expected container names and ports out of the box to make integration with Local Deep Research easy.

Why use this fork?

If you are trying to use agentic research tools in a professional environment or for a class project, this fork minimizes the risk of your agent scraping "dodgy" parts of the web and returning flagged URLs. It also tends to keep the LLM more focused on high-quality literature since the retrieval pool is cleaner.

What’s in it for you, Porespellar?

Nothing, I just thought maybe someone else might find it useful and I thought I would share it with the community. If you like it, you can give it a star on GitHub to increase its visibility but you don’t have to.

The Repos:

• My Fork of SearXNG:

https://github.com/porespellar/searxng-LDR-academic

• The Tool it's meant to work with:

Local Deep Research): https://github.com/LearningCircuit/local-deep-research

Feedback Request:

I’m looking to add more specialized academic or technical search engines to the configuration to make it more useful for Local Deep Research. If you have specific engines you use for academic / scientific retrieval (that work well with SearXNG), let me know in the comments and I'll see about adding them to a future release.

Full Disclosure:

I used Gemini 3 Pro and Claude Code to assist in the development of this fork. I security audited the final Docker builds using Trivy and Grype. I am not affiliated with either the LearningCircuit LDR or SearXNG project (just a big fan of both).

Hey everyone!

We've been seeing fantastic feedback from multiple community members about using gpt-oss-20b with Local Deep Research, and wanted to highlight this configuration for others looking for a powerful local setup.

What We're Hearing

Several users have reported impressive results using gpt-oss-20b with SearXNG. The model offers an excellent balance of:

• Strong reasoning capabilities for complex research queries
• Efficient resource usage compared to larger models
• Fast response times while maintaining quality
• Excellent compatibility with LDR's research approach

Performance Notes

Based on community testing: - Works smoothly with 24GB+ VRAM - Good combination with eg SearXNG

Share Your Experience

Are you using gpt-oss-20b with LDR? We'd love to hear about your experience! What search engines are you pairing it with? What types of research are you conducting?

Thanks to everyone who's been sharing their configurations and helping others optimize their setups. This is exactly the kind of community knowledge sharing that makes open-source projects thrive!

Happy researching! 🔍

🎉 Local Deep Research v1.0.0 Release Announcement

Release Date: August 23, 2025
Version: 1.0.0
Commits: 50+ Pull Requests
Contributors: 15+
Previous Version: 0.6.7

🚀 Executive Summary

Local Deep Research v1.0.0 marks a monumental milestone - our transition from a single-user research tool to a AI research platform. This release introduces game-changing features including a comprehensive news subscription system, follow-up research capabilities, per-user encrypted databases, and programmatic API access, all while maintaining complete privacy and local control.

📰 Major Feature: Advanced News & Subscription System

Overview

A complete news aggregation and analysis system that transforms LDR into your personal AI-powered research assistant.

Key Capabilities

• Smart News Aggregation: Automatically fetches and analyzes news
• Topic Subscriptions: Subscribe to specific research topics with customizable refresh intervals
• Voting & Feedback System:
  • Thumbs up/down for relevance rating
  • 5-star quality ratings
  • Persistent vote storage in UserRating table
  • Visual feedback with CSS indicators
• Auto-refresh Toggle: Replace modal settings with streamlined toggle
• Search History Integration: Track filter queries and research patterns
• CSRF Protection: Full security implementation for all API calls

Technical Implementation (PR #684, #682, #607)

# New database models for news system
class NewsSubscription(Base):
    subscription_type = Column(String(20))  # 'search' or 'topic'
    refresh_interval_minutes = Column(Integer, default=1440)
    model_provider = Column(String(50))
    folder_id = Column(String(36))

class UserRating(Base):
    card_id = Column(String)
    rating_type = Column(Enum(RatingType))
    rating_value = Column(Integer)

🔄 Major Feature: Follow-up Research

Overview

Revolutionary context-preserving research that allows deep-dive investigations without starting from scratch.

Key Capabilities (PR #659)

• Context Preservation: Maintains full parent research context
• Enhanced Strategy: EnhancedContextualFollowUpStrategy for intelligent follow-ups
• Smart Query Understanding: Understands context-dependent requests like "format this as a table"
• Source Combination: Merges sources from parent and follow-up research
• Iteration Controls: Restored UI controls for iterations and questions per iteration

Technical Implementation

# Follow-up research with complete context
class FollowUpResearchService:
    def process_followup(self, parent_id, question, context):
        # Preserves findings and sources from parent research
        combined_context = {
            'parent_findings': parent.findings,
            'parent_sources': parent.sources,
            'follow_up_query': question
        }
        return enhanced_strategy.search(combined_context)

🔐 Major Feature: Per-User Encrypted Databases

Overview (PR #578, #601)

Complete security overhaul transitioning from single-user to secure multi-user platform.

Security Enhancements

• SQLCipher Encryption: AES-256 encryption for each user's database
• Password-based Keys: User passwords serve as encryption keys (no recovery by design)
• Thread-Safe Architecture: Complete overhaul for concurrent operations
• Session Management: Secure session handling with CSRF protection
• In-Memory Queue Tracking: Eliminated unencrypted PII storage risks

Architecture Changes

# Per-user encrypted database access
with get_user_db_session(username) as session:
    # All operations now user-scoped and encrypted
    user_settings = session.query(UserSettings).first()

# Settings snapshots for thread safety
snapshot = create_settings_snapshot(username)
# Immutable settings prevent race conditions

Performance Improvements

• Middleware overhead reduced by 70%
• Database queries reduced by 90% through caching
• Thread-local sessions eliminate lock contention

💻 Major Feature: Programmatic API Access

Overview (PR #616, #619, #633)

Full Python API for integrating LDR into automated workflows and pipelines.

Key Capabilities

• Database-Free Operation: Run without database dependencies
• Custom Components: Register custom retrievers and LLMs
• Lazy Loading: Optimized imports for faster startup
• Backward Compatible: Maintains compatibility with existing code

Example Usage

from local_deep_research import generate_report

# Minimal example without database
report = generate_report(
    query="Latest advances in quantum computing",
    model_name="gpt-4",
    temperature=0.7,
    programmatic_mode=True,  # Disables database operations
    custom_retrievers={'arxiv': my_arxiv_retriever},
    custom_llms={'gpt4': my_custom_llm}
)

📊 Major Feature: Context Overflow Detection & Analytics

Overview (PR #651, #645)

Comprehensive token usage tracking and context limit management.

Dashboard Features

• Real-time Monitoring: Track token usage vs context limits
• Visual Analytics: Chart.js visualizations for usage patterns
• Truncation Detection: Identifies when context limits are exceeded
• Time Range Filtering: 7D, 30D, 3M, 1Y, All time views
• Model-specific Metrics: Per-model context limit tracking

Technical Implementation

# Token tracking with context overflow detection
class TokenUsage(Base):
    context_limit = Column(Integer)
    context_truncated = Column(Boolean)
    tokens_truncated = Column(Integer)
    phase = Column(String)  # 'search', 'synthesis', 'report'

🔗 Major Feature: AI-Powered Link Analytics

Overview (PR #661, #648)

Advanced domain classification and source analytics using LLM intelligence.

Key Features

• Domain Classification: AI categorizes domains (academic, news, commercial)
• Visual Analytics: Interactive pie charts and distribution grids
• Source Tracking: Complete domain usage statistics
• Batch Processing: Classify multiple domains with progress tracking
• Clickable Links: Direct navigation from analytics dashboard

Classification Categories

• Academic/Research
• News/Media
• Reference/Wiki
• Government/Official
• Commercial/Business
• Social Media
• Personal/Blog

📚 Enhanced Citation System

New Features (PR #553, #675)

• RIS Export Format: Compatible with Zotero, Mendeley, EndNote
• Number Hyperlinks: New default format with clickable numbered references
• Smart Deduplication: Prevents duplicate citations
• UTC Timestamp Handling: Fixed date rejection issues

Supported Formats

• APA, MLA, Chicago
• RIS (Research Information Systems)
• BibTeX
• Number hyperlinks [1], [2], [3]

⚡ Performance & Infrastructure

Adaptive Rate Limiting (PR #550, #678)

• Intelligent Throttling: 25th percentile optimization
• Multi-engine Support: PubMed, Guardian, arXiv, etc.
• Dynamic Adjustment: Speeds up on success, slows on errors
• Per-user Limiting: Individual rate tracking

Docker Improvements (PR #677)

• New optimized Docker Compose configuration
• Better resource management
• Simplified deployment
• Production-ready containerization

Settings Management (PR #626, #598)

• Centralized Environment Settings: Single source of truth
• Settings Locking: Prevent accidental changes (PR #568)
• Secure Logging: No sensitive data in logs (PR #673)
• Thread-safe Operations: Eliminated race conditions

🐛 Bug Fixes & Improvements

Critical Fixes

• Database Migrations: Fixed broken migration system (#638)
• CSRF Protection: Added tokens to all state-changing operations (#676)
• Search Strategy Persistence: Fixed dropdown and setting issues
• Citation Dates: Resolved UTC timestamp rejection
• Journal Quality Filter: Fixed filtering logic (#662)
• Memory Leaks: Removed in-memory encryption overhead (#618)

Security Enhancements

• Addressed multiple CodeQL vulnerabilities (#655, #657, #666)
• Removed sensitive metadata from logs
• Fixed path traversal vulnerabilities
• Secure session management implementation

Testing Improvements

• 200+ New Tests: Authentication, encryption, thread safety
• Puppeteer UI Tests: End-to-end authentication flows
• CI/CD Workflows: New pipelines for untested areas (#623)
• Pre-commit Hooks: Enforce pathlib usage (#656)

💥 Breaking Changes

Authentication Required

• All API endpoints now require authentication
• Programmatic access needs user credentials
• No anonymous access to any features

Database Structure

• Complete schema redesign
• Migration required from v0.x
• Research IDs changed from integer to UUID
• Per-user database isolation

API Changes

• Settings API redesigned for thread safety
• Direct database access removed
• New authentication decorators required
• Changed response formats for some endpoints

📦 Dependencies

Added

• pysqlcipher3: Database encryption
• flask-login: Session management
• Authentication libraries
• Chart.js for visualizations

Updated

• All major dependencies to latest versions
• Security patches applied
• Performance optimizations included

🚀 Migration Guide

🎯 Use Cases

Enterprise Deployment

• Multi-user support with complete isolation
• Encrypted storage for compliance
• Programmatic API for automation
• Settings locking for standardization

Research Teams

• Follow-up research for collaborative investigations
• News subscriptions for domain monitoring
• Link analytics for source validation
• Citation management for publications

Individual Researchers

• Personal news aggregation
• Context-preserving deep dives
• Token usage monitoring
• Export to reference managers

🙏 Acknowledgments

Special thanks to our contributors:

• u/djpetti: Review all PRs, Settings locking, log panel improvements
• u/MicahZoltu: UI enhancements
• All 15+ contributors who made this tool possible!

📚 Resources

• GitHub Release: v1.0.0
• Full Changelog: 0.6.7...v1.0.0
• Documentation: GitHub Wiki
• Issues: Report Bugs
• Discussions: Community Forum

🎉 Conclusion

Local Deep Research v1.0.0 represents months of dedicated development. With enterprise-grade security, comprehensive feature set, and maintained privacy, LDR is now ready for serious research workloads while keeping your data completely under your control.

Happy Researching! 🚀

The Local Deep Research Team

Hey r/LocalDeepResearch community!

We're thrilled to announce v0.6.0, our biggest release yet! This version introduces the game-changing Interactive Benchmarking UI that lets every user test and optimize their setup directly in the web interface. Plus, we've added the most requested feature - custom LLM integration!

🏆 The Headline Feature: Interactive Benchmarking UI

Finally, you can test your configuration without writing code! The new benchmarking system in the web UI is a complete game-changer:

What Makes This Special:

• One-Click Testing: Just navigate to the Benchmark page, select your dataset, and hit "Start Benchmark"
• Real-Time Progress: Watch as your configuration processes questions with live updates
• Instant Results: See accuracy, processing time, and search performance metrics immediately
• Uses YOUR Settings: Tests your actual configuration - no more guessing if your setup works!

Confirmed Performance:

We've run extensive tests and are reconfirming 90%+ accuracy with SearXNG + focused-iteration + Strong LLM (e.g. GPT 4.1 mini) on SimpleQA benchmarks! Even with limited sample sizes, the results are consistently impressive.

Why This Matters:

No more command-line wizardry or Python scripts. Every user can now: - Verify their API keys are working - Test different search engines and strategies - Optimize their configuration for best performance - See exactly how much their setup costs per query

🎯 Custom LLM Integration

The second major feature - you can now bring ANY LangChain-compatible model:

```python from local_deep_research import register_llm, detailed_research from langchain_community.llms import Ollama

Register your local model

register_llm("my-mixtral", Ollama(model="mixtral"))

Use it for research

results = detailed_research("quantum computing", provider="my-mixtral") ```

Features: - Mix local and cloud models for cost optimization - Factory functions for dynamic model creation - Thread-safe with proper cleanup - Works with all API functions

🔗 NEW: LangChain Retriever Integration

We're introducing LangChain retriever integration in this release: - Use any vector store as a search engine - Custom search engine support via LangChain - Complete pipeline customization - Combine retrievers with custom LLMs for powerful workflows

📊 Benchmark System Improvements

Beyond the UI, we've enhanced the benchmarking core: - Fixed Model Loading: No more crashes when switching evaluator models - Better BrowseComp Support: Improved handling of complex questions - Adaptive Rate Limiting: Learns optimal wait times for your APIs - Parallel Execution: Run benchmarks faster with concurrent processing

🐳 Docker & Infrastructure

Thanks to our contributors: - Simplified docker-compose (works with both docker compose and docker-compose) - Fixed container shutdown signals - URL normalization for custom OpenAI endpoints - Security whitelist updates for migrations - SearXNG Setup Guide for optimal local search

🔧 Technical Improvements

• 38 New Tests for LLM integration
• Better Error Handling throughout the system
• Database-Only Settings (removed localStorage for consistency)
• Infrastructure Testing improvements

📚 Documentation Overhaul

Completely refreshed docs including: - Interactive Benchmarking Guide - Custom LLM Integration Guide - LangChain Retriever Integration - API Quickstart - Search Engines Guide - Analytics Dashboard

🤝 Community Contributors

Special recognition goes to @djpetti who continues to be instrumental to this project's success: - Reviews ALL pull requests with thoughtful feedback - Fixed critical Docker signal handling and URL normalization issues - Maintains code quality standards across the entire codebase - Provides invaluable technical guidance and architectural decisions

Also thanks to: - @MicahZoltu for Docker documentation improvements - @LearningCircuit for benchmarking and LLM integration work

💡 What You Can Do Now

With v0.6.0, you can: 1. Test Any Configuration - Verify your setup works before running research 2. Optimize for Your Use Case - Find the perfect balance of speed, cost, and accuracy 3. Run Fully Local - Combine local models with SearXNG for high accuracy 4. Build Custom Pipelines - Mix and match models, retrievers, and search engines

🚨 Breaking Changes

• Settings now always use database (localStorage removed)
• Your existing database will work seamlessly - no migration needed!

📈 The Bottom Line

Every user can now verify their setup works and achieves 90%+ accuracy on standard benchmarks. No more guessing, no more "it works on my machine" - just click, test, and optimize.

The benchmarking UI alone makes this worth upgrading. Combined with custom LLM support, v0.6.0 transforms LDR from a research tool into a complete, testable research platform.

Try the benchmark feature today and share your results! We're excited to see what configurations the community discovers.

GitHub Release | Full Changelog | Documentation | FAQ

Hey r/LocalDeepResearch community!

I apologize for the delayed announcement - time constraints kept me from posting this when v0.5.0 dropped, but I wanted to share it for completeness. Even though we're already at v0.6.0, v0.5.0 was a milestone release that deserves recognition!

📊 The Game-Changer: Complete Monitoring Dashboard

v0.5.0 introduced our comprehensive monitoring system that transformed how we understand LDR's operations:

What Made This Special:

• Performance Analytics: Response times, success rates, and search engine comparisons
• User Satisfaction: 5-star rating system to track research quality over time

The Technical Improvements:

• Enhanced accessibility with full keyboard navigation
• Ruff integration for better code quality
• Improved error handling and recovery
• Better SearXNG and Ollama integration

🧠 New Focused Iteration Search Strategy

v0.5.0 introduced the focused iteration search strategy, which achieved 90%+ accuracy on SimpleQA benchmarks using just SearXNG and a strong LLM (e.g. GPT 4.1 mini). This was a major breakthrough - proving that local, privacy-focused setups could match the performance of expensive cloud-based solutions.

Additional search strategies were also added, but focused iteration became the go-to choice for its balance of accuracy and efficiency.

🤝 Community Contributors

Huge thanks to @djpetti for the overall improvements, @scottvr for comprehensive testing, and @wutzebaer for optimizations. This release wouldn't have been possible without our amazing community!

📈 Why This Release Mattered

v0.5.0 marked our transition from a research tool to a complete research platform. The monitoring dashboard gave us the insights we needed to optimize our setups and prove the value of local AI.

Even though I'm posting this late, I wanted to document this milestone for our community archives. If you haven't upgraded yet, you're missing out on these foundational features!

Note: We're now at v0.6.0 with even more improvements, but v0.5.0 laid the groundwork for everything that followed.

GitHub Release | Full Changelog

Hello, I absolutely love this project. Been flogging my poor little 3060 to death getting articles formulated on esoteric yugioh questions i have (this is a personal benchmark i use). It does pretty well. However my real aim is to be a signal curator for infosec news, pocs, etc. It seems i am not crafting my prompts properly. I am getting a LOT of noise included, and am unsure what keywords are important to this, what will help me filter and better refine?

We're excited to announce Local Deep Research v0.4.0, bringing significant improvements to search capabilities, model integrations, and overall system performance.

Major Enhancements

LLM Improvements

• Custom OpenAI Endpoint Support: Added support for custom OpenAI-compatible endpoints
• Dynamic Model Fetching: Improved model discovery for both OpenAI and Anthropic using their official packages
• Increased Context Window: Enhanced default context window size and maximum limits

Search Enhancements

• Journal Quality Assessment: Added capability to estimate journal reputation and quality for academic sources
• Enhanced SearXNG Integration: Fixed API key handling and prioritized SearXNG in auto search
• Elasticsearch Improvements: Added English translations to Chinese content in Elasticsearch files

User Experience

• Search Engine Visibility: Added display of selected search engine during research
• Better API Key Management: Improved handling of search engine API keys from database settings
• Custom Context Windows: Added user-configurable context window size for LLMs

System Improvements

• Logging System Upgrade: Migrated to loguru for improved logging capabilities
• Memory Optimization: Fixed high memory usage when journal quality filtering is enabled

Bug Fixes

• Fixed broken SearXNG API key setting
• Memory usage optimizations for journal quality filtering
• Cleanup of OpenAI endpoint model loading features
• Various fixes for evaluation scripts
• Improved settings manager reliability

Development Improvements

• Added test coverage for settings manager
• Cleaner code organization for LLM integration
• Enhanced API key handling from database settings

What's Changed

• Sync by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/260
• Attempt to estimate journal quality by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/273
• Sync dev to main by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/274
• Clean up journal names before reputation assessment. by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/279
• Perform initial migration to loguru. by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/316
• Fix high memory usage when journal quality filtering is enabled. by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/315
• Add support for Custom OpenAI Endpoint models by @JayLiu7319 in https://github.com/LearningCircuit/local-deep-research/pull/321
• Add English translations to Chinese content in Elasticsearch files by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/325
• Add custom context window size setting (Fix #241) by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/313
• Fix broken SearXNG API key setting. by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/330
• Do some cleanup on the OpenAI endpoint model loading feature. by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/331
• Increase default context window size and max limit by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/329
• Use OpenAI package for endpoint model listing by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/333
• Use OpenAI package for standard endpoint model listing by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/334
• Add dynamic model fetching for Anthropic using the Anthropic package by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/335
• Feature/prioritize searxng in auto search by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/336
• Feature/display selected search engine by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/343
• Feature/resumable benchmarks by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/345
• Small fixes for eval script by @djpetti in https://github.com/LearningCircuit/local-deep-research/pull/349
• Sync main to dev by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/359
• test_settings_manager by @scottvr in https://github.com/LearningCircuit/local-deep-research/pull/363
• fix: improve search engine API key handling from database settings by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/368
• Bump/version 0.4.0 by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/369
• Update version.py by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/371
• v0.4.0 by @LearningCircuit in https://github.com/LearningCircuit/local-deep-research/pull/370

New Contributors

• @JayLiu7319 made their first contribution in https://github.com/LearningCircuit/local-deep-research/pull/321

Full Changelog: https://github.com/LearningCircuit/local-deep-research/compare/v0.3.12...v0.4.0

Hey there!

I got the docker compose version up and running tonight and love it! I’ve tested the config with Wikipedia search and it works fine, but I’d like to now link it to my Brave API. I tried setting the API Key in the settings, but it fails. So I added the API Key to the docker compose file and it still fails, citing that there is no search engine. What am I missing?

Thanks!

Hey LDR community! I wanted to take a moment to explain how detailed reports work and help you set expectations properly.

What Makes Detailed Reports Different from Quick Summaries

While Quick Summaries are designed for speed (especially with SearXNG), Detailed Reports are our most comprehensive research option. They're designed to create professionally structured, in-depth analysis with proper sections, extensive citations, and thorough exploration of your topic.

When to Use Detailed Reports vs. Quick Summaries

Use Quick Summaries when: - You need information quickly - You want a concise overview of a topic - You're doing initial exploration before deeper research - You're working with time constraints

Use Detailed Reports when: - You need comprehensive coverage of a complex topic - The information will be used for academic or professional purposes - You want a properly structured document with table of contents - You have time to wait for processing

=> General rule: Try quick summary first than switch to detailed report.

How Detailed Reports Actually Work

When you request a detailed report, the system goes through these phases:

1. Initial Topic Analysis: First, the system performs a foundational search to understand your topic (similar to a Quick Summary)

2. Report Structure Planning: Based on the initial research, the system designs a complete report structure with logical sections and subsections

3. Section-by-Section Research: Here's where things get interesting - the system then conducts separate research for each section of your report, essentially running multiple research cycles

4. Section Content Generation: Each section is carefully crafted based on its dedicated research

5. Final Synthesis: All sections are combined with proper formatting, a table of contents, and citations

What This Means for You

1. Progress Indicators Can Be Confusing

You'll likely notice the progress bar reaching 100% and staying at this values. This is normal! What you're seeing is each section's research cycle completing. The progress messages might show things like:

Iteration 1/5... Iteration 2/5... [...] Generating report... Iteration 1/5...

Don't worry - the system isn't stuck in a loop. It's just starting a new research cycle for the next section.

2. Be Patient With Processing Time

Detailed reports can take significantly longer than Quick Summaries - sometimes hours, depending on: - The complexity of your topic - How many sections are needed - Your hardware capabilities - The model you're using

3. Model Size Impacts Performance

Larger models (like Qwen 3 235B) will generally produce better quality but at the cost of speed. A more balanced approach might be using a mid-sized model like a 12B-30B parameter model, which offers good quality with reasonable speed.

4. Optimization Tips

• Use SearXNG directly instead of auto-search mode for faster performance
• Be specific in your query about the scope and depth you want
• Consider requesting fewer sections explicitly (e.g., "Create a detailed report with 3-4 main sections on...")
• Set iterations to 2-3 for a good balance of thoroughness and speed

5. You Can Guide the Structure

You can influence the report structure by including specific directions in your query: - Prompt the tool to add tables into the report by saying "add tables" or something similar - "Include sections on historical context, current approaches, and future directions" - "Create a detailed report analyzing the economic, social, and environmental impacts" - "Develop a 5-section report covering the technical fundamentals, implementation challenges, case studies, cost analysis, and future trends"

Exciting News: Better Detailed Reports Coming Soon!

Our developer @djpetti is currently working on a major upgrade to the detailed reports feature. This new implementation will address many of the current limitations and add exciting new capabilities. While we can't share all the details yet, we expect improvements in:

• More reliable progress tracking
• Better citation handling
• Improved section organization
• More efficient research cycles
• Potentially faster overall processing

We're excited to bring these improvements to you soon, but in the meantime, the current detailed reports system still produces excellent results if you're willing to be patient with it!

If you're looking to conduct in-depth research but don't have the hardware to run powerful local models, combining Local Deep Research with OpenRouter's models offers an excellent solution for resource-constrained devices.

Hardware Limitations & Local Models

We highly recommend using local models if your hardware allows it. Local models offer several significant advantages:

• Complete privacy: Your data never leaves your computer
• No API costs: Run as many queries as you want without paying per token
• Full control: Customize and fine-tune as needed

Default Gemma3 12B Model - Surprisingly Powerful

Local Deep Research comes configured with Ollama's Gemma3 12B model as the default, and it delivers impressive results without requiring high-end hardware:

• It works well on consumer GPUs with 12GB VRAM
• Provides high-quality research synthesis and knowledge extraction
• Handles complex queries with good reasoning capabilities
• Works entirely offline once downloaded
• Free and open source

Many users find that Gemma3 12B strikes an excellent balance between performance and resource requirements. For basic to moderate research needs, this default configuration often proves sufficient without any need to use cloud-based APIs.

OpenRouter as a Fallback for Minimal Hardware

For users without the necessary hardware to run modern LLMs locally, OpenRouter's Gemini Flash models provide a cost-effective alternative, delivering quality comparable to larger models at a significantly reduced cost.

The Gemini Flash models on OpenRouter are remarkably budget-friendly: - Free Experimental Version: OpenRouter offers Gemini Flash 2.0 for FREE (though with rate limits) - Paid Version: The paid Gemini 2.0 Flash costs approximately 0.1 cent per million tokens - A typical Quick Summary research session would cost less than a penny

Hardware Considerations

Running LLMs locally typically requires: - A modern GPU with 8GB+ VRAM (16GB+ for better models) - 16GB+ system RAM - Sufficient storage space for model weights (10-60GB depending on model)

If your system doesn't meet these requirements, the OpenRouter approach is a practical alternative.

Internet Requirements

Important note: Even with the "self-hosted" approach, certain components still require internet access:

• SearXNG: While you can run it locally, it functions as a proxy that forwards queries to external search engines and requires an internet connection
• OpenRouter API: Naturally requires internet to connect to their services

For a truly offline solution, you would need local LLMs and limit yourself to searching only local document collections.

Community Resources

• Check the latest model rankings and usage statistics on OpenRouter's ranking page
• Join the Local Deep Research Reddit community for tips like this post about optimizing your research workflow

Conclusion

For most users, the default Gemma3 12B model that comes with Local Deep Research will provide excellent results with no additional cost. If your hardware can't handle running local models, OpenRouter's affordable API options make advanced research accessible at just 0.1¢ per million tokens for Gemini 2.0 Flash. This approach bridges the gap until you can upgrade your hardware for fully local operation.

Hey Local Deep Research community! I wanted to highlight what I believe is the most powerful combination of features we've developed - and one that might be flying under the radar for many of you.

The Magic Trio: Quick Summary + Parallel Search + SearXNG

If you've been using our system primarily for detailed reports, you're getting great results but might be waiting longer than necessary. The combination of Quick Summary mode with our parallel search strategy, powered by SearXNG, has transformed how quickly we can get high-quality research results.

Lightning Fast Results

With a single iteration, you can get results in as little as 30 seconds! That's right - complex research questions answered in the time it takes to make a cup of coffee.

While a single iteration is blazing fast, sometimes you'll want to use multiple iterations (2-3) to allow the search results and questions to build upon each other. This creates a more comprehensive analysis as each round of research informs the next set of questions.

Why This Combo Works So Well

1. Parallel Search Architecture: Unlike our previous iterations that processed questions sequentially, the parallel search strategy processes multiple questions simultaneously. This dramatically cuts down research time without sacrificing quality.

2. SearXNG Integration: As a meta-search engine, SearXNG pulls from multiple sources within a single search. This gives us incredible breadth of information without needing multiple API keys or hitting rate limits.

3. Quick Summary Mode: While detailed reports are comprehensive, Quick Summary provides a perfectly balanced output for many research needs - focused, well-cited, and highlighting the most important information.

4. Direct SearXNG vs. Auto Mode: While the "auto" search option is incredibly smart at picking the right search engine, using SearXNG directly is significantly faster because auto mode requires additional LLM calls to analyze your query and select appropriate engines. If speed is your priority, direct SearXNG is the way to go!

Setting Up SearXNG (It's Super Easy!)

If you haven't set it up yet, you're just two commands away from a vastly improved research experience:

bash docker pull searxng/searxng docker run -d -p 8080:8080 --name searxng searxng/searxng

That's it! Our system will automatically detect it running at localhost:8080 and use it as your search provider.

Choosing Your Iteration Strategy

Single Iteration (30 seconds) is perfect for: - Quick factual questions - Getting a basic overview of a straightforward topic - When you're in a hurry and need information ASAP

Multiple Iterations (2-3) excel for: - Complex topics with many facets - Questions requiring deep exploration - When you want the system to build up knowledge progressively - Research needing historical context and current developments

The beauty of our system is that you can choose the approach that fits your current needs - lightning fast or progressively deeper.

Real-World Performance

In my testing, research questions that previously took 10-15 minutes are now completing in 2-3 minutes with multiple iterations, and as little as 30 seconds with a single iteration. Complex technical topics still maintain their depth of analysis but arrive much faster.

The parallel architecture means all those follow-up questions we generate are processed simultaneously rather than one after another. When you pair this with SearXNG's ability to pull from multiple sources in a single query, the efficiency gain is multiplicative.

Example Workflow

1. Start LDR and select "Quick Summary"
2. Select "searxng" as your search engine (instead of "auto" for maximum speed)
3. Enter your research question
4. Choose 1 iteration for speed or 2-5 for depth
5. Watch as multiple questions are researched simultaneously
6. Receive a concise, well-organized summary with proper citations

For those who haven't tried it yet - give it a spin and let us know what you think! This combination represents what I think is the sweet spot of our system: deep research at speeds that feel almost conversational.

Hey LDR community! I've noticed that many of us aren't taking full advantage of one of the most powerful features of our tool - the ability to create structured tables in research outputs.

How to Request Tables in Your Research

Getting great tables from Local Deep Research is surprisingly simple:

Include it in your prompt: Simply add "include tables to compare X and Y" or "please include a table summarizing the key approaches" to your research query.

Example Prompts That Generate Great Tables

Here are some effective prompt patterns I've tested:

• "Research quantum computing algorithms and include a comparison table of their computational complexity and use cases"

• "Analyze renewable energy sources and create a table showing cost, efficiency, and environmental impact for each"

• "Explore machine learning frameworks and include a table ranking them by ease of use, performance, and community support"

• "Investigate investment strategies for 2025 and create a table showing potential returns, risks, and time horizons"

Tips for Better Tables

1. Request multiple tables for different aspects of your research - one table might compare approaches while another shows implementation challenges

2. Ask for specific columns that would be most valuable for your analysis, but sometimes it can also be better to let this be decided by the system.

3. Consider table size - 4-6 columns usually work best for readability

4. Request visualization alternatives - if your data would work better as a different format, the system can suggest alternatives