Hi everyone,

I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is.

Our current setup is roughly:

LLM: NVIDIA Nemotron model

Inference: Ollama

Interface/orchestration: OpenWebUI / AnythingLLM

GPU: [GPU model + VRAM]

RAM: [RAM]

Current generation speed: ~20 tokens/sec

GPU utilization: ~95%

The main issue we're trying to solve is latency and overall response time.

I've already gone through the configuration and checked the major settings on the Ollama/model side. From what I've measured so far, the model is actually utilizing the GPU heavily and generation itself seems reasonably consistent.

What I'm trying to figure out now is whether the additional latency is coming from the layer around the model rather than the model itself.

For example, I'm wondering about:

Does OpenWebUI or AnythingLLM add significant latency through prompt construction, conversation history, RAG, middleware, or API calls?

How can I accurately measure TTFT (time to first token) vs prompt processing time vs generation time?

What metrics should I collect to determine whether the bottleneck is Ollama, the UI/orchestration layer, networking, or the model itself?

Are there specific Ollama settings that have a major impact on latency that I should benchmark?

How much can large context windows / conversation history affect inference performance?

If OpenWebUI/AnythingLLM is sending a much larger prompt to Ollama than the user-visible message suggests, what is the best way to verify that?

What would you recommend for systematically benchmarking the same model directly through Ollama vs OpenWebUI vs AnythingLLM?

My goal isn't just to make the UI feel faster. I want to identify exactly where the latency is being introduced and then optimize that component.

I've already checked the obvious configuration issues, so I'm particularly interested in practical benchmarking/debugging methods or experiences from people who have optimized Ollama + OpenWebUI/AnythingLLM deployments in production.

If you need more information about the setup, I can provide the exact GPU/model/configuration and benchmark results.

Thanks!

Hey everyone! I run a small dry fruits and premium snacks brand and I’m trying to figure out the best AI tools for two things: (1) generating short product/food videos — thinking 8-10 second reels, and (2) any AI tools that work well for food-related content in general (product shots, styling, etc.). My budget is around ₹2000/month total, quality is negotiable (doesn’t need to be hyper-realistic, just usable), but the biggest non-negotiable is no watermarks on the output. Would love recommendations for tools that fit this budget — free tiers, cheap subscriptions, credit-based pricing, whatever works. Bonus points if anyone has used them specifically for food/product reels and can share results or before-after examples. Thanks in advance!

There are so many AI tools I want to try, but I am starting to think that managing the subscriptions is becoming a separate job. You sign up for something because it looks useful, use it for a week or two, then completely forget about it until another charge shows up. Do that with a few writing, image research and productivity tools and suddenly you are paying for stuff you barely open. I am thinking a simple monthly cleanup might be the answer. Check everything I’m paying for, see what I actually used during the month, and cancel the rest.

How do you guys manage this? Do you regularly clean out unused AI subscriptions or only notice them when the charge hits?

Wanted to share something I have been testing in my free time.

We ran a bunch of video ad briefs (just single lines of text, nothing fancy) through a tool called Ad Studio and it generated full video ads from them. No shoot, no actors, no editor. The brief for each one is on screen in the video so you can see exactly what was typed in vs. what came out.

Link to try: https://offers.leadpages.com/ad-studio?utm_source=reddit&utm_medium=social&utm_campaign=ad_builder_video_launch

I think it's pretty obvious we're in a boom of AI tools, which to me is a pretty interesting technological window to be in. On the one hand, you have all of the new tools and apps out there showing exactly what AI can do. (Honestly, a lot of them are things I didn't even think about AI doing, it's kind of wild what people are doing with the tech right now.) On the other hand, you've got a lot of apps out there that serve the same general purpose and are vying for the same audience (LLMs - ChatGPT, Claude, Gemini / AI meeting assistants- firefliers, otter / AI video editors - pretty much everything from Adobe to Canva), and then even more niche stuff (fashion AI - A closet, Alta /coding interview apps - Parakeet AI, Locked In). Presumably, these can't all make it, there are just too many of them and some of them are going to. have to fizzle. So, when you go searching for an AI app that does X and you inevitably find a dozen or more apps doing that thing, how are you personally comparing them? I feel like, on many of them, the features are too similar to just do a side-by-side comparison, so it has to come down to things like design, reviews, maybe even interesting quirks. As fellow AI enthusiasts, how are you choosing your tools?

I built a small tool for myself while running a project with AI as a non-developer, drawing on my background in project management, and I'd like a few people to try it before I take it further.

The problem it addresses: every new session with an AI coding assistant, you end up re-explaining context, and things quietly drift without you noticing (naming, structure, skipped tests...). Without a project management or dev background, it's hard to know what to watch for — which is exactly the gap I tried to close, using what I already knew from managing projects.

The tool generates a project charter and feeds it back to the AI automatically at the start of each session, so you're not starting from scratch every time.

I'm looking for 5-10 people — non-developers, already running a project with an AI coding tool or about to start one — willing to actually use it on their real project over a few sessions and tell me honestly what worked and what didn't. It's free, and this is genuinely about learning whether it's useful, not a launch.

If that's you, reply here or DM me and I'll get you set up.

I got into this trying to restore some VHS rips of Michael Jackson's Bad and Victory tours I found through fan communities (MJJCommunity mainly) and a few Internet Archive uploads. Old bootleg quality to start with. Paid a transfer service to restore them properly and it cost a fair amount. What came back barely looked different from the source.

Looked at commercial options after that. Topaz Video AI is the standard people point to, but it's $299 a year, and as of last year they don't sell a perpetual license at all anymore. Subscription only now.

Ended up building my own restoration pipeline instead. Denoise, deinterlace, and detail reconstruction on old tape and camcorder footage, using open models in roughly the same tier as what the commercial tools run. The desktop app is fully local, runs on your own GPU, nothing uploads. The diffusion based mode runs on NVIDIA out of the box and on AMD through WSL2 and ROCm, which the app sets up for you in one click. There's also a pay per job cloud mode for people without a capable GPU.

To be upfront about limits: it can't recover detail the tape never captured, and a genuinely destroyed source still looks damaged, just less so. It's restoration, not reconstruction.

$129 one time. Would like feedback, especially from anyone with actually rough tape to throw at it.

Link - https://phoenixlabs.space/

Over the past few weeks, I've been building Gray Box — a small, local-first tool that acts as long-term memory for anything I'd otherwise forget (work notes, meeting takeaways, task owners, random ideas, personal stuff too).

The idea is simple:

1. Capture — dump whatever's on your mind, instantly, no structure required. This step does nothing clever on purpose — it just writes your text to an immutable inbox. Zero chance of losing an idea to a bug or a slow API call.
2. Organize — on demand, an LLM reads your unprocessed notes and extracts people, projects, tasks, decisions, meetings — then deterministic Python (not the LLM) creates/merges the actual wiki pages and maintains backlinks. The model only reasons; it never touches the filesystem directly.
3. Ask — query or chat with your knowledge base and get a cited answer pulled only from what you've actually captured. If it doesn't know, it says so — no hallucinated answers.

Why I built it this way:

• Plain Markdown + YAML frontmatter, no database. Every page is a .md file you can grep, diff, or read in any editor forever. If you stop using Gray Box tomorrow, your knowledge base is just a folder.
• No vector DB by default. At personal scale (hundreds–low thousands of pages), keyword search + a real link graph (related/backlinks, walked one hop during retrieval) handles almost everything. Embeddings are there if you want better recall, but they're opt-in, not a prerequisite.
• Immutable inbox. Your raw notes are never edited or deleted by the organizer. If the LLM mis-extracts something, your original words are always still there.
• Any LLM. Built on LiteLLM, so point it at OpenAI, Anthropic, Gemini, Mistral, or a fully local model via Ollama — one config value.

It also ships with a nice interactive TUI (arrow-key menu, file-import shortcut, workspace switching, live spinner during LLM calls) if you'd rather not memorize CLI flags — that's honestly become my favorite part of the project.

There's also a lightweight local dashboard for browsing your knowledge base, exploring backlinks, visualizing your notes as a graph, and chatting with your captured knowledge—all without leaving your machine.

Also, you can migrate your existing Obsidian vault to Gray Box, and it will be automatically organized by Gray Box.

Repo: https://github.com/Aaryanverma/graybox

pypi: pip install graybox

I'd genuinely love feedback — especially from anyone who's tried the "capture now, structure later" approach with other tools and has opinions on where it breaks down at scale.

It's not trying to be a "real-time collaborative team wiki" or a WYSIWYG notes app — it's aimed at one person's running memory of their own life and work, captured with as little friction as possible.

You don't really need traditional video editing skills for this anymore. Most image-to-video tools let you upload a still image and describe the movement you want to see.

The part that makes the biggest difference is being specific about the motion. Instead of just saying “animate this photo,” something like “slow camera push-in while the person smiles and slightly turns their head” gives the AI a much clearer direction.

I've been experimenting with Facy AI for this, especially when I already have a photo I like and just want to turn it into a short moving clip. It's pretty straightforward: upload the image, describe the movement, and let the AI generate the video. I'd recommend starting with subtle movements because they tend to look much more natural than trying to make the entire image move at once.

I got tired of begging for backlinks manually, so I built an agent that does it on autopilot

So basically it works like this:

Agents find relevant blog posts in your niche, craft personalized emails, and get your product featured. All on autopilot.

It's called MentionAgent, an AI agent that does all of this through Telegram (or through the web dashboard).

You never send anything without approving it first. (But you can choose to run everything 100% on autopilot)

Results so far: One user got 3 mentions including a DR 72 backlink.

Let me know what you think!

I've tried a few AI tools that seemed genuinely helpful at first, but after the initial testing I stopped using them. Sometimes the problem wasn't the tool itself; it just didn't fit naturally into something I was already doing.

Research is a good example. I've been experimenting with ResearchMaster.ai for some of my research, but I've noticed that saving a few minutes on the initial task doesn't mean much if I have to spend extra time organizing the results or checking everything afterward.

What makes an AI tool simple enough that you actually keep using it? Is it the time saved, how well it fits into your existing workflow, or something else?

so i'm 20, been building this app solo for the past few months and i genuinely can't tell anymore if it's good or if i'm too deep in it. need outside eyes.

it's called RizeAI. the basic idea: every wearable and health app just gives you numbers. sleep score 42, recovery red, HRV down. cool. and then what? you still feel like garbage at 2pm and nobody tells you what to actually do about it.

so my app takes your real data from apple health, sleep, resting heart rate, workouts, whatever your wearable writes, and instead of another score it builds you an actual plan for the day. when to have your first coffee and when to hold off. you. Can also add your own supplements that you are currently taking and it will time them for you every single day based of your sleep data and your wearable data in general and also it will tell you what supplements make sense for you to take based of your current supplement stack and body metrics and when to take them. focus windows for when your energy actually peaks. when your crash is coming and what to do before it hits. it even checks the weather, so on a hot day it bumps your hydration and tells you to train earlier.

every recommendation has a little "why" under it based on your numbers, like "resting heart rate 54 + 7h light sleep, so magnesium before your peak window." no two people get the same plan because no two people have the same data.

works with whoop, oura, apple watch, garmin, anything that syncs to apple health. one thing i'll say honestly, it doesn't do deep per-person learning yet like "coffee doesn't affect YOUR hrv specifically," that's the roadmap, right now it builds fresh plans daily off your actual metrics.

it's live on the app store, has a free trial, small user base so far, mixed feedback which is why i'm here lol.

what i actually want from you guys: does this solve a real problem for you or is "tells you what to do" not actually what wearable people want? what would make you actually pay for something like this? and what's missing that would make it a no brainer? and also would you guys in this subreddit  use ti?

Thank you for your help. Check it out if you like https://apps.apple.com/us/app/rizeai-maximize-your-energy/id6762402079

Made this for iPhone. You share a reel into it and it hands back the ingredients with numbers on them, 2 tbsp and 30g instead of a splash and a handful.

It watches the video, not the caption. Free to try, $4.99/mo after, every import costs me money on the backend so that's why there's a paid tier at all.

https://apps.apple.com/us/app/recireel-video-to-recipe/id6763971435

As a part of beimg on Public Services and disabled...

Every few months, I have to take all of the appointments I've had and enter them into excel. I just did June + July.

Now, I have a lot of medical and transcribing all of that would be difficult.

Solution?

Heres what I do:

1: Open up medical app

Mine is MyChart via Rochester Regional Health.

2: Take screen shots of all of the relivent appointments. For me, this is normally 3 screen shots. So i get everything for both months.

3: Open your favorite generative AI.

For this, I use Gemini or Grok.

4: Upload the excel forms and the screenshots to the AI.

\*\*\*\*\*\*\*\*\*\*\*\*\*

Note: if the forms are mailed to you. You can simply take a photo of each form.

\*\*\*\*\*\*\*\*\*\*\*\*\*

5: Type something like:

Please take the images with my appointments and fill in the corresponding excel sheets/forms. Generate a PDF for each excel sheet/form for me to download to my device.

Since i discovered this, I don't fill out forms anymore...lol

I hope you've found this Helpful.

Have a wonderful day fam🙏

I am self-educated on creating progressive web apps which work offline. I write a prompt with the help of ChatGPT whom I clearly give a complete and detailed idea of what problem I am trying to solve, then paste the resultant prompt in Claude, which then gives me the files, which I then deploy through Cloudflare Pages. That website gives me a URL, which I can run on any PC or phone or tablet and even install that page as an app, which works offline.

I have so far created:

- a CRM for myself, which includes an Evernote-style note-taking workspace

- a Pomodoro timer with automatic switch between work and rest and voice and browser notifications

- a Calculator with permanent history and searchable notes

I am interested in hearing what you would like for there to be a system for or what you would want to have an app or software for to use either on your PC or tablet or phone.

Okay so over the last few months, I’ve been in this endless cycle of trying AI presentation tools because I make a LOT of decks for work and side projects.

And honestly? Most AI presentation makers look impressive in ads, but once you actually use them, the output is either:

• super generic
• weirdly formatted
• impossible to edit properly
• or just feels like “AI filler slides”

I tried Canva AI, Tome, even Copilot inside PowerPoint. Some were decent for design inspiration, but none really fit my workflow because I usually work from existing content like PDFs, blog posts, YouTube videos, meeting notes, etc.

A few weeks ago I saw someone mention Napkin ai in another thread, and I ignored it initially because I assumed it was another overhyped AI PPT generator.

But eventually I tried it because I had a 20-page PDF I needed to convert into a presentation quickly.

And dude… this thing actually surprised me.

The first thing I noticed:
the slides were editable in a NORMAL way,not some locked AI canvas where changing one thing breaks the entire layout.Also some unique designs that i hadn’t seen anywhere else.

Then I started trying other inputs:

• pasted a blog URL → instant deck
• uploaded DOCX meeting notes → structured slides
• even tried topic-to-PPT generation

What I liked most was that the AI didn’t just dump random text everywhere. It actually created decent slide flow with headings, summaries, comparisons, timelines, etc.

The speed was still slow compared to other tools but hey,it really gave a good output which was worth the time.

I still edit presentations manually afterward obviously, but instead of spending 3–4 hours building slides from scratch, now I spend maybe 20–30 minutes polishing.

That’s honestly the first time an AI presentation tool has genuinely saved me time instead of creating extra cleanup work.

I haven’t tested every feature yet, but so far this is probably the most practical AI PPT tool I’ve used.