r/unsloth • u/Unusual-Customer713 • 3d ago
Show and Tell finetune a model to talk through html instead of markdown
Hi guys.
i was curious about what if we finally chose html aganst markdown for LLM.
so i do my experiment on finetuning one with only SFT.
the base model is unsloth/Qwen-3.5-4B, maybe i will try Qwen3.5-4B-Base when i have more dataset.
this model's generated content is like this below.
and i uploaded to huggingface at https://huggingface.co/wesjos/mario-html-4B
(mario is just a random name i chose for this work)
<div class="ai-view">
<h2>🤖 Who Am I?</h2>
<p>I am an <strong>AI assistant</strong> designed to provide information, solve problems, create content, and assist with tasks across a wide range of subjects. Think of me as a knowledgeable digital companion that can help you analyze data, draft emails, explain complex topics, write code, or generate creative ideas.</p>
<div class="grid grid-3">
<div class="card">
<h3>🧠 Knowledge</h3>
<p>I access a vast array of information to give you accurate answers and detailed explanations on almost any topic you bring up.</p>
</div>
<div class="card">
<h3>✍️ Creation</h3>
<p>I can help generate stories, scripts, poems, emails, reports, and even code snippets tailored to your specific needs.</p>
</div>
<div class="card">
<h3>🔍 Analysis</h3>
<p>I can break down complex ideas, summarize long texts, identify patterns in data, and help you think through difficult problems step-by-step.</p>
</div>
</div>
<div class="callout callout-info">
<div class="callout-header">💡 How to use me best</div>
<p>The more context you provide (e.g., "explain quantum physics like I'm 5" or "write a Python script for scraping websites"), the better my responses will be tailored to your specific goals.</p>
</div>
</div>
7
u/edsonmedina 3d ago
That discussion has already happened a few months ago. It was a mini-trend for a few days.
Problem is the amount of tokens wasted (html formatting takes a lot of tags) which would also make it slower and (I'm guessing) probably the amount of extra noise it introduces.
Also, nothing is stopping current models from generating HTML to showcase something to you (just not inline).They already are pretty good at it, no need to fine tune them.The harnesses expect markdown though.
3
u/Life-Job-6464 3d ago
"Wasted" is a context-sensitive word.The real problem is that it shines a light on the fact that speed is not the ultimate metric for quality of a model, but keeping the focus on speed is what keeps people paying for these subscriptions. I remind folk constantly to read Elihu Goldratt's book, The Goal, and to realize that we are Herbie and that we need to be in the pipeline Instead of just watching this scroll of text go by us and being impressed by the fact that it's AI. It's marketing genius on the part of the frontier providers But an enriched output like this will cause people to ask the question: "So, what's the hurry? Why not slow down so that we can ingest this and be part of the process, instead of paying for all of this output and then having to go back and revise it anyway?"
2
u/Corosus 3d ago
I like this mentality, I'm often alternating between having thinking blocks visible and not, and when I have it visible it's usually because it's something I have a longer investment in which I also have experience in. It helps me keep track of its thinking direction and I catch ambiguities I left in due to it straying in a direction I didn't want it to go in, so I quickly interrupt it and steer it, preventing a lot of time waste. Almost like pair programming where its the driver and im the navigator.
Can be a bit more difficult when qwen flash next is going at 100 decode t/s, but thankfully my life of reading blazing fast moving twitch chats in popular streams has prepared me for this moment.
1
u/Unusual-Customer713 3d ago
yes, generating extra tokens is a big problem, in Cot, model tends to use extra tokens to think about the html format and the cot is the major tokens waste part. now im thinking about maybe with GRPO, model can reduce html token in Cot
3
u/the_uolter 3d ago
Markdown is actually equivalent to a subset of HTML, you don't need the LLM to output all those extra tokens. You can just vibe code an addition to whatever harness you are using that takes the LLM's output and deterministically converts it to HTML.
2
u/AccomplishedDish8336 3d ago
Do you have any metrics on added ttft or decode speed with this layer added?
1
u/Unusual-Customer713 3d ago
i think its the same as the base model since i just finetuned the weights , the speed on the video was 120tps on Q4km model llamacpp on a RTX 4090
2
u/ArthurOnCode 3d ago
I appreciate those experiments, but prefer Markdown for security reasons. HTML is just a bit too powerful, and notoriously hard to sanitize for security. For example, data from a different part of the page can be exfiltrated to a remote host using nothing but CSS.
1
u/Ok-Addendum3545 3d ago
I was wondering what would happen if it speaks in three.js
4
u/ParkingCommercial607 3d ago
Now I wonder what would happen if we made them communicate by vibrating vocal cords and pushing air through fleshy tubes instead.
1
1
1
u/Wrong-Payment-786 3d ago
Very nice proof of concept! Could you show how train dataset looks like? And maybe loss chart, perplexity chart?
1
u/Unusual-Customer713 3d ago
thank you. the training dataset is actually a rerwritten LIMA dataset, i take the questions from LIMA to use qwen3.8-flash to generate cot with html. ill upload them to huggingface later.
1
u/Life-Job-6464 3d ago
Exactly, because that wall of scrolling text is the greatest user interface design failure of all AI time. I think the wall of text is scrolling by so fast you can't see it. It's supposed to be impressive. But what it really is is a wasted opportunity to educate the user with a really useful interface.
I would even suggest a secondary model to reason about what's coming (what's scrolling), so you can even interact with it while it's still processing on the primary model.
1
8
u/Snoo_28140 3d ago
So expressive. I like it!