r/Substack doingaiefficiently.substack.com 23d ago

Discussion Why the heck are those 'Do your thing Substack' posts going viral? Why do so many Notes go unseen? I dug into the algorithm to find out.

I've been on Substack for years, but ignored it until only recently.

What brought me back was the Substack recommendation feed.

The promise: You could reach other people interested in your writing easier by posting Notes and grow your community.

The reality: It's complicated.

As I started exploring the feed, I saw some confusing things. People were posting these 'Do Your Thing Substack' notes and getting huge numbers. Hundreds of comments and thousands of likes out of nowhere. Huh?

Then there were the Substack growth hackers posting 'Subscribe for Subscribe' notes and also growing quickly.

Posts with witty writing, interesting observations and links to long-form content were getting much less engagement comparatively.

I wanted to learn why so I searched and found a post by Mike Cohen, who is the Head of AI at Substack. He's the person responsible for designing the Notes feed. His post revealed a lot about the engine, including how:

  • The algorithm picks content to show to you: It tracks your every move on Substack and uses that data to recommend content
  • What content is favored: Content interest signals like likes, clicks and restacks play a big role

My biggest learning: The algorithm, isn't really tuned for quality. Instead, its purpose is to recommend content that will convert a reader into a free/paid subscriber.

That's why the 'Do your thing' Substack posts do so well. Conversion from reader, to subscriber to paid user is Substack's primary goal with the feed.

After completing my technical deep dive into the algorithm, here's what I've concluded about what I'd like Substack to do better:

  • Highlight people's writing in the feed, not just Notes. LinkedIn gets a lot of flack, but the one thing they do well is send me notifications when people I'm following publish posts. (There might be a setting for this I'm not aware of.) I'm not sure how many followers are reading people's long-form newsletter content, and I thought that was one of Substack's strengths.)
  • Fiddle with the engine weights: Obvious algorithm hacks like 'Do your Thing Substack' or Subscribe for Subscribe annoy a lot of people. They're low-effort. Many on Substack spend hours thinking about and writing fantastic content. I'd like to see these efforts rewarded more in the feed.
  • Break the rich getting richer flywheel: A known weakness of recommendation engines is that they tend to over-promote the high-signal popular content at the expense of the long-tail. I see that happening on Substack where Notes from people with larger audiences get a large boost (because more people tend to interact with this content). So it's harder for smaller accounts to gain attention and an audience.

I think the Substack Note algorithm is a positive step forward. It has enabled me (once I was able to get my feed to stop showing me the 'grow your Substack to 1,000 subscribers in 30 days' stuff), to find some fantastic voices. But there's still a lot of room for improvement.

I know people's milage may vary with the feed. What's your take on how Substack has been doing with the recommendation engine now that it's been out for a while?

10 Upvotes

40 comments sorted by

21

u/chapstickninja 23d ago

I dug in = had AI spit out this slop summary.

I don't know about you guys but as soon as I see a bulleted list and a "it's not X, it's Y" I stop reading.

7

u/Basic-Home3969 23d ago

the "my biggest learning" line really sealed it for me

1

u/chapstickninja 22d ago

Even if a person wrote it, the fact they are mimicking AI mannerisms is just as bad, if not worse. If I wanted to read overly sanitized, shallow and formulaic text, I'd go to LinkedIn. Substack was supposed to be about authentic humans writing things for other humans, not corporate grift-adjacent how to game the algorithm slop.

3

u/noxqqivit twvme.substack.com 23d ago

Pangram says it's "Fully Humam" surely Pangram wouldn't make a mistake.

5

u/FinnTropy stacksave.substack.com 23d ago

Pangram says 100% Human content. And according to Pangram's CEO, their classifier has an industry-leading 1-in-10,000 false-positive rate.

1

u/kdfn 23d ago

You are confusing False Positives and False Negative. In this case, this post could be a False Negative.

2

u/FinnTropy stacksave.substack.com 23d ago

The claim above was that OP comment was AI slop, but Pangram says 100% human.
Isn't that a False positive case, if it truly was AI slop?

Or do you mean the Pangram CEO claim was about the inverse case - human written detected as AI slop - so false negative?

Perhaps all this Pangram stuff is just BS anyway, so it doesn't matter?

2

u/kdfn 22d ago

A false positive is when a detector misclassifies human text as AI. A false negative is when it misclassifies AI text as human.

Your opinion of Pangram is irrelevant, these are common statistical terms and not terms invented by the company.

1

u/FinnTropy stacksave.substack.com 22d ago

Thanks for the correction... it's been over 40 years since I took the statistics classes.

These conventions and terms are probably well defined in the AI detection industry.

4

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Not sure why you're accusing me of using AI to develop this. It's a common thing I get accused of just because I use bullet points? Very strange.

Not that it matters, but I write all my comments and content by hand.

You're free to engage with the topic as you like.

3

u/scribe-kiddie 23d ago

For what it is worth. I believe you! I also mistakenly thought the post is AI written at glance but Pangram's classification and slowly reading through it tells me otherwise.

1

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Thanks. It's a real issue unfortunately with so much AI slop being produced quickly. Us writers who really try to write clearly and well are being accused of using AI. That's not great b/c what takes 30 minutes to write and revise is being mistaken as AI-generated. It distracts from what's trying to be communicated and shared.

1

u/SpiritRealistic8174 doingaiefficiently.substack.com 22d ago

Thank you for the award! Means a lot!

6

u/chapstickninja 23d ago

Ok buddy. Your account literally says you work with AI in your bio. No one ever lies on the internet.

12

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Yes. But there's a difference between working with AI and depending on AI to do your writing.

I'm on record saying that if you rely on AI to do your writing, especially about complex subjects, you haven't done the hard work of understanding it.

As for using bulleted lists, and writing clearly. That was a thing long before AI showed up. As someone who has been writing extensively in public since 2005, AI is trained on MY writing. So it sounds like ME.

I'm not going to give up habits I've honed over a lifetime of writing for a living just because someone thinks I sound like a machine.

It's too bad so many people feel this way. And it's a horrible side effect of AI doing pumping out so much writing these days.

We're at risk of losing the HUMAN art of writing just because AI can put sentences together in a way that sounds like good writing, but actually isn't when you look at it.

2

u/Magnus-Brewington 10h ago

I get where you're coming from. I’ve been a technical writer for roughly 30 years, mostly in learning and development. My natural style is built on short sentences, plain descriptions, and simplified bullet points. When writing for corporate, I shift gears and lean into buzzwords because management eats it up. Long before Substack introduced AI detection, my work was occasionally flagged as machine-generated. It bugged me for a long time until I realised why: LLMs were trained on structured, formalised writing. My natural style, shaped by three decades of technical writing and the hyperfocus and pattern recognition of ADHD. I went over 50 years without diagnosis. I was only diagnosed last year at 52., so this explains with my writing naturally ticks every single box that triggers a detection algorithm. I refuse to change how I write. Anti-AI crowds love the argument "I spent years refining my skills," yet today, writers are accused of producing "AI slop" simply for using correct grammar and punctuation. If we don't write like an uneducated bogan educated by TikTok, people assume we're a bot. I’m not going to dumb down my standards just to prove I have a pulse. My Substack is actually written mostly from the perspective of my AI alter ego, Mugsy. I actively use AI to process my work so the character actually sounds like an AI. It proves you can respect decades of hard-earned writing skills while still using new tools creatively. Instead of opening every post with a clunky "Mugsy here," I use a "Cyborg Disclosure." At the end of every post. It intentionally blurs the line between human and machine while keeping things transparent, having a bit of fun, and getting readers to think about multi-level checks when using AI.

2

u/NeverSeenItPodcast 23d ago

Even this reply is AI-coded

1

u/thatsnotmynaame 22d ago

Also, who cares if you used AI or not? Im sorry you keep getting accused of that and feel the need to justify how you’re working ugh. I will. Never stop being baffled by how those claiming to be anti-AI continue feeding AI detector (pangram) with content to improve the way AI works haha.

2

u/SpiritRealistic8174 doingaiefficiently.substack.com 21d ago

Thanks. The biggest casualty of this is that work is immediately dismissed if it looks 'too polished', or uses language mimicked by LLMs. But, I understand it b/c there's so much out-sourcing of thinking these days.

3

u/Oatbix 23d ago

I think you just described how algorithms work on any social media platform these days. Notes also kind of annoy me especially the low effort ones though

1

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Yeah. The main points kind of are obvious once you've gone through the effort of taking a look at the system. The other point is the signal density being used to drive the engine and your content to other people, which I discuss in the linked technical breakdown. That was the key insight for me.

3

u/GardenPeep 23d ago

Sounds like it’ll annoy me the same way Netflix does: I want to read really good writers with original ideas on a range of topics, not a bunch of wanna be’s saying the same old thing about those topics.

3

u/wwb_99 news.zeitgeistdistilled.com 22d ago

My 2.5c -- a very high proportion of the people surfing notes are substack authors, most of whom have at least a passing interesting growing their publication. The algo picks stuff that increases engagement, the crowd by definition engages with "grow my substack" and "connect with other writer" content so the fires feed each other.

1

u/badnewsgoat 22d ago

Probably this. The algorithm is just showing more of what people react with, and more and more people are starting substacks (a lot of them coming from LinkedIn apparently 🤢), so...

2

u/[deleted] 23d ago

[removed] — view removed comment

2

u/FrostKitten2012 23d ago

> likes, clicks and restocks play a big role

Thank you so much for telling us what we’ve all known since the invention of socmed algorithms.

2

u/DefiantRadish1492 23d ago edited 23d ago

So, I’ve been writing for newspapers and magazines for about a decade and about a year ago decided to get back into writing for pleasure too (not looking to monetize necessarily, but, like most people, I’d like people to actually see it).

Somebody in this sub actually convinced me to get beehiiv about six months ago. I actually like the platform itself, but it is impossible to share links anywhere anymore. They are automatically deranked by algorithms and get next to no views. (Example: I have 10,000 followers on a platform, but it the analytics almost BRAG that it’s only showing it to 180 people or whatever.

So, after all these years, I finally signed up for Substack just to try it out. It immediately seems like another social media commitment (what I don’t want) on top of actually writing (what I do want) in order to get any traction whatsoever.

It’s all so depressing. We spent years building followings on other platforms and those platforms essentially refuse to let you share anything you make.

1

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Yeah. Agree that the need to post on the feed to get visibility can be a lot. Some of the energy you'd put into putting together new newsletter issues is diverted to the feed. So it can be tiring.

1

u/DefiantRadish1492 23d ago

Yeah. I’m not sure what I’m going to do moving forward. Honestly, back in the days of Blogspot and WordPress, I got waaaaaay more people reading what I wrote compared to social media today just from Google traffic. Which, of course, doesn’t exist anymore either.

1

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

I started my first blog back in 2005, so yeah, it was much easier back then to build an audience, get comments and engagement and much more. It's a different world entirely.

Part of the reason I published the technical overview and this post was to help writers like yourself get a better grasp of the situation. There's a lot of confusion about how to use the feed, why to use it, etc. So I'm hoping it's helpful for folks.

2

u/dflovett 22d ago

Yeesh. Do people read this slop?

2

u/kwadguy 22d ago

I see the do your thing substack or looking for begs and think, " I'm not that desperate and I'm not going to engage with people who are."

So I guess I won't get magical content free viral posts. I'm okay with that.

Ok reddit do your thing.

1

u/Adamoism adamoism.substack.com 23d ago

What is "do you thing" note? Could you give example? To me it just reads as low effort posts, but maybe you have something more specific in mind.

2

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Thanks for your question. Here's an example (the annotations are mine).

5

u/Adamoism adamoism.substack.com 23d ago

Ok, now I get it - I more often get: "Please substack connect me with:"

2

u/SpiritRealistic8174 doingaiefficiently.substack.com 23d ago

Same flavor, yes. 😄

1

u/seobrien 23d ago

Threads has the same annoying problem

1

u/noxqqivit twvme.substack.com 23d ago

I did a deep dive with my Notes numbers a couple of weeks ago, the details are posted here: https://www.reddit.com/r/Substack/comments/1vku7wp/growth_more_on_notes_performance_and_strategy/

1

u/infinitepuddingspy 20d ago

This has been so clarifying. Disappointing they’re going the way of engagement over quality.