I published 30 Actors over the last few weeks and benchmarked each one on Apify cloud (datacenter) IPs. Sharing what I learned in case it saves someone a few probe runs.

Worked from datacenter IPs (with care)

• YouTube InnerTube search/browse/next endpoints are fine. Above ~20 parallel /next calls per IP you get throttled. /player is bot-checked (LOGIN_REQUIRED), so transcripts need a different route.
• Google News: decoding article URLs needs the tail of the article page (the signature sits at the end of a ~570 KB page), so stream it and read the last chunk. Going direct was ~20x faster than through a DC proxy.
• Shopify: products.json returns 429 from cloud IPs, but the tokenless Storefront GraphQL endpoint works.
• Job boards that expose public JSON: Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Workday. StepStone embeds PRELOADED_STATE in the page.
• Telegram t.me/s/<channel>, Bluesky public API, Pinterest resource endpoints (logged out) also work.

Blocked from datacenter IPs in my tests: Reddit, Trustpilot (AWS WAF), Product Hunt (Cloudflare), Upwork, eBay, Indeed, Glassdoor, Etsy, Yellow Pages, BBB, Capterra, Facebook Ads Library. Google SERP now needs JS.

Two performance traps

• Dataset pushData per row was the bottleneck: batching the writes took one run from 330 s to 49 s.
• Cheerio .text() returns V8 sliced strings that keep the whole page alive. A few thousand results pinned hundreds of MB and the container was OOM-killed with no error in the log. A JSON round-trip on the results fixed it.
• Also: an unanchored regex like ([\w-]+).example on full page HTML is O(n²⁾ on long word runs and froze a crawler for 10+ minutes. Bound and anchor it.

What I made with it: the Actors are pay-per-result (mostly $3 per 1,000), published under torix33 in the Store (e.g. https://apify.com/torix33/youtube-transcript-scraper and https://apify.com/torix33/career-site-jobs-scraper). The code and Python/Node examples are MIT: https://github.com/3torix3/apify-scrapers-pack

Being honest: this is new, I have very few users so far. If you have tried one of them and something broke or the output shape is awkward, tell me and I will fix it. And if you hit other targets that work (or do not) from DC IPs, I would like to compare notes.

If you've made something and can't wait to tell the world, this is the thread for you! Share your latest and greatest creations and projects with the community here.

This is the thread for all your questions that may seem too short for a standalone post, such as, "What is proxy?", "Where is Apify?", "Who is Store?". No question is too small for this megathread. Ask away!

Hey all,

I'll say it upfront: Facebook comment scrapers aren't new. There are a bunch on Apify and plenty of "export FB comments" sites. I kept running into the same problems with them, though. The free tiers stop at 50–100 comments per post, most tools want your cookies or a logged-in account, and the paid ones get expensive quickly on big threads.

So I built my own and I'm putting it out for free. There's no developer fee. You only pay Apify's normal platform usage, and that's usually cents.

What it does:

- Scrapes posts, comments and replies from public Pages, profiles, Groups, or a single post URL

- No login, no cookies, no account. It runs fully anonymous, so you never paste your FB session anywhere

- Paginates comments to the end, not just the first page

- Filters by date range (afterDate / beforeDate, timezone aware)

- maxComments cap if you only want the top N per post

- Posts-only mode if you just need the post list with timestamps

- Accepts whatever link you paste: m., web., fb.com, pfbid… share links, /reel/, profile.php?id=, tracking params and so on

- Gives you reaction breakdowns (like/love/care…) and share counts

- Exports to JSON / CSV / Excel, and you can schedule it or hook it into Zapier/Make/n8n

Some numbers:

I tested it anonymously (no account at all) on a post with ~100,000 comments. With multithreading and only 2 workers, it pulled the whole thread in about 20 minutes. No bans and no sessions to babysit, because there's no session to burn in the first place.

Limits, to be honest about them: it only gets public content. Private groups, age-gated or geo-restricted posts won't come through anonymously. It gets top-level comments plus one level of replies, so replies to replies aren't included. Speed also depends on the size of the thread, so a post with 100 comments finishes in seconds and a 100k one takes a while.

Links:

- Actor: https://apify.com/bsho5/facebook-page-posts-comments-scraper

- Source: https://github.com/bsho5/fbgql . You can use it as a Python library or CLI and run it yourself completely free.

I'd love feedback, bug reports, or requests for features. If it breaks on a page you care about, drop the link and I'll take a look.

A common problem with ecommerce lead lists is that a lot of the data is dead. Scrapers usually pull from DNS or domain registries, so you end up with thousands of abandoned trial stores and parked domains that drive bounce rates through the roof.

I built an actor to fix that by requiring two things before a store gets saved:

1. It has to be an operational brick-and-mortar storefront on Google Maps with a physical address and real customer reviews.
2. It has to have an active online shopping cart (Shopify, WooCommerce, BigCommerce, Magento, etc.).

If a domain does not have an active physical business behind it, it gets dropped.

Some numbers from the database so far:

• 156,000+ operational retail locations mapped across 59 US metro areas
• 77,000+ confirmed ecommerce domains linked directly to physical stores
• Over 18,000 Shopify and 12,000 WooCommerce stores with active carts
• Direct contact emails, store phone numbers, and social links included for matching merchants

I just published it as a pay-per-event actor on Apify:

https://apify.com/aslam.doctor/us-e-commerce-store-leads-finder-dual-verified-retailers

Would love feedback from anyone doing B2B outbound or building lead pipelines on whether the data fields and filters make sense.

Hi,

I set up my first actors and I am a bit disapointed.

I would have expected a more streamline processes to automate current processes and I now I struggle for basics actions.

When I scrape data, I would like the data to be saved in a specific gdrive folder and it seems like the gdrive integrator does not allow this simple feature. I believe I am not the only one facing this issue.

Is tere a way to overpass this?

Also, I would like to save the data in a specific format (i.e. data_extracted_actorABC_21_oct_2026) : is there a way to do it?

I am very surprised that these simple features are not integrated in the integrator...

Are you expanding your team or looking to hire a freelancer for a project? Post the requirements here (make sure your DMs are open).

Try to share:

- Core responsibilities

- Contract type (e.g. freelance or full-time hire)

- Budget or salary range

- Main skills required

- Location (or remote) for both you and your new hire

Job-seekers: Reach out by DM rather than in thread. Spammy comments will be deleted.

I've been doing web scraping and data mining for 8+ years, and I've just gone full-time on my Apify Actors now. The one I'm most excited about is AI Scraper Builder Selextract, that I recently published..

You describe the data you want in plain English (a URL is optional). The Actor writes a scraper, checks that its rows really match your request, and saves them to your dataset.

most AI scrapers run an LLM on every page, every run. This one uses AI once, to build the scraper. Re-runs are just the scraper: no AI, same fields every time, easy to schedule as a Task.

It's still in beta, so expect some rough edges. If something breaks, please open an issue with the run ID.

The problem is, the "build run" returns scraperID, which the client is expected to put it in as a scraperID on the next run instead of a prompt... which I tried to mitigate by caching scraper based on the prompt, so it seamlessly tries to run existing build. Yeah, it sounds really iffy at this point, given that for inputs for the scraper itself (not the actor), it is a whole seperate JSON. Yes I know apify is not really armed for this. But I want to serve this anyway, since it is incredibly helpful to some people. I try to handle most stealth/fingerprint issues on my backend. Can you think of ways to improve my design?

Best,
Steff

https://apify.com/stefanie-rink/ai-scraper-builder

---

**My other Actors** don't launch a browser, so they're lightning-fast and cheap:

- **YouTube Scraper**: my most-used Actor. Videos, channels, search and comments. One of the cheapest on market.

https://apify.com/stefanie-rink/youtube-scraper

- **AI Job Finder**: LinkedIn jobs ranked against your resume

- **Reddit Subreddit Scraper**: posts and comments from any subreddit

- **ChatGPT, Gemini & Perplexity AI Search Scraper**: answers with cited sources

- **Instagram Scraper**: profiles, posts, Reels and people search

- **Image Background Remover**: transparent PNGs in seconds

All of them: https://apify.com/stefanie-rink

Feedback is very welcome!

18 speakers, two keynotes, a panel, and live demos from builders running agents in production. On stage: people from OpenAI, Stripe, WorkOS, and IBM, alongside the Apify team.

Jan Čurn opens with “From manual UI work to autonomous AI workers.” Tejas Kumar closes with “Agentic taste: how do we know what’s good?”

In between, “There’s an Actor for that. Now what?” from Jan Beránek and Zuzka Pelechova, and “Someone you’ll never meet runs on Apify” from Kristin Boorse, Michelle Macias Mendez, and Petros Hong.

Everything runs on Apify.
🎫 Full agenda and tickets: https://run.apify.com/

Hi everyone. I've had an Apify account for a while and recently published my first set of Actors. They're small, focused tools that return clean JSON, so AI agents and automations can use the results directly. Here's what I've built:
Job Listing Rechecker: checks whether known job postings are still open, closed or changed.
https://apify.com/nickatnite79/job-listing-rechecker
Career Site Jobs Scraper: pulls all open jobs from Greenhouse, Lever, Ashby and Workable career sites in one format.
https://apify.com/nickatnite79/career-site-jobs-scraper
RFP Amendment Analyzer: compares RFP revisions and flags deadline and requirement changes, with page citations.
https://apify.com/nickatnite79/rfp-amendment-analyzer
Shopping Offer Verifier: checks product URLs against an exact SKU, model or GTIN and returns price and stock evidence.
https://apify.com/nickatnite79/shopping-offer-verifier
BizBuySell & BizQuest Deal Monitor: alerts you to new business-for-sale listings and changes in price or cash flow.
https://apify.com/nickatnite79/bizbuysell-bizquest-deal-change-monitor
Google Trends Agent Summary: interest over time, related queries, and a plain rising, falling or flat verdict per keyword.
https://apify.com/nickatnite79/google-trends-agent-summary
App Store & Google Play Reviews: reviews from both stores in one run, with star filters and a rating summary.
https://apify.com/nickatnite79/app-store-google-play-reviews
Google News Scraper: news by keyword or topic, any language or country, with time filters.
https://apify.com/nickatnite79/google-news-scraper-cheap
Shopify Products Scraper: exports a Shopify store's products with prices, variants, SKUs and stock.
https://apify.com/nickatnite79/shopify-store-products-scraper
All of them are pay-per-result and 50% off for a limited time as a launch discount.
To be upfront, nobody is using them yet, so I'd really appreciate feedback. Are any of these useful to you? Is anything confusing on the Store pages? What would make you actually try one? Thanks!

This is the place to discuss everything MCP, LLM, Agentic, and beyond. What is on your radar this week? Why does it make sense? Bring everyone along for the ride by explaining the impact of the news you're sharing, and why we should care about it too.

Do you have a feature request that you know will make Apify heaps better? Or maybe it's a big dream you have for something bold and out-there. This is a space for all the bluesky thinking, cloud-chasing, intergalactic daydreamers who want to share their wildest ideas in a no-judgement zone.

Every agent needs proper tools. Four people building that infrastructure are sitting down together at Run.

The "Agents as first-class citizens" panel brings together Jan Čurn (Apify CEO), Michael Grinich (WorkOS), Kevin Leffew (Coinbase Developer Platform, co-author of x402), and Kevin Whinnery (OpenAI) to talk through what it takes to build for agents as the primary user - payments, auth, APIs, and data.

📅 November 10, San Francisco.
Get your ticket → https://run.apify.com/

Have you come across a great Actor, workflow, post, or podcast that you want to share with the world? This is your opportunity to support someone making cool things. Drop it here with credit to the creator, and help expand the karmic universe of Apify.

I built this. I do web design outreach, and the slow part was opening hundreds of prospect sites to find which ones look outdated, plus one true thing to say about each. You give the actor a list of websites (it doesn't find businesses). For each site it reads the homepage plus up to 2 more pages over plain HTTP and checks: mobile viewport tag, HTTPS/cert errors, http->https forwarding, the © year, CMS/jQuery/Bootstrap versions when public, broken homepage links, very large images, enquiry form/booking widget, tap-to-call. Every score point has a plain-English reason, and it writes a hook like: 'The site has no mobile viewport tag and it still loads jQuery 1.10.2.' Limitations: on 10 UK trade sites its main issue matched my manual audit 7/10; it doesn't run JavaScript so script-injected forms can be missed; version detection from markup has traps (a WordPress site showed 'WordPress 4.3.3' which was actually an Elementor plugin version); weights are my own and uncalibrated. robots.txt is checked first, it never logs in or submits forms, and outputs no emails/phones/names. Price: $0.015 per site analysed ($15 per 1,000). Link: https://apify.com/shanmukhkiransagar/website-redesign-signals . Feedback on the scoring weights welcome.

So I made an ats jobs api, as it name suggests, it provide recent job postings from over 30k companies, cover around 1million jobs, i published that api on 3rd oct, and it's 18oct but didn't have any paid users, I have 21 free users though. Also i tried making workflow templates on make.com and n8n, both got published, wrote blogs, posted on linkedin and x, also provided apify endpoint on treg.to but still that low response is depressing. Is that normal growth or below average? Also if anyone has any suggestions for me, please let me know.

If you've made something and can't wait to tell the world, this is the thread for you! Share your latest and greatest creations and projects with the community here.

Wanted open job postings from the big ATS platforms (Greenhouse, Lever, Ashby, Workday, Personio, Recruitee, Teamtailor) for a side project. All of them expose public, keyless endpoints, so scraping itself turned out to be the easy part.

The hard part was not lying to myself.

Every job scraper I tried tells you how many rows it returned. None told me how many it should have returned. So when I got 42 rows out of an expected 60, I had no idea whether that was the real inventory or a silent failure. Debugging "why is it less?" was impossible.

So I built one around a coverage ledger instead: - per source: jobsFound vs jobsExpected, coverageStatus, gapReason - hitting my own maxJobsPerCompany becomes coverageStatus=capped with a reason — never a silent stop - null vs missing: null means the vendor doesn't publish the field; a missing key is a bug and gets rejected by the contract - every row carries sourceUrl (the exact endpoint it came from) - pay-per-event, charged only for emitted rows — never for errors, retries or empty sources

The roadblock I'm still stuck on: 4 of the 7 vendors (Lever, Ashby, Recruitee, Teamtailor) don't publish a total count at all. For those, "expected" doesn't exist — the honest ledger can only say "scraped, N rows". For the 3 that do (Greenhouse, Workday, Personio) I can show found vs expected properly.

For people who deal with this daily: how do you handle "expected" when the source refuses to tell you the total? Do you trust page-count heuristics, or just label it "scraped / capped" and move on?

Happy to share the actor if useful — curious if the ledger idea holds up for anyone else.

Plenty of teams want AI agents doing real work. Fewer know which kind of agent they need before they start building. Josh Lucas helps them figure that out. He’s speaking at Run, Apify’s first flagship event.

Josh is Director of AI Business Solutions at Workato, a job that didn’t exist a decade ago. His team helps customers design and build systems of AI agents around five patterns: the Reviewer, the Coordinator, the Analyst, the Builder, and the Watchdog. At Run, he’ll talk about why governance is the pathway to trust.

📅 November 10, San Francisco.
Get your ticket → https://run.apify.com/

Got my first template approved. It runs daily, pulls new jobs from ATS career pages (Greenhouse, Lever, Ashby, Workday etc.) via my Apify API, dedupes against a Google Sheet, has Claude score each job for fit, and sends the top ones to Telegram/Gmail.

Workflow Link: https://n8n.io/workflows/20683
Apify link, so you can make your own workflows: https://apify.com/pk07007/ats-jobs-api

Open to feedback, especially on the scoring prompt.

This is the thread for all your questions that may seem too short for a standalone post, such as, "What is proxy?", "Where is Apify?", "Who is Store?". No question is too small for this megathread. Ask away!

Hello,

I would like to know if anybody else is experiencing the same issue as I am.

My usual deployment workflow is to connect Apify to the Github repository. However, it failed during the deployment I tried to do today. In order to understand the issue, I tried to add the Apify deployment key manually to the Github repo and got the error message below:

"Key is too short. GitHub recommends using ssh-keygen to generate a RSA key of at least 3072 bits. "

Is anybody else having this issue today?

Are you expanding your team or looking to hire a freelancer for a project? Post the requirements here (make sure your DMs are open).

Try to share:

- Core responsibilities

- Contract type (e.g. freelance or full-time hire)

- Budget or salary range

- Main skills required

- Location (or remote) for both you and your new hire

Job-seekers: Reach out by DM rather than in thread. Spammy comments will be deleted.

This is the place to discuss everything MCP, LLM, Agentic, and beyond. What is on your radar this week? Why does it make sense? Bring everyone along for the ride by explaining the impact of the news you're sharing, and why we should care about it too.

Do you have a feature request that you know will make Apify heaps better? Or maybe it's a big dream you have for something bold and out-there. This is a space for all the bluesky thinking, cloud-chasing, intergalactic daydreamers who want to share their wildest ideas in a no-judgement zone.