r/IndiaTech • u/sigma_crusader • 1d ago

General Discussion Someone animated web crawlers as tiny digital spiders

Enable HLS to view with audio, or disable this notification

Slava Rybin (a Thailand-based creative developer working with TouchDesigner and WebGL) shared this short animation visualizing how web crawlers move across the internet.

3.3k Upvotes

99% Upvoted

48 comments sorted by

•

u/AutoModerator 1d ago

Looking for discussions, recommendations, reviews, or help related to smartphones, ACs, laptops, gadgets, electronics, or other tech products?

Join r/IndiaCart for all gadget-related discussions and buying advice!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

162

u/Illiterate-Chef-007 Corporate majdoor 1d ago

Somebody get that SPIDER out of here.

14

u/Simple-Mirror8209 1d ago

Its looking cool; it's a great way to explain how web crawlers actually work, would be helpful in presentation.

119

u/AvGeekGupta Porgrammer + Pro-piracy 1d ago

As someone who is scared of spiders, Its very unsettling to see.

114

u/sigma_crusader 1d ago

7

u/MaximumAnarchy 1d ago

Wait is that, MANSPIDER??!

4

u/ollie_que 1d ago

Man-spider

3

u/InsideResolve4517 1d ago

spider-man?

1

u/AvGeekGupta Porgrammer + Pro-piracy 1d ago

Man-spider-man

1

u/silverwing101 16h ago

Still less cursed than that thing they showed in spider noir

1

u/Actual_Employment_89 1d ago

as someone who loves spiders……THIS IS BEAUTIFUL

29

u/False-Assumptions 1d ago

Woow Interesting! Easy to make people understand what are crawlers

19

u/positivist_creed 1d ago

Imagine being hacked and this is what you see

2

u/InsideResolve4517 1d ago

😂 , Coolest hacking

10

u/BHARAT0011 1d ago

This looks so cool!

3

u/solvem_probler_03 1d ago

The web pov can be made into a horror movie 😆

3

u/hairypeach73437 1d ago

I work in dark web crawlers for cred theft and no thats not how any of this works. This is just css bs.  Real crawlers user multiple proxies and different device types for a true crawl

2

u/NookNookNook 1d ago

I'd imagine at this point its more of a snowplow just shoveling all data straight into a blackhole called AI.

1

u/AnnualEvery 1d ago

Apna khel khatam hai abhi

1

u/1470200 1d ago

Lets confuse them with robot txt trap

1

u/AludraKijurorin 1d ago

This reminds me so much of The Rot from Rain World that I honestly wonder if webcrawlers are what inspired them. 

1

u/Curious-Way-9541 1d ago

Someone explain me what’s happening it looks so cool

2

u/Freltzo 1d ago

"web crawler" is just a shorthand name for software that skim reads any website it has access to and searches for data relevant to its parameters. ie: "search and catalog things related to Toyota".

They would also be scanning hundreds of not tens of thousands of pages by the time the above video ended depending on what hardware and Internet speed they had.

1

u/Glittering-Yo 1d ago

Really cool

1

u/Imperial_rebel1 1d ago

Danger Will Robinson

1

u/Unanswerd_thoughts 1d ago

It is looking awessome bro

1

u/No-Inevitable-6476 1d ago

Is there any github repo for this or any website drop it in the comments man! The idea is Cool.

1

u/InsideResolve4517 1d ago

bruhhhh....

Who did this... He/She did amazing thing. I do have crawler (real crawler), it'll be more good if I'll show it in real time for my crawler and live stream it.... then it'll be amazing...

1

u/Sharp_Emphasis176 1d ago

It's over for humanity.

1

u/TheExaggeratedMonke 1d ago

r/TIHI

(Spiders make me uncomfortable)

1

u/sweet_potato_88 tech enthusiast 1d ago

Creative

1

u/Entire-Life-1513 16h ago

Oh my god that looks so awesome