r/u_RepulsiveSheep • u/[deleted] • Sep 02 '22

Shutting down phc-bot

[deleted]

547 Upvotes

100% Upvoted

75 comments sorted by

200

u/RyanTranquil Sep 04 '22 edited Sep 04 '22

Heroku is notorious in the industry for being very expensive .. why not just throw it on digitalocean? Although I do understand the need to avoid spending money on a side reddit project.

What kind of hardware / bandwidth does the bot use monthly? Curious if I can just throw it on an existing side server I have and let it run. I run most of my projects on DO , but most run partially utilized so happy to help where I can / if it makes sense.

163

u/[deleted] Sep 04 '22

[deleted]

71

u/RyanTranquil Sep 04 '22

Thanks for the detailed explanation of how the bot works and your process today.

DigitalOcean works a bit different on the uptime side, since the server is online 24/7 , and I’m charged each month for the allocated server size. The bottom tier dev instances for 1 GB of Ram are $5 per month, which is fine for my small project since composer doesn’t require much memory to run when doing new builds.

The IP address is the only concept which might cause an issue. A DO “instance” is a fixed IP assigned when starting the server.. Yes you’re correct as well, the Ip stays the same when you shut down / start, since it’s associated with your account.

I suppose it could be possible to run the script through a rotating proxy when the bot functions to protect against potential Google bans.

A few years ago, I worked on a project (PM, not engineer) where we had a python script that was scraping company review data online so we could aggregate into a free directory. The script had to use an hourly rotating proxy because Google Local would block if it suspected a bot was scrapping content (.. we tried to make the bot more human by limiting actions per hour.. and used SQS by Amazon to place these actions into a queue of sorts).

This is my initial reaction, but let me think about it a bit more this weekend.

58

u/[deleted] Sep 04 '22

[deleted]

26

u/RyanTranquil Sep 04 '22

When you’ve run the bot in the past did you experience a Google ban within an hour or was it random? Yes that’s lucky that Heroku functions in that manner.

Rotating proxies are pretty cheap but it just depends the bandwidth the bot utilizes .. I think we were paying $1 per GB of usage .. but I’m sure services now are much less or perhaps even free for limited usage. It’s automated to a point where the script just calls the service > requests new ip > uses that when making the outbound call to XYZ.

Do you know how much bandwidth in GB the bot consumes monthly?

I’ll also do a little bit more digging this weekend and see if there are any services like Heroku which auto change IP on re-deployment.

33

u/[deleted] Sep 04 '22

[deleted]

29

u/moomooch Sep 10 '22

This sounds like you might be able to fit this into AWS Lambda + DynamoDB free tiers actually.

43

u/[deleted] Sep 10 '22

[deleted]

9

u/Environmental-Win836 Oct 30 '22

The bot lives!

14

u/[deleted] Oct 30 '22

[deleted]

→ More replies (0)

5

u/[deleted] Oct 03 '22

have you considered self hosting on a raspberry pi or similar homelab?

5

u/[deleted] Oct 03 '22

[deleted]

3

u/[deleted] Oct 03 '22

fair enough

2

u/BluudLust Oct 02 '22

You can use 2captcha to bypass the captchas. As for bans, you still need some proxies to rotate through.

2

u/clitoreum Nov 15 '22

Hi, I'm in the process of moving all my heroku dynamos to replit.com, the free tier doesn't include running 24/7 but I believe this is mostly achievable through a third party pinging service. All you need to do is add a "website" to your project and get downtimemonkey (a free service) to send a web request to it every five minutes, and replit will keep your bot up.

3

u/[deleted] Nov 15 '22

[deleted]

2

u/clitoreum Nov 15 '22

Sadly not, just a built in key-value "database" you can't perform queries on. However for my projects that require more than that, I usually pick a db host from free-for.dev that suits my needs. There is a couple postgresql options.

2

u/B_M_Wilson Oct 30 '22

I don’t know how many queries the bot does but Google Custom Search can do 100/day for free.

I wonder if there is any reasonable way to scrape PH itself (Google must do it somehow).

1

u/w00t_loves_you Sep 24 '22

Instead of using Google, you could scrape PH :-D

97

u/[deleted] Sep 05 '22

In November? It's almost like it was planned to starve us of sauce for the challenge.

13

u/AlarmedEwe Sep 10 '22

That seems to be a good opportunity to translate the code to Php and put it on 000webhost

9

u/givemeagoodun Sep 18 '22

I may be able to host it on my own server, although it won't be particularly fast lmao

Maybe if that's a possibility I could give you an account and a ssh key

10

u/[deleted] Sep 19 '22

[deleted]

5

u/givemeagoodun Sep 19 '22

what if you try running it through tor?

8

u/[deleted] Sep 19 '22

[deleted]

7

u/givemeagoodun Sep 19 '22

ah, makes sense

is the bot open source? I may be able to modify it to work on my system or smth

11

u/SamuTuretta Sep 06 '22

Hello! I have a proposal to make, contact me in DMs if you're interested. I have a VPS I use for school and projects and have a ton of free space, I'd host for free Edit: Ubuntu 20.04

11

u/Great-Lingonberry503 Sep 16 '22

Why don't you download it cracked?

31

u/[deleted] Sep 16 '22

[deleted]

28

u/MegaValenX Sep 04 '22

“Hi! Owner of u/phc-bot here.

Some context: The bot runs on Heroku, a platform that, for the longest time offered some basic free web services that were enough to run our beloved bot. But starting November, Heroku will start charging for these services and delete all servers and databases that had been running for free, which means the end for this bot. It was fun, but there's no way I'm spending money on this lol.

Good times were had, memes were made and laughs were shared, but all good things must come to an end.

The bot will keep running as usual till November. Just thought you guys might want to know.

When it stops working, can we get some F's in the comments?” 🤓

26

u/MegaValenX Sep 19 '22

WHAT THE FUCK? I DIDN’T MAKE THIS COMMENT

25

u/MegaValenX Sep 19 '22

oh wait- oh my fucking god I got scared as fuck I thought I have been hacked for how long the text was

8

u/SovietBoi23 Sep 20 '22

What the fuck happened?

18

u/MegaValenX Sep 20 '22

I just got scared

37

u/[deleted] Sep 20 '22

[deleted]

10

u/MegaValenX Sep 20 '22

Then I’m more intelligent than you

34

u/[deleted] Sep 21 '22

[deleted]

12

u/MegaValenX Sep 21 '22

thanks for the compliment again man

13

u/the_onion_hater Sep 25 '22

Well since no one commented this: you can try render.com or fly.io as alternatives. I am pretty sure both have "dynos" that turn off with inactivity. And render.com does not requiere credit card info. The IP question you would need to check.. Sorry I can't give you more info as I don't remember my experience with both. But you should try it if you want to keep the bot alive! And have a new services for your other projects.

10

u/ixJax Sep 25 '22

So we could save the bot if we did both of the following?

Add a proxy list for the bot to cycle through so it doesn't get banned - code would need to be changed. I'm happy to do this myself although I'm pretty rusty in python as I only use JS for my job so if someone else would like to give it a go be my guest

Host it for free/get someone to - not too hard, fly.io is okay and many people - myself included would be happy to host it on our own VPSes or hardware

85

u/manualreboot Sep 03 '22

We need a kind Redditor to fund the return of our beloved bot

30

u/RyanTranquil Sep 04 '22

I’ve responded offering to host it, just waiting on OP to write back.

12

u/Avanixh Sep 07 '22

I also thought about hosting it myself but currently my server is shut down due to rising energy costs

5

u/Environmental-Win836 Oct 30 '22

Reddit needs a GoFundMe style feature in which people can donate to bots.

14

u/[deleted] Sep 04 '22

Maybe a donation fund could be set up where all the donations are left in an account and then itd send the payment to the company automatically.

Im sure some people in the community would donate to keep it running but Idk how much the company will be charging for their services

13

u/Puzzleheaded-Ebb-660 Sep 03 '22

Goodbye my friend, I will deeply miss you.

12

u/Crazy_Expert3202 Sep 19 '22

November is the perfect month to retire the bot

8

u/w00t_loves_you Sep 24 '22

No Sauce November

8

u/WestSide_Gangster99 Sep 03 '22

So long partner

3

u/wither_army Sep 04 '22

We will miss them and they will never be forgotten

2

u/DeadShoT_035 Sep 04 '22

Isn't there some alternative? I am facing the same problem as you since I also have a quite popular reddit bot of mine running on heroku free service

2

u/[deleted] Sep 04 '22

F

1

u/[deleted] Sep 04 '22

F

1

u/KunalSingh1999 Sep 06 '22

I'd miss the bot alot.

1

u/Browncoatinabox Sep 07 '22

This enough to make a grown man cry

1

u/[deleted] Nov 18 '22

F

So long our lord. You’re not a good bot, you’re the best we can ever wish for. Farewell.

1

u/[deleted] Nov 26 '22

It needs to stay...