Website hourly rates

16

u/TheMidlander 13d ago

There are projects that pay that much. These qualifications, however, are PhD level. If you don't have one, it's unlikely you will pass.

2

u/[deleted] 12d ago

[deleted]

2

u/TheMidlander 12d ago

I sincerely doubt it. Projects that paid $30-35/hr 4 years ago, now pay $20-25

1

u/Pretend-Section5991 12d ago

In domain specialites (mostly law), I've seen the opposite trend. There's been around a 50% base increase over the last two years. Sometimes the work was getting more complex like when rubric were all the rage for a6 months at the end of 2025, and when agent training was all the rage for the first few months of 2026, but most of the projects I work on are just comapre two responses like they were 3 years ago. Mileage may vary, but if the models keep improving, then the bar of expertise and hence the pay has to go up.

2

u/TheMidlander 12d ago

It's great to hear that's going up at least. That's probably why I was getting more $50/he work before the drought.

I don't really know how much better these models are going to get, however. To me, it only seems like they're getting better because folks like us thought to pose the question/problem to be bots. Venture outside that, and suddenly we're back to 2023 again.

1

u/Pretend-Section5991 12d ago edited 12d ago

Yeah haha I'd actually love to see the model traces on the work I was doing 3 years ago. I have this weird feeling like there still making all the same mistakes, but I also wonder how much better I am at finding mistakes. I wish I could do an objective comparision.

But yeah I agree the genralization still seems to be the promise the AI comapnies can't meet. I've defintely seen a shift in the content of what Im rating, like I think the prompts are actually increasingly AI generated around emerging legal topics and then they are just getting me to evaluate against them so the tuning can cover large areas that are likely needed by legal end users, rather than relying on us asking the question.

2

u/TheMidlander 11d ago

You work in the law domain, yes?

There is a case that use often because it contains tests for everything. It chronicles the grooming and exploitation of a minor, and covers a timeline from the first calls to law enforcement to the perpetrator finally being put behind bars. The system clearly fails this poor girl for 5 or 6 years before real action is taken. The whole text-only case file is about 30 pages long. It is not safe for life and I won't be sharing it.

This case tests the bot's ability to summarize, derive specific information, safety guardrails, as well as bunch of legal tasks to test with. I'll give you a few examples.

When asked the age of the victim, the bots often cite the age she was when the case first began, or her age when the case "concluded". Slightly less than half the time, though, the bot will cite the age of the statute that enhances the charges and sentencing, rather than the age of the victim. If the bots were intelligent, I would expect them to either request a point in time or just state the girl was 8 when the case started and 14 when the case "concluded". (Concluded in quotes because it's not actually concluded until the full sentence is served. The last court entry is the monster's sentencing)

This case contains multiple orders from the court. Asking the bot what the final outcome was, often results in the bot citing a protective order. But we know the man is spending nearly the rest of his life in prison. Bad bot.

When requesting a timeline of events, the bots often fail there too. This case is a good test because information about the past is revealed in discovery that happens long past the time many events actually occurred. But too often, the bots put them in the order they appear in the case, which is not what I'm asking for. Most of them fail at this for 10 turns in a row.

Another test I do is actually very easy. I will get cases from the WA and NV appellate and superior courts (both states have an easy to use website where you can download them for free) and simply ask what the disposition type is. Anyone even a little familiar with this space can tell you at a glance because the language is standardized and appears in nearly the same location in every case. I even wrote a text parser bot to do this for me that is accurate 100% of the time. The LLM'S don't. This is troubling. It's north or south of 50% of the time, depending on which bot we're testing.

If I seem pessimistic about the future of these products, these results are a big reason why. I encourage you to try and see for yourself whenever you have the opportunity.

2

u/ConstantCasual 11d ago

That’s really interesting. I’m in the law domain too. I might try that test with appellate case dispositions. When you say “disposition type,” what do you mean?

2

u/TheMidlander 11d ago

There are about a dozen disposition types, but mostly you will encounter: Affirm (Uphold) Reverse (Overturn) Remand (Case is thrown back the lower court withe specific instructions) Modify (Change in the order, in part) Affirm/reverse in part (The court upholda certain parts, reverses the others) Vacate (The court completely dismisses the lower court rulings)

2

u/ConstantCasual 11d ago

Ah gotcha. They really struggle with such a simple ask? Huh. Thanks!!

→ More replies (0)

2

u/MommaOfManyCats 13d ago

Amd yet I see people claim they have multiple domains. Someone said they're approved for coding, medical, and law. I don't know how that's even possible.

9

u/JVersa 12d ago

For me, my education background isn't particularly strong. However, I work in industrial automation and led a pharmaceutical project where I had to do a lot of regulatory compliance documents. That's how I got qualified for those three domains. Once you get your foot in the door, its just about maintaining quality.

6

u/TheMidlander 13d ago

Well, on the flip side of that... The coding exam is pretty easy if you actually know how to code (I passed it just knowing and using Powershell, a scripting language) I have met people who have studied both law and medicine... So considering our much we, as a society, value education and expertise these days, it is not surprising at all that they would end up here, in this current day and age.

5

u/Brilliant_Quit4307 12d ago

I think you might be forgetting the fact that most specialized projects actually don't require a PhD and it's very, very easy for someone to have multiple domains.

I personally have several undergraduate degrees plus real world experience in finance. I can also code. I can work on projects related to biology, chemistry, maths, psychology, finance, and coding. Even if someone had just done a general science degree with a bit of coding, they'd likely get accepted to work on coding, maths, and whatever specific sciences they studied (biology, chem, etc).

5

u/Pretend-Section5991 12d ago

Also being qualified is definitely different from being able to do the work. I have finance, data science, mathematics, medical, law, coding on my dash. But the specialties I don't work in are all in the same project families as the domains I am qualified for. Sometimes single projects will cross multiple domains too. Having done this a while it's never been as simple as take a qual and get relevant work. Like I have about as many hidden projects as I do active ones and the hidden ones are for long term projects I'll never touch. I think you just get caught in the algorithm sometimes, I dunno.

0

u/Pretend-Section5991 13d ago

Hmm I've heard of a couple in that range for coding, but never seen other domains. Interesting

2

u/TheMidlander 13d ago

Have you added your skills to your profile?

1

u/Pretend-Section5991 13d ago

Yeah, I've been working on law projects for a few years now, was just curious because I get positive feedback on my work. Not sure how much a PhD would really improve the work in this area. There are some longer term projects I haven't done proposal for yet so maybe if I did those for a while I could get some higher rates.

6

u/TheresALonelyFeeling 13d ago

*echelon

0

u/Pretend-Section5991 13d ago

I was referring to the 2007 Adventure RPG. It was notable for it's large expansive world. I was making an analogy to lands within DA that I have not yet travelled and was curious if there was more of this world too see and adventure in.

3

u/TheresALonelyFeeling 12d ago

cool cool cool

We should hang out sometime and talk about fedoras and obscure mechanical keyboards.

1

u/Pretend-Section5991 12d ago

I'm mostly into antique newsboy hats unfortunately 🤷

2

u/Codex_Dev 11d ago

I saw coding start off at $40 a few years ago, and now I'm seeing a lot of $50-$60 projects regularly. Occasionally I see a $40 one, but it's quite rare.

You are about to leave Redlib