‘overworked, Underpaid’ Humans Train Google’s AI

[0] https://youtu.be/0bF_AQvHs1M?si=rpMG2CY3TxnG3EYQ

But the next paradigm breakthrough is hard to forecast, and the current paradigm's asymptote is just as hard to predict, so it's +EV to say "tomorrow" and "forever".

When the second becomes clear before the first, you turk and expert label like it's 1988 and pray that the next paradigm breakthrough is soon, you bridge the gap with expert labeling and compute until it works or you run out of money and the DoD guy stops taking your calls. AI Winter is cold.

And just like Game of Thrones, no I mean no one, not Altman, not Amodei, not Allah Most Blessed knows when the seasons in A Song of Math and Grift will change.

This was one of the first links I found re: Scale’s labor practices https://techcrunch.com/2025/01/22/scale-ai-is-facing-a-third...

Here’s another: https://relationaldemocracy.medium.com/an-authoritarian-work...

And which are these universal human values and preferences ? Or are we talking about silicon valley's executives values ?

Depends how you look at it. I think a brand like Google should vet a mere one level down the supply chain.

to ensure the AI models are more aligned with Google's values and preferences.

FTFY

Or join an hospital as nurse, but then you are asked to perform surgery as you were a doctor?

There are serious issues outlined in the article.

It does so indirectly, so it's a true albeit misleading statement.

That's how validation works.

To counter your question, what makes you think that's not the case? Do you think Mistral/Moonshot/Qwen/etc. are all employing their own data labelers? Why would you expect this kind of transparency from for-profit bodies that are evaluated in the billions?

https://www.theverge.com/features/23764584/ai-artificial-int...

it explores the world of outsourced labeling work. Unfortunately hard numbers on the number of people involved are hard to come by because as the article notes:

"This tangled supply chain is deliberately hard to map. According to people in the industry, the companies buying the data demand strict confidentiality. (This is the reason Scale cited to explain why Remotasks has a different name.) Annotation reveals too much about the systems being developed, and the huge number of workers required makes leaks difficult to prevent. Annotators are warned repeatedly not to tell anyone about their jobs, not even their friends and co-workers, but corporate aliases, project code names, and, crucially, the extreme division of labor ensure they don’t have enough information about them to talk even if they wanted to. (Most workers requested pseudonyms for fear of being booted from the platforms.) Consequently, there are no granular estimates of the number of people who work in annotation, but it is a lot, and it is growing. A recent Google Research paper gave an order-of-magnitude figure of “millions” with the potential to become “billions.” "

I too would love to know more about how much human effort is going into labeling and feedback for each of these models, it would be interesting to know.

The business process outsourcing companies labelling things for AI training are often the same outsourcing companies providing moderation services to facebook and other social media companies.

I need 100k images labelled by the type of flower shown, for my flower-identifying AI, so I contract a business that does that sort of thing.

Facebook need 100k flagged images labelled by is-it-an-isis-beheading-video to keep on top of human reviews for their moderation queues. They contract with the same business.

The outsourcing company rotates workers between tasks, so nobody has to be on isis beheading videos for a whole shift.

Even theoretically.

These types of articles always have an elitist view of the workers hired. That's a big source of the right (in the US) despising the left. The left don't say it directly, but when they talk about how shitty their town is and how the job they have is exploitative, there's an implicit judgment on the persons who live/work there.

How do I join?

For reference, the median hourly wage is $27/hour.

https://nationalequityatlas.org/indicators/Wages_Median

Many experts are holding out, and I don't blame them. Why would you want to train AI to replace your job?

Unfortunately, I decided to take software engineering more serious and try to make it my career and then the entire market nosedived, with no signs of recovering anytime soon. Breaking into this market has more or less been impossible for a junior, and dare I say: a junior in their mid 30s. At least within this job I do get to work with code every so often, and I get to do it from home while I'm at it which is a bonus.

It's inconsistent so I'm still learning and looking for software, but for the meantime it's been incredible.

This reminds me of the early voice-to-text start ups in the 00's that had these miraculous demos that required people in call centers to type it all up and pretend it was machine.

Great to see that they have not learned from this experience, and are repeating the mistake with Gemini.

Are there companies that focus on labeling of inputs rather than RLHF of outputs?