Antislop: a Framework for Eliminating Repetitive Patterns in Language Models

That's sloppy (hehe), if you are going to redefine a common word for the first time (i.e. references are not possible) at least do it explicitly.

Colloquially it means ‘poor quality’ and always has done. So buzzfeed is journalism slop, just like poor quality AI content is AI slop.

Anecdotally, I use them less often these days, because of the association with AI.

Why?!

I want the star trek experience. The computer just says "working" and then gives you the answer without any chit-chat. And it doesn't refer to itself as if it's a person.

What we have now is Hal 9000 before it went insane.

They could hide it so that it doesn't annoy you, but I think it's not a waste of tokens. It's there so the tokens that follow are more likely to align with what you asked for. It's harder for it to then say "This is a lot of work, we'll just do a placeholder for now" or give otherwise "lazy" responses, or to continue saying a wrong thing that you've corrected it about.

I bet it also probably makes it more likely to gaslight you when you're asking something it's just not capable of, though.

- how to cope with the sadness of losing their cat

- ranting about the annoying habits of their friends

- finding all the nice places to eat in a city

etc.

They do not want that "robot" personality and they are the majority.

Was this after many iterations? Try letting it get some "sleep". Hear me out...

I haven't used Codex, so maybe not relevant, but with Claude I always notice a slow degradation in quality, refusals, and "<implementation here>" placeholders with iterations within the same context window. One time, after making a mistake, it apologized and said something like "that's what I get for writing code at 2am". Statistically, this makes sense: long conversations between developers would go into the night, and they get tired, their code gets sparser and crappier.

So, I told it "Ok, let's get some sleep and do this tomorrow.", then the very next message (since the LLM has no concept of time), "Good morning! Let's do this!" and bam, output a completely functional, giant, block of code.

Human behavior is deeeeep in the statistics.

Schools will need to reinvent themselves in some ways.

I don't. I think they're useful for flagging the existence of mode-collapse and also providing convenient tracers for AI-written prose. Erasing only the verbal tics with the equivalent of 's/ - /; /g' (look ma! no more 4o em dashes!) is about the worst solution you could come up with and if adopted would lead to a kind of global gaslighting. The equivalent of a vaccine for COVID which only suppresses coughing but doesn't change R, or fixing a compiler warning by disabling the check.

If you wanted to do useful research here, you'd be doing the opposite. You'd be figuring out how to make the verbal expressions even more sensitive to the underlying mode-collapse, to help research into fixing it and raising awareness. (This would be useful even on the released models, to more precisely quantify their overall mode-collapse, which is poorly captured by existing creative writing benchmarks, I think, and one reason I've had a hard time believing things like Eqbench rankings.)