We Ran Over 600 Image Generations to Compare AI Image Models

There is no better recent example than AI comedy made by a professional comedian [0]

Of course, this makes sense once you think about it for a second. Even AGI, without a BCI, could not read your mind to understand what you want. Of course, the people who have been communicating these ideas with other humans up to this point, are the best at doing that.

[0] old.reddit.com/r/ChatGPT/comments/1oqnwvt/ai_comedy_made_by_a_professional_comedian/

Photography didn’t make artists obsolete.

For that matter, the car didn’t make horse riding completely obsolete either.

For artists, the question is whether generative AI is like photography or the car. My guess, at this stage, is photography.

For what it’s worth I think the proponents of generative AI are grossly overestimating the utility and economic value of meh-OK images that approximate the thing you’ve asked for.

Maybe just the advent of the microwave oven is the analogy.

Either way, I am out. I have spent many days fiddling with AI image generation but, looking back on what I thought was 'wow' at the time, I now think everything AI art is practically useless. I only managed one image I was happy with and most of that was GIMP, not AI.

This study has confirmed my suspicions, hence I am out.

Going back to the fast food analogy, for the one restaurant that actually cooks actual food from actual ingredients, if everyone else is selling junk food then the competition has been decimated. However, the customers have been decimated too. This isn't too bad as those customers clearly never appreciated proper food in the first place, so why waste effort on them? It is a pearls and swine type of thing.

And some day the news will announce that the last human actor has died.

That aside, humans are necessary for making up new forms and styles. There was no cubism before Picasso and Braque, or pointillism before Seurat and Signac. I don’t think I’ve seen anyone argue that if you trained a diffusion model on only the art that Osamu Tezuka was exposed to before he turned 24 it would output Astro Boy.

Same will happen to music, artists etc. They won't vanish. But only a few per city will be left

I could see that changing in a few years.

To me, I like to think in times the model failed versus success. So what I did, is I looked every time at the worst result. To me, the one which stood out (negatively) is Gemini. OpenAI had some very good results but also some missing the mark. SeeDream (which I never heard of previously) missed the mark less often than Gemini, and at times where OpenAI failed, SeeDream came out clearly on top.

So, if I were to use the effects of the mentioned models, I wouldn't bother with Gemini; only OpenAI and SeeDream.

Even Sam Altman's "Ghiblified" twitter avatar looks nothing like him (at least to me).

Other models seem much more able to operate directly on the input image.

This leads to incredibly efficient, dense semantic consistency because every object in an image is essentially recreated from (intuitively) an entire chapter of a book dedicated to describing that object's features.

However, it loses direct pixel reference. For some things that doesn't matter much, but humans are very discerning regarding faces.

Chatgpt is architecturally unable to reproduce exactly the input pixels - they're always encoded into tokens, then decoded. This matters more for subjects for which we are sensitive to detail loss, like faces.

Peak quality in terms of realistic color rendering was probably the initial release of DALL-E 3. Once they saw what was going to happen, they fixed that bug fast.

Seedream will always alter the global color balance with edits.

It also helped to specify which other parts should not be changed, otherwise it was rather unpredictable about whether it would randomly change other aspects.

I’m editing mostly pics of food and beverages though, it wouldn’t surprise me if it is situationally better or worse.

It is just very hard to make any generalizations because any single prompt will lead to so many different types of images.

The only thing I would really say to generalize is every model has strengths and weaknesses depending on what you are going for.

It is also generally very hard to explore all the possibilities of a model. So many times I thought I seen what the model could do just to be completely blown away by a particular generation.

I am a small time youtuber

NanoBanana and Flux Kontext are the models that get closest to traditional SDXL inpainting techniques.

Seedream is a strong contender by virtue of its ability to natively handle higher resolutions (up to around 4 megapixels) so you lose less detail - however it also tends to alter the color palette more often then not.

Finally GPT-image-1 (yellowish filter notwithstanding) exhibits very strong prompt adherence but will almost always change a number of the details.

The main issue is latency and bandwidth across the oceans since Asia far away from the US where a lot of servers live, and even for services that are distributed, I live in a rural prefectural capitol of Japan 1000 km away from Tokyo where all the "Japan" data centers are, so my ping is always unimpressive despite the bandwidth.