Diffusion Finetuning Myself

Posted3 months agoActive3 months ago

frotaur

14 points

5 comments

vassi.lifeTechstory

calmpositive

Debate

20/100

Diffusion ModelsFine-TuningAI

Key topics

Diffusion Models

Fine-Tuning

The author shares their project on fine-tuning a diffusion model on themselves, sparking a discussion on the implications and technical details of such a process.

Snapshot generated from the HN discussion

Discussion Activity

Light discussion

First comment

Peak period

Day 1

Avg / period

2.5

Key moments

01Story posted
Sep 24, 2025 at 3:44 PM EDT
3 months ago
Step 01
02First comment
Sep 24, 2025 at 4:49 PM EDT
1h after posting
Step 02
03Peak activity
4 comments in Day 1
Hottest window of the conversation
Step 03
04Latest activity
Oct 5, 2025 at 3:55 PM EDT
3 months ago
Step 04

Generating AI Summary...

Analyzing up to 500 comments to identify key contributors and discussion patterns

Discussion (5 comments)

Showing 5 comments

throwaway314155

3 months ago

1 reply

Your concern about catastrophic forgetting is mostly unfounded in the regime of fine-tuning large diffusion models. The weights in this case will maybe suffer from some damage to accuracy on some downstream tasks. In general though, it is not “catastrophic”. I believe this is due to the attention mechanism but I’m happy to be corrected.

frotaurAuthor

3 months ago

1 reply

I see, it was probably my high learning rate that caused problems. To be honest, I got a bit lazy to retry full finetuning since LoRA worked so well, but maybe I'll revisit this in the future, maybe with Qwen Image.

throwaway314155

3 months ago

Perhaps what you were dealing with was actually exploding gradients using fp16 training which _are_ prone to corrupting a model and this can depend on the learning rate.

adzm

3 months ago

1 reply

Minor observation: the formula text appears to go above the sticky header in the website.

frotaurAuthor

3 months ago

True, I hadn't noticed, thanks! I'll try to fix that in the near future.

Edit : Fixed!

View full discussion on Hacker News

ID: 45365107Type: storyLast synced: 11/20/2025, 5:57:30 PM

Want the full context?