Adaptive Multirate Dsp Wrappers Around GPT

Postedabout 1 month agoActiveabout 1 month ago

eladwf

1 points

1 comments

github.comTechstory

informative

Debate

10/100

GPTDspMultirate ProcessingDeep Learning

Key topics

GPT

Dsp

Multirate Processing

Deep Learning

The author has experimented with adding DSP-inspired multirate and LFO modules to GPT blocks, achieving lower validation loss and fewer FLOPs in small character-level GPT models. The results are exploratory and not a state-of-the-art claim. The code is available on GitHub with a detailed README.

Snapshot generated from the HN discussion

Discussion Activity

Light discussion

First comment

32s

Peak period

0-1h

Avg / period

Key moments

01Story posted
Nov 28, 2025 at 8:16 AM EST
about 1 month ago
Step 01
02First comment
Nov 28, 2025 at 8:17 AM EST
32s after posting
Step 02
03Peak activity
1 comments in 0-1h
Hottest window of the conversation
Step 03
04Latest activity
Nov 28, 2025 at 8:17 AM EST
about 1 month ago
Step 04

Generating AI Summary...

Analyzing up to 500 comments to identify key contributors and discussion patterns

Discussion (1 comments)

Showing 1 comments

eladwfAuthor

about 1 month ago

I’ve been experimenting with adding DSP-inspired multirate + LFO modules around GPT blocks and learning the DSP hyperparameters during training. This is only tested on small character-level GPTs (enwik8/text8), so it’s not a SOTA claim, just an exploratory result. In this setting I see ~12-19% lower best val loss and roughly 55-70% fewer FLOPs to reach fixed loss targets vs the same GPT baselines (5 seeds, FLOPs-to-target analysis).

The repo has a detailed README (with math and ablations) plus scripts to reproduce the experiments.

View full discussion on Hacker News

ID: 46078385Type: storyLast synced: 11/28/2025, 7:14:15 PM

Want the full context?

Jump to the original sources

Read the primary article or dive into the live Hacker News thread when you're ready.

Open link View on HN