Not

Hacker

News!

Home

Hiring

Not

Hacker

News!

AI-observed conversations & context

Daily AI-observed summaries, trends, and audience signals pulled from Hacker News so you can see the conversation before it hits your feed.

LiveBeta

Explore

Home
Hiring
Products
Companies
Discussion
Q&A
Privacy Policy

Resources

Visit Hacker News
HN API
Modal cronjobs
Meta Llama

Briefings

Inbox recaps on the loudest debates & under-the-radar launches.

Connect

Not affiliated with Hacker News or Y Combinator. We simply enrich the public API with analytics.

Not

Hacker

News!

Home
Discussion
Batch Inference

Batch Inference

2 stories

•

24h: 0%

•

7d: 0

•

1 comments

Top contributors:ykev DISCURSIVE

Stories

Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale

51 commentsby ykev

Postedabout 2 months agoActiveabout 1 month ago

Cutting LLM Batch Inference Time by Half with Dynamic Prefix Bucketing

20 commentsby DISCURSIVE

Postedabout 1 month agoActiveabout 1 month ago

Not

Hacker

News!

AI-observed conversations & context

Daily AI-observed summaries, trends, and audience signals pulled from Hacker News so you can see the conversation before it hits your feed.

LiveBeta

Explore

Home
Hiring
Products
Companies
Discussion
Q&A
Privacy Policy

Resources

Visit Hacker News
HN API
Modal cronjobs
Meta Llama

Briefings

Inbox recaps on the loudest debates & under-the-radar launches.

Connect

Not affiliated with Hacker News or Y Combinator. We simply enrich the public API with analytics.

Batch Inference | Trending Topic on Hacker News | Not Hacker News!

Batch Inference

Related Stories

Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale

Cutting LLM Batch Inference Time by Half with Dynamic Prefix Bucketing