HN Meta: 99% Packet Loss on News.ycombinator.com

Posted5 months agoActive5 months ago

eqvinox

4 points

5 comments

Otherstory

skepticalnegative

Debate

20/100

Network IssuesPacket LossInfrastructure Problems

Key topics

Network Issues

Packet Loss

Infrastructure Problems

I'm currently seeing >99% packet loss to news.ycombinator.com (2606:7100:1:67::26):

  13. br06-te1-2.lwdc.americanis.net                   0.0%   231   14.3  17.2  14.0 172.5  19.5
  14. sdtc.sd-me01.te0-1.host1.20546.americanis.net   99.6%   231   12.9  12.9  12.9  12.9   0.0
  15. 2606:7100:1:67::26                              99.1%   231   11.8  11.8  11.8  11.8   0.0

Checked from both US and EU connections, same result, would be a very odd coincidence if it was a problem on my end. Trace looks like the uplink provider is having issues?

IPv4 seems to be unaffected.

User reports >99% packet loss to news.ycombinator.com, suggesting a potential infrastructure issue.

Snapshot generated from the HN discussion

Discussion Activity

Light discussion

First comment

Peak period

4-6h

Avg / period

1.7

Key moments

01Story posted
Aug 21, 2025 at 9:47 AM EDT
5 months ago
Step 01
02First comment
Aug 21, 2025 at 2:24 PM EDT
5h after posting
Step 02
03Peak activity
3 comments in 4-6h
Hottest window of the conversation
Step 03
04Latest activity
Aug 22, 2025 at 10:06 AM EDT
5 months ago
Step 04

Generating AI Summary...

Analyzing up to 500 comments to identify key contributors and discussion patterns

Discussion (5 comments)

Showing 5 comments

Bender

5 months ago

1 reply

Packet loss over ICMP can be artificial based on the backplane load of routers, ACL's and artificial rate limits. They will de-prioritize responding to ICMP and fail to respond if CPU load is high but that does not mean the packets are not being forwarded 100%. Linux and BSD also have knobs for this behavior as well. HN is BSD.

Are you seeing TCP retransmits to port 443?

    for i in $(seq 69);do nc -vz -w1 news.ycombinator.com 443;sleep 2;done

Do that at the same time as watching

    watch -d -n4 --no-title 'nstat -a|grep -Ei "ret|los"'

It would be most useful to do this from your location and also from VM's on a few different providers in different locations to find which thing is not like the other. The 'nc' I am using is part of the nmap distribution. To force IPv6 replace the name with the IPv6 address.

eqvinoxAuthor

5 months ago

1 reply

> Packet loss over ICMP can be artificial based on the backplane load of routers.

This doesn't generally apply to the end host; note the last hop in the trace I posted is the web server itself. Also, the second last hop having very similar loss% makes it likely that neither of the two systems is hitting rate limits (as they would be different).

> Are you seeing TCP retransmits to port 443?

If the webpage loaded normally, I wouldn't have investigated to begin with ;).

Right now, the loss on TCP SYNs is about 70%. (mtr -T -P 443) Second to last hop still has roughly the same loss. Something's h0rked at americanis.net.

Bender

5 months ago

1 reply

If you are confident where the loss is then the only thing you could really do within HN is email Daniel (dang) and ask him to replicate the issue and open a ticket with their upstream provider to check routes and for losses. hn@ycombinator.com

Outside of HN I suppose you could check the Nanog mailing lists [1] and chat servers to see if there is any discussion ongoing for those networks.

[1] - https://lists.nanog.org/mailman3/

eqvinoxAuthor

5 months ago

1 reply

I decided to wait a bit, and it's fixed now by way of "americanis.net" no longer being on the path.

Bender

5 months ago

Groovy. Glad it worked out.

View full discussion on Hacker News

ID: 44972745Type: storyLast synced: 11/18/2025, 1:46:21 AM

Want the full context?