High Performance Ssh/scp
Key topics
The quest for high-performance SSH and SCP has sparked a lively debate, with many commenters weighing in on the pros and cons of using HPN-SSH, a fork of OpenSSH that promises improved performance. While some are hesitant to adopt a fork due to concerns about security audits and potential vulnerabilities, the maintainers of HPN-SSH argue that their code is rigorously tested and reviewed. The discussion also veered into a tangent about OpenBSD's filesystem performance, with some defending UFS2's capabilities and others pointing out its limitations. As it turns out, HPN-SSH's patches have been languishing out-of-tree for over a decade, rejected by upstream OpenSSH, making the trade-offs between performance and security a pressing concern.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
5d
Peak period
71
Day 6
Avg / period
18.5
Based on 74 loaded comments
Key moments
- 01Story posted
Dec 10, 2025 at 11:23 PM EST
24 days ago
Step 01 - 02First comment
Dec 16, 2025 at 6:01 AM EST
5d after posting
Step 02 - 03Peak activity
71 comments in Day 6
Hottest window of the conversation
Step 03 - 04Latest activity
Dec 19, 2025 at 6:29 PM EST
15 days ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
- IF you don't trust it
- AND you want to use it
=> run it on a private network
You don't have to trust it for security to use it. Putting services on secure networks when the public doesn't need access is standard practice.
But I don’t miss having those limitations.
On what basis are making that claim? Because AFAICT, concern about it being less secure is entirely reasonable and is one of the big caveats to it.
You may as well just use tailscale ssh in that case. It already disables ssh encryption because your connection is encrypted with WireGuard anyway.
I've been developing it for 20+ years and if you have any specific questions I'd be happy to answer them.
OpenBSD's filesystem operations are slow not because of UFS2, but because they simply hasn't been optimized the way Ext4 has been Linux or UFS2 on FreeBSD. And of course, OpenBSD's implementation doesn't have a journal (both UFS and Ext had journaling bolted late in life) so filesystem checks can take a long time, which often causes people to think their system has frozen or didn't come up. That user interface problem notwithstanding, UFS2 is extremely robust. OpenBSD is very conservative about optimizations, especially when they increase code complexity.
I believe that these were recently removed. Perhaps they don't play well with SMP.
* https://en.wikipedia.org/wiki/Soft_updates
I often find myself needing to move a single large file rather than many smaller ones but TCP overhead and latency will always keep speeds down.
The design of sftp is such that it cannot exploit "TCP sliding windows" to maximize bandwidth on high-latency connections. Thus, the migration from scp to sftp has involved a performance loss, which is well-known.
https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers...
The rsync question is not a workable answer, as OpenBSD has reimplemented the rsync protocol in a new codebase:
https://www.openrsync.org/
An attempt to combine the BSD-licensed rsync with OpenSSH would likely see it stripped out of GPL-focused implementations, where the original GPL release has long standing.
It would be more straightforward to design a new SFTP implementation that implements sliding windows.
I understand (but have not measured) that forcibly reverting to the original scp protocol will also raise performance in high-latency conditions.
If you want to use the historic scp server instead, a command line option is provided to allow this:
"In case of incompatibility, the scp(1) client may be instructed to use the legacy scp/rcp using the -O flag."
https://www.openssh.org/releasenotes.html
The old scp behavior hasn't been removed, but you need to specifically request it. It is not the default.
If your BDP is less than 2MB you still might get some benefit if you are CPU limited and use the parallel ciphers. However, the fastest cipher is AES-GCM and we haven't parallelized that as of yet (that's next on the list).
Moving a terabyte database in an upgrade, I have connected three ports direct (no switch), then used xargs to keep all three connections busy with transferring the 2gb data files. I can get the transfer done in under an hour this way.
I don't currently have a performance need for an encrypted transfer, but one may arise.
I'm assuming that would have different limits than you outlined above, although I don't think they do multi core.
I thought openrsync existed solely because of rpki. Even OpenBSD devs recommend using the real version from ports.
[1] https://github.com/freebsd/freebsd-ports/blob/main/security/...
[2] https://github.com/NixOS/nixpkgs/blob/d85ef06512a3afbd6f9082...
I've been using mosh now for over a decade and it is amazing. Add on rsync for file transfers and I've felt pretty set. If you haven't checked out mosh, you should definitely do so!
The only change I see here that is probably harmless and a speed boost is using AES-NI for AES-CTR. This should probably be an upstream patch. The rest is more iffy.
The main performance improvement is from the buffer normalization. This can provide, on the right path, a 100x improvement in throughput performance without any compromise in security.
There are some minor usability bugs and I think both endpoints need to have it installed to take advantage. I remember asking ages ago why it wasn’t upstreamed, there were reasons…
As an aside - you only really need HPN-SSH on the receiving side of the bulk data to get the buffer normalization performance benefits. It turns out the bottleneck is almost entirely on the receiver and the client will send out data as quickly as you like. At least it was like that until OpenSSH 8.8. At that point changes were made where the client would crash if the send buffer exceeded 16MB. So we had to limit OpenSSH to HPN-SSH flows to a maximum of 16MB receive space. Which is annoying but that's still going to be a win for a lot of users.
The main problem is that it packetizes the data and waits for responses, effectively re-implementing the TCP window inside a TCP stream. You can only have so many packets outstanding in the standard SFTP implementation (64 is the default) and the buffers are quite small (32k by default) which gives a total outstanding data of 2MB. The highest transfer rate you can make depends on the latency of the link. If you have 100 ms of latency then you can send at most 20 MB/s which is about 200 Mbit/s - nowhere near filling a fast wide pipe.
You can tweak the buffer size (up to 256k I think) and the number of outstanding requests, but you hit limits in the popular servers quite quickly.
To mitigate this rclone lets you do multipart concurrent uploads and downloads to sftp which means you can have multiple streams operating at 200 Mbit/s which helps.
The fastest protocols are the TLS/HTTP based ones which stream data. They open up the TCP window properly and the kernel and networking stack is well optimized for this use. Webdav is a good example.
why is it designed this way? what problems it's supposed to solve?
It just has a in-flight message/queue limit like basically every other communication protocol. You can only buffer so many messages until you run out of space. The problem there is just that the default amount of buffering is very low and is not adaptive to the available space/bandwidth.
SFTP was designed as a remote file system system access protocol rather than transfer a single file like scp.
I suspect that the root of the problem is that SFTP works over a single SSH channel. SSH connections can have multiple channels but usually the server binds a single channel to a single executable so it makes sense to use only a single channel.
Everything flows from that decision - packetisation becomes necessary otherwise you have to wait for all the files to transfer before you can do anything else (eg list a directory) and that is no good for your remote filesystem access.
Perhaps the packets could have been streamed but the way it works is more like an RPC protocol with requests and responses. Each request has a serial number which is copied to the response. This means the client can have many requests in-flight.
There was a proposal for rclone to use scp for the data connections. So we'd use sftp for the day to day file listings, creating directories etc, but do actual file transfers with scp. Scp uses one SSH channel per file so doesn't suffer from the same problems as sftp. I think we abandoned that idea though as many sftp servers aren't configured with scp as well. Also modern versions of OpenSSH (OpenSSH 9.0 released April 2022) use SFTP instead of scp anyway. This was done to fix various vulnerabilities in scp as I understand.
I think maybe you are referring to QUIC [0]? It'd be interesting to see some userspace clients/servers for QUIC that compete with Aspera's FASP [1] and operate on a point to point basis like scp. Both use UDP to decrease the overhead of TCP.
0. https://en.wikipedia.org/wiki/QUIC
1. https://en.wikipedia.org/wiki/Fast_and_Secure_Protocol
However, I observed that curl [0] uses openssl' quic implementation (for one of its experimental implementations). Another backend for curl is Quiche [1] which has client and server components already, has the userspace crypto etc. It's a little confusing to me, but CloudFlare also has a project quiche [2] which is a Rust crate with a CLI to share and consume files.
0. https://curl.se/docs/http3.html
1. https://github.com/google/quiche/tree/main/quiche/quic
2. https://github.com/cloudflare/quiche
To be fair, that is not really a problem of the protocol, just the implementations. You can comfortably drive 10x that bandwidth with a reasonable design.
[1] https://microsoft.github.io/msquic/
I just think for streaming lots of data quickly HTTP/1.x plus TLS plus TCP has received many more engineering hours of optimization than any other combo.
0. https://en.wikipedia.org/wiki/Worse_is_better
Besides being faster, with rsync and the right command options you can be certain that it makes exact file copies, together with any file metadata, even between different operating systems and file systems.
I have not checked if in recent years all the bugs of scp and sftp have been fixed, but some years ago there were cases when scp and sftp were losing silently, without warnings, some file metadata (e.g. high-precision timestamps, which were truncated, or extended file attributes).
For instance, I always use this:
https://gist.github.com/rapier1/325de17bbb85f1ce663ccb866ce2...
[1]: https://mosh.org/