DeepSearch

github.com

mvplibrarydevelopers

Key Features

high-performanceSMB directory scanning

Tech Stack

Rust

smb

directory-scanner

network-security

data-privacy

Discussion (3 comments)

Showing 3 comments

dohuyhoangvn93

15 days ago

1 reply

Hi HN,

I'm the author of DeepSearch.

I built this while dealing with large SMB shares over WiFi, where traditional directory scanning tools were either too slow or caused unnecessary network load.

The main idea is a layered scanning strategy: - Minimize SMB round-trips - Use Rayon for controlled parallelism - Avoid spawning excessive IO-bound tasks - Keep memory usage predictable

This is not meant to be a fancy UI tool, but a fast and deterministic building block that can be embedded into other systems.

The project is open-source (Apache 2.0). I'm especially interested in feedback on: - The concurrency model - SMB traversal strategy - Any edge cases you've seen in similar systems

Thanks for taking a look.

holg

15 days ago

1 reply

Hi thx for sharing, in general

about concurrency Rayon excels on CPU-heavy tasks, but this is more IO

for traversal avoid any extra round-trips, simple the metadata already given

edge-cases would be infinite loops on symlinks, the Windows 255 char limits and on WIFI packet loss due to too many reqs

dohuyhoangvn93

14 days ago

Thank you for your feedback. In the initial CLI test, I tried using Tokio and found that performance didn't improve and the earliest result return speed was quite poor. I skipped the root cause analysis and kept the original rayon design.

I will try redoing the tests and taking more detailed measurements for evaluation.

Thank you again.

View on Hacker NewsHN ID: 46354733Mood: informative

Privacy Policy

Resources

Briefings

Inbox recaps on the loudest debates & under-the-radar launches.

Connect

Not affiliated with Hacker News or Y Combinator. We simply enrich the public API with analytics.