We Saved $500k Per Year by Rolling Our Own "s3"
Posted2 months agoActive2 months ago
engineering.nanit.comTechstoryHigh profile
heatedmixed
Debate
80/100
Cloud StorageCost OptimizationBaby Monitors
Key topics
Cloud Storage
Cost Optimization
Baby Monitors
Nanit saved $500k/year by implementing an in-memory cache to reduce S3 costs, but the discussion raises concerns about their architecture and the ethics of their baby monitor product.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
4h
Peak period
65
6-12h
Avg / period
20
Comment distribution160 data points
Loading chart...
Based on 160 loaded comments
Key moments
- 01Story posted
Oct 26, 2025 at 5:05 PM EDT
2 months ago
Step 01 - 02First comment
Oct 26, 2025 at 9:26 PM EDT
4h after posting
Step 02 - 03Peak activity
65 comments in 6-12h
Hottest window of the conversation
Step 03 - 04Latest activity
Oct 29, 2025 at 12:59 PM EDT
2 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
ID: 45715204Type: storyLast synced: 11/20/2025, 8:28:07 PM
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
Sticking something with 2 second lifespan on disk to shoehorn it into aws serverless paradigm created problems and cost out of thin air here
Good solution moving at least partially to a in memory solution though
Regardless, I enjoyed the article and I appreciate that people are still finding ways to build systems tailored to their workflows.
> Disable keep-alive: close the connection immediately after each upload completes.
Very odd idea.
Using S3 for an MVP and marking this component as “done” seems like the right solution, regardless of the serverless paradigm.
Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.
Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and...
Without on-prem, saving a file is as simple as s3.put_object() !
> Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and...
Wow that's a lot to learn before using s3... I wonder how much it costs in salaries.
> With what network topology?
You don't need to care about this when using SSDs/HDDs.
> With what access policies?
Whichever is defined in your code, no restrictions unlike in S3. No need to study complicated AWS documentation and navigate through multiple consoles (this also costs you salaries by the way). No risk of leaking files due to misconfigured cloud services.
> With what backup strategy?
Automatically backed up with rest of your server data, no need to spend time on this.
You do need to care when you move beyond a single server in a closet that runs your database, webserver and storage.
> No risk of leaking files due to misconfigured cloud services.
One misconfigured .htaccess file for example, could result in leaking files.
First, I hope nobody is using Apache anymore, second, you typically store files outside of web directory.
> One misconfigured .htaccess file for example, could result in leaking files.
I don't think you are making a compelling case here, since both scenarios result in an undesirable exposure. Unless your point is both cloud services and local file systems can be equally exploited?
> Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and...
Most of these concerns can be addressed with ZFS[0] provided by FreeBSD systems hosted in triple-A data centers.
See also iSCSI[1].
0 - https://docs.freebsd.org/en/books/handbook/zfs/
1 - https://en.wikipedia.org/wiki/ISCSI
There may be some additional features that S3 has over a direct filesystem write to a SSD in your closet. The people paying for cloud spend are paying for those features.
Question: How do you save a small fortune in cloud savings?
Answer: First start with a large fortune.
I think you mean a small fraction of 3 engineers. And small fractions aren't that small.
The end of the article has this:
> Consider custom infrastructure when you have both: sufficient scale for meaningful cost savings, and specific constraints that enable a simple solution. The engineering effort to build and maintain your system must be less than the infrastructure costs it eliminates. In our case, specific requirements (ephemeral storage, loss tolerance, S3 fallback) let us build something simple enough that maintenance costs stay low. Without both factors, stick with managed services.
Seems they were well aware of the tradeoffs.
Also, just take an old phone from your drawer full of old phones, slap some free camera app on it, zip tie a car phone mount to the crib, and boom you have a free baby monitor.
Have you ever thought of using a postgresql db (also on aws) to store those files and use CDC to publish messages about those files to a kafka topic? In your original way, we need 3 aws services: s3, lambda and sqs. With this way, we need 2: postgresql and kafka. I'm not sure how well this method works though :-)
1GB with the bytea data type (https://www.postgresql.org/docs/current/datatype-binary.html) and 4TB with the BLOB data type (https://wiki.postgresql.org/wiki/BinaryFilesInDB).
So if you have experience with this and it did work well, I'm curious to hear about it! That's why i asked about if it worked well, not about the maximum size postgres allowed in various data types.
If you have no experience with it, but are just posting advice based on what AI tells you about max sizes of data allowed by pg that I can get from the same source too, then okay, fair enough, and certainly no need to give me any more of that!
Why hesitant? Just ask AI. It'll tell you how to do it and then you can experiment it yourself.
My first thought is, why bother with local storage if your turnaround on video chunks is 2 seconds? What's disk going to add besides a little bit more resiliency in that 2 second time frame? This at the cost of having slower pod startups given you have to mount the PVC, and a small performance hit of writing to a filesystem instead of memory.
All moot anyway given that the cameras/proxy allegedly has retries built-in, but interested to hear your thoughts.
Resiliency is the point. How would you protect against machine's loss/crash?
Nanit needs this storage because they run cloud based baby cameras. Every Nanit user is uploading video and audio of their home/baby live to Nanit without any E2EE. It's a hot mic sending anything you say near it to the cloud.
Their hardware essentially requires a subscription to use, even though it costs $200/camera. You must spend an additional $200 on a Nanit floor stand if you want sleep tracking. This is purely a software limitation since there's plenty of other ways to get an overhead camera mount. (I'm curious how they even detect if you're using the stand since it's just a USB-C cable. Maybe etags?)
Of course Nanit is a popular and successful product that many parents swear by. It just pains me to see cloud based in-home audio/video storage being so normalized. Self-hosted video isn't that hard but no one makes a baby-monitor centric solution. I'm sure the cloud based video storage model will continue to be popular because it's easy, but also because it helps justifies a recurring subscription.
edit: just noticed an irony in my comment. I'm ranting about Nanit locking users into their 3rd party cloud video storage, and the article is about Nanit's engineering team moving off a 3rd party (S3) and self-hosting their own storage. Props to them for getting off S3.
The comment you’re replying to literally started by saying you don’t need it.
The v-tech ones are fine though. No need for anything with an Internet connection (though some of them even do now).
Source: also 2 kids.
I wish we could have local-first and e2ee consumer software for this sort of thing, but given the choice of that or actually usable software, I am going to pick the latter.
But I do miss the lack of any baby-specific features like sleep tracking. It has support for crying detection, but that's it.
It's not really self hosted since it relies on Apple but it's the least evil at this point. Giving unencrypted video and audio to some company (if what OP says is right) would be way beyond my risk tolerance point.
My definition of "trivial" seems to be different.
If you don't want a baby camera system that's also a part-time hobby...Nanit does seem like the best option. I just lament that the best option requires giving up so much.
I have a little server rack cobbled together out of wood under my basement stairs, with a UDM Pro, 24 port POE switch, and an ancient Dell 2U poweredge for TrueNAS.
Unifi accidentally made a fantastic baby monitor.
The recent APIs they’ve built makes me hopeful that I could run an AI model against the footage eventually and build those Ai features for myself.
You just need a console (NVR) and the a camera. Here's what I use:
- https://store.ui.com/us/en/category/cloud-gateways-compact/c...
- and a wireless camera: https://store.ui.com/us/en/products/uvc-g6-ins
and the camera has a standard 1/4" female thread mount, so also a stand to hold the camera. And in the UniFi protect setting enable "Hallway Mode" to rotate it 90 degrees to get the length of baby.
- For a Unifi Protect console (basically an NVR), there's lots of other options too. The one I linked I also use as my Network controller and wifi gateway. I assume it's possible to just use it as a Protect server/NVR (but I also recommend Unifi Wifi)
- The G6 camera, since its meant for normal security uses, is wide by default. But I like it flipped 90 degrees for baby monitoring. Just find the advanced setting that lets you enabled "Hallway Mode" and it flips it 90 degrees so you a tall skinny screen
- Also, to save money, you can get an UCG-Max (linked above) without an SSD, then just install your own nvme SSD. Can get a semi-cheap one since you don't really need super high write and read speeds. But decent endurance would be good.
- I also use the 'privacy area' feature to black out areas in my camera that aren't baby. That way I make sure I'm not recording anything else, and it makes the video storage more efficient.
- You can run Tailscale on the UCG-Max which is another way to enable out-of-home access if you don't want to enable Ubiquiti's cloud services: https://github.com/SierraSoftworks/tailscale-udm
It's not perfect because wifi networks might block the VPN, but for the one wifi network I use the most, Wireguard on port 53 works splendidly, for now.
My impression is live feed is a solved problem.
They've mostly sold off bits of themselves, and/or licensed their name to other producers. It's highly unlikely that Philips actually made that camera.
I had to set up a dedicated Nanit-only AP in my house in order to stabilize the connection. It would not work any other way, tried many different configurations, even other APs.
if i'm understanding "anywhere you happen to be" right: Real question -- I'm not a parent. What is your use case for wanting to monitor your baby remotely from a different location than your baby? Obviously someone is with them at the house or location with the baby! You don't trust em? Or just like seeing/hearing your baby when you are out?
I see why a baby monitor in general is helpful so you can be in another room in the house and still keep an eye/ear on baby, but obv someone has to actually be in the location with the baby! (and the monitor at least needs to be on the wifi, right? So the monitor is in a place you have network access to, yes?)
* Doing garage or yard work where Wifi coverage was spotty. May seem like an edge case but remember that when baby is sleeping is exactly when you want to be doing things like yard work.
* Hanging out across the street cooking out with the neighbors while baby sleeps
* Having a couple drinks at the hotel bar on vacation after baby goes to sleep. You're only ~30 seconds from your room if baby wakes up, but it's nice to not have to sit in a dark room for the whole evening after 7pm.
- I'm at a small party 1 block away. Baby is sleeping in the bedroom with mama but I'm trying to protect her sleep. I listen to baby with an airpod in my ear at the party. If baby shows signs of waking I come back and either bottle him or help mama feed him.
Also just because I'm out of the house and miss my baby and want to stare at him...
Self-hosting video is not something the typical user of a baby monitor would ever even consider.
I'm not leaving a baby at home while I go on vacation. I would never be on another network, even. Why need the cloud?
The typical parent has never heard of Synology or Ubiquiti, doesn’t have a NAS, and gets whatever tech their ISP gave/rents them.
A common use case for baby monitors is being able to wander short distances away and still listen in: Work in yard, talk to a neighbor, go out to the detached garage.
Having a baby monitor which is not tethered to the WiFi coverage is a selling point. Many cheap monitors are WiFi connected or use their own WiFi network and the range is limited.
A lot of people in this thread are also completely missing the selling points of Nanit which include breathing tracking and sleep tracking features. It’s a product that could technically be implemented locally with enough extra processing power and cloud servers for coordinating out of home access and bouncing notifications, but obviously the number of people who would pay extra for that (instead of trying to roll their own solution HN style) is not large.
I’d least I never heard a parent complain that their biggest problem dealing with a baby is lack of E2EE.
In that case no parent needs to know about Synology or even IP addresses.
But they need to know about networking enough to be on the same network. I understand that sounds easy, but every time someone gets confused about their cursed setup the company making the device will get a returned product and an angry review. Client isolation, multiple wifi networks, some devices being on wifi some on the mobile network.
Pay money to solve a problem and time-save as a parent is a valid business idea/strategy. The externalities that the parents might suffer if these businesses do not completely adhere to good security practices don't seem to come back to bite them (and most parents get lucky and not have any bad consequences - yet).
Maybe you want to check up on the babysitter (as creepy as that sounds, there might be good reasons). Or you're traveling but your partner is home, and you want to be able to see your sleeping child from half a world away.
I do think we've gone to far in the direction of cloud-only, but I don't think it's a bad option of have. The problem I have is that many of the companies running these services have really terrible security. So for S3 for a nanny cam, I'd assume that each customer have their own bucket, with their own credentials, but I doubt that's the case.
I hope you do tell them in advance. Secret surveillance is indeed in the creep territory.
Not sure about your setup, but I replied to this.
From the product description though it sounds like sleep analysis is what you're paying for, which they do on servers analyzing the video.
It's hilariously crazy but we were given the cams as a gift so we stuck with them.
If UniFi Protect was re-skinned and had a bunch of its security camera complexity removed and optimized for the baby-camera use case it'd be normal consumer level friendly.
Your way of phrasing it makes it sound like it would be fine to upload the video if it were end-to-end-encrypted. I think this is worth clarifying (since many don’t really understand the E2EE trade-off): E2EE is for smart clients that do all the processing, plus dumb servers that are only used for blind routing and storage. In this instance, it sounds like Nanit aren’t doing any routing or (persistent) storage: the sole purpose of the upload is offloading processing to the cloud. Given that, you can have transport encryption (typically TLS), but end-to-end encryption is not possible.
If you wanted the same functionality with end-to-end encryption, you’d need to do the video analysis locally, and upload the results, instead of uploading the entire video. This would presumably require more powerful hardware, or some way of offloading that to a nominated computer or phone.
In the case of this product, there is only one client (and a server).
E2EE bills then down to having the traffic encrypted like you have with a https website.
In the case in point, the parent (camera owner) is one party and Nanit is another party. (Prior to the work in the linked post, AWS S3 was another party). The goal of E2EE is to deny plaintext access to some of these parties. So, in an E2EE deployment, Nanit (and AWS) would not have unencrypted access to the video content, even though they're storing it.
As chrismorgan pointed out, if Nanit did not have access to the unencrypted data, they could not do server-side video processing.
(Also, FWIW, there are multiple clients in this scenario -- the parents' phones are clients, and need unencrypted access to the video stream.)
(As an aside, where I used to work, we did some cool stuff with granting conditional access to certain server-side subsystems, so that the general data flow was all end-to-end encrypted, but customers could allow certain of our processes to be "ends" and have key access. This was really elegant; customers could dial in the level of server-side access that we had, and could see via the key authorization metadata which services had that access.)
> It’s all end-to-end encrypted
> The video is privately analyzed by your home hub using on-device intelligence to determine if people, pets, or cars are present.
You can use a cloud provider's infrastructure without giving it access to your material. My devices generate the content, my devices do the processing and analysis, I consume the content. The cloud just coordinates the data in flight, and stores it at rest, all encrypted. It's possible but most companies don't bother because they have to put effort and their "payoff" is that they can't monetize your data anymore.
I can absolutely imagine an architecture where video can be streamed in an encrypted manner, or stored in encrypted time-stamped blobs, allowing the server to provide rough searching, and then the client can perform fine-grained scanning.
This obviously doesn't enable any kind of processing of the video data on the server side, and doing it on the receiving client would require the feed to be active This means that any kind of processing would almost necessarily have to happen on the sending device, which would probably increase the power and compute requirements by a lot.
The third option is unreliable because if that "client" (a desktop app, a phone app, etc.) dies, then the process stops working completely. The second option is unreliable because if you increase the cost of the camera then most users will buy the other camera because everyone is financially constrained these days.
That basically just leaves the first option as the only practical one at an appealing price point.
That’s not what most people expect though.
Apple has done some interesting this with privacy-centric cloud processing. Might be some way to eventually get the benefits of cloud based detections without revealing your video.
also my other gripe is they also store audio. Which personally I feel like is even more sensitive. Wish their was an option to allow live audio listening but not store any audio in the cloud.
from [0]
https://en.wikipedia.org/wiki/SIDS
So, 24/7 kinda, yeah... Realistically, the risk is relatively low I'd say, so to still stay a functioning parent with other duties (for baby or otherwise), you don't look 24/7
https://en.wikipedia.org/wiki/SIDS
ox and hearth rate baby monitor definitely alert on sids. prevent, no, and that's why they are not medical devices, and wouldn't make sense to pay a randomized controlled trial to certify as one.
works? yeah. hearth stops beating, ox goes under parameter, parents get an alert.
here's the FDA statement about it https://www.accessdata.fda.gov/cdrh_docs/pdf22/K222597.pdf
It's much easier to create a device<->internet connection + a smartphone<->internet connection that it is to deal with the myriad of issues that occur if you try to do local device<->smartphone connections in networks with unknown topology and quirks (e.g. ISP provider being overly conservative in their firewall presets). If that in general would be a more trivial issue you would see less cloud services.
(You would probably still a similar amount of cloud services due the increased monetization options, but this would level the playing field for local-only options.)
- most people want to build lovely structures in the cloud, as it's hard to fix bugs in software on devices
- you'd need to open up a firewall on the home router
- auth might be tricky
- can't bolt on value added "enhancements"
Baby monitors around here -Alecto is a popular brand - cost twice as much and have only half the capabilities.
91 more comments available on Hacker News