Perkeep – Personal Storage System for Life

But it really is nearly abandoned, and outside of the happy-path the primary author uses it for, it's desolate. There is no community around growing its usage, and pull requests have sat around for months before the maintainer replies. Which is fine if that's what the author wants (he's quite busy!), but disappointing to potential adopters. I've looked at using it, but with data types that sit outside the author's use case, and you'd really need to fork it and change code all over the repo to effectively use it. It just never hit the ideal of "store everything" it promises when it has hard-coded data types for indexing and system support.

(and yes, I did look at forking it and creating my own indexer, but some things just aren't meant to be)

A permanent private data store needs to have straightforward ways to get that data into it, and then search and consume it again once there.

Kinda sad, as this looks interesting.

https://github.com/gildas-lormeau/SingleFile

I've been using an extension called WebScrapBook to locally save copies of interesting webpages. I use the basic functionality, but it comes with tons of options and settings.

For pages with dynamic content that can't be trivially reproduced by their HTTP streams— E.G., opening the archive triggers GETs with a mismatched timestamp, even if the file it's looking for is in the WARC under a different URI— There's always SingleFile [1], and Chromium's built-in MHTML Ctrl+S export, which "bake" the content into a static page.

0: https://chromewebstore.google.com/detail/webrecorder-archive...

1: https://github.com/gildas-lormeau/SingleFile

However, I regret this decision. Git-Annex is not usable anymore on my data because the amount of files has grown so much (millions) and Git-Annex is just too slow (it takes minutes up to even hours for some Git operation, and the FS is decently fast). I assume I would not have had those problems with Perkeep.

This is strange in the sense that (a) didn't stop the Linux kernel from becoming more popular - if the tool satisfies the itch, use it, otherwise not. And the lack of releases could be fine if the bugs reported are minor.

Is the tool robust (no data loss)?

What has other folks on here stopped from e.g. writing more importers (if that is the main shortcoming)?

edit: typo corrected

Indeed, big fan of the idea of Perkeep, and its authors (I learned a lot about writing network code in Go from reading from Brad Fitzpatrick's contributions.)

Where Perkeep uses a super cool blob server design that abstracts the underlying storage, Timelinize keeps things simpler by just using regular files on disk and a sqlite DB for the index (and to contain small text items, so as not to litter your file system).

Perkeep's storage architecture is probably more well thought-out. Timelinize's is still developing, but I think in principle I prefer to keep it simple.

I'm also hoping that, with time, Timelinize will be more accessible to a broader, less-technical audience.