Jmap for Calendars, Contacts and Files Now in Stalwart

https://stalw.art/compare/

> Stalwart Enterprise leverages AI technology to provide unparalleled email security and management. With AI-powered features, Stalwart Enterprise excels in accurately classifying spam, detecting sophisticated phishing attempts, and blocking various types of network attacks. This intelligent approach ensures that your email environment remains secure and reliable. Stalwart Enterprise comes equipped with a pre-trained large language model (LLM), offering robust out-of-the-box protection. Additionally, it supports integration with leading AI providers such as OpenAI, Anthropic, and other cutting-edge platforms, allowing you to enhance and customize your security measures. By utilizing AI, Stalwart Enterprise delivers a smarter, more efficient email solution that proactively safeguards your communications and data.

[0]: https://stalw.art/enterprise/

I made up ULFI because I thought MIME has some problems.

> JSON may not look efficient

Efficiency is not the only issue; there is also the consideration of e.g. what data types you want to use. JSON does not have a proper integer type, does not have a proper binary data type (you must encode it as hex or base64 instead), and is limited about what character sets can be used.

(Also, like other text formats, escaping will be needed.)

> I feel like "JSON over HTTP" is a subtle improvement over "custom text formats over telnet"

I think it can be, depending on the specific use; sometimes it isn't, and will make it worse. (HTTP does have the advantage of having URLs and virtual hosting, although I think it adds too much complexity more than should be needed.) However, I still think that DER is generally better than JSON.

> HTTP isn't the worst bootstrap layer the internet could use to build new protocols and apps on top of.

I think it depends on the specific application. However, even then, I think there are better ways than using HTTP with the complexity that it involves, most of which should not be necessary (even though a few parts are helpful, such as virtual hosting).

That's just not true. Telnet and SMTP are built on top of TCP. They live on the same layer. They were originally both protocols that transmitted data with printable ascii, hence why they look similar. There are many other protocols like Telnet and SMTP that worked like that, auch as nntp, irc, and yes, even http.

  [0] https://github.com/cypht-org/cypht
  [1] https://jmap.io/software.html

But in general, I agree that it has not been a very smooth experience. Having messed around with maddy and mox, Stalwart has had quite a few gotchas. Despite being a single binary promising simplicity, I'm finding it to be a real challenge figuring out how it all fits together, and I'm mostly learning by trial and error since the documentation is often outdated.

My biggest gripe is that it doesn't use the config.toml for every setting, or at least doesn't seem to have the option to do so. I broke my installation and had to find the posgresql key-value pairs for the settings, which was made harder by the fact that everything was stored as binary, which also made me have to edit it as binary as well. These were very simple settings that would have been a breeze in a flat configuration file. I absolutely do not like how necessary the WebAdmin is to manage simple things.

That said, the integration with calendar/contacts is nice even without JMAP... Getting Thunderbird and Roundcube setup with plugins and proper settings made it so easy to get several users setup with calendars, contacts, and shared email-boxes and shared contacts right upon first login.

The S3 storage is also working great (Hetzner Frankfurt VPS paired with AWS eu-central-1), and AWS downtime a few days ago notwithstanding, I'm feeling good about the reliability that gives me, leaving me mainly with the PosgresQL data store the main thing to keep backed up.

This is a hugely ambitious software and as such, there will be many things that I will have a hard time getting used to as a hobbyist, but also a lot to be gained. I'm sticking around for now and waiting for version 1, improved documentation, and more clarity on how it all works.

For example, it automatically handles Let's Encrypt certs for you. You get JMAP, CalDAV, WebDAV, CardDAV, IMAP4rev2, DKIM/SPF/DMARC, MTA-STS, DANE, spam filtering, SQL+blob+object storage backends, search, clustering, OpenTelemetry, etc all in one tiny binary.

Downsides: some features are gated behind an enterprise version and I think the dev team is one guy, or at least it was a while ago.

Having ran both for a long time, I'm sticking with Stalwart from now on as long as development continues.

If you want to push a new technology, you need to start somewhere. That's exactly what's happening with JMAP. It was created by Fastmail to use as a bridge between their servers and their own apps a case for which popularity doesn't matter. It's basically a modern vendor proprietary protocol but done in the open.

From there, support is only a matter of someone being interested enough to implement it and manifestly it's working. There are now three servers (Apache James, Cyrus and Stalwart) and some clients.

That's a really cruel response, because this is important work. I don't want my kids beholden to bigco.

I think it's real & important.

I also wanna make sure people like me, who have to keep tabs on the intersection of "how can I help liberate from BigCo" and "how can I make a livable wage doing so"

It is, quite literally, real, but also something you shouldn't waste time on if you're already busy. (c.f. https://jmap.io/software.html)

Never hosted Postfix / Dovecot stack, in fact this is the first time I host emails, but from what I understand Stalwart is designed to handle inbound directly.

For very high throughput inbound you could check out KumaMTA - it was designed specifically for that, but I think Stalwart doesn’t have bottlenecks in it’s clustered topologies which would require it unless you are doing something crazy.

They have very good docs in general IMO, here are docs on how to cluster - https://stalw.art/docs/cluster/configuration

Haven't looked into spam more closely yet. After first glance on most publicly shared email address - there is around 2 spam messages per hour.

Here is report prepared by llm which looked through the last 20 email headers found in spam. All of them were categorized correctly, however there were few emails in the past few days which went to spam where they shouldn't but I think this is fixable.

- Critical Authentication Failures: A large number of the messages failed basic email authentication. We see many instances of SPF_FAIL and VIOLATED_DIRECT_SPF, meaning the sending IP address was not authorized to send emails for that domain. This is a major red flag for spoofing.

- Poor Sender IP Reputation: Many senders were listed on well-known Real-time Blackhole Lists (RBLs). Rules like RBL_SPAMCOP, RBL_MAILSPIKE_VERYBAD, and RBL_VIRUSFREE_BOTNET indicate the sending IPs are known sources of spam or are part of botnets.

- Suspicious Content and Links: The spam filter identified content patterns statistically similar to known spam (BAYES_SPAM) and found links to malicious websites (ABUSE_SURBL, PHISHING).

- Fundamental Technical Misconfigurations: Many sending servers had no Reverse DNS (RDNS_NONE), a common trait of compromised machines used for spam.

There have been few messages which went to spam which didn't meet any of this spam criteria but actually they were cold marketing emails, so it's good too. In addition to this stalwart emits info log for each possible spam message ingested. Not sure if this can get any better than this.

You don't need major providers to support it, they support SMTP and that's how messages are relayed. JMAP is just so you: the client, can fetch your mail from wherever you host your mail.

This is not the case for all versions, but I've found it to be common enough that I have to read all of the release notes between point versions when upgrading.

It can definitely be improved.

In that case the overhead is just a small amount of kernel accounting.

JMAP is better than IMAP because IMAP is a too stateful design, the IMAP/SMTP distinction allows for misconfigurations where sending doesn't work, has dozens of extensions where key extensions are inconsistently supported, doesn't have as many batched operations, etc. One could make an effort to improve IMAP - but the effort to do this consistently in server software would likely be comparable to adding JMAP and the result worse...

OTOH, the new protocols intrude on areas that go far beyond email software (you're very unlikely to get support for these in older Androids/iOS/Windows even if the modern OSs ever consider them), and don't offer as much as JMAP offers over IMAP. The cost/benefit is worse. They may make sense for a JMAP email client but IMHO not elsewhere.

I looked into adding JMAP support to Thunderbird but the client is so tied around the ideas and principles of IMAP, it needs surgical refactoring of many parts of it and I don’t love C++.

So instead in my spare time I am developing a JMAP only gnome email client, using many Stalwart libraries. Think Geary but Rust instead of Vala, GTK4 instead of GTK3 and JMAP instead of IMAP. It’s been mostly an excuse to play with Rust and gtk-rs and Relm4 (beautiful Elm inspired rust bindings for GTK4). Someday, it will be released.

Client support for a new protocol is never that quick, but I believe adoption will happen, at least outside of the big providers, who will never support it.

The next, next big thing would be the Chatmail relays[1] supporting JMAP based servers (right now it's Dovecot) and this new targeted push extension for faster notifications without battery drain on mobile. I can see how the Fastmail mobile client will benefit from this RFC as well (it's already incredibly battery efficient, thanks to the team).

[1] https://github.com/chatmail/relay