Vaultgemma: the Most Capable Differentially Private LLM

training_algorithm(training data with a row that has "ForHackernews blood test report...") hard to distinguish from training_algorithm(training data without that) upto a factor of epsilon. They have explained further in the article itself with concrete values for epsilon.

prob(training_process(data)(Work account 2 has a serious medical condition called) = anaemia) <= e^epsilon * prob(training_process(data without that piece of information)(Work account 2 has a serious medical condition called) = anaemia)) + delta

Here epsilon = 2, and delta is small. Basically, there is a theoretical guarantee that if it had trained on that sentence, it would be no more than 7x as likely to output that in response to any prompt, compared to when it hadn't trained on that sentence at all. Sentence here is defined to be 1024 tokens long[1].

You might think 7x is not that big of a deal, but note that this is a theoretical guarantee( and with some mathematics it's possible to get an even tighter bound(see: Renyi DP)). In practice, actually getting private data out of a DP-trained model is difficult even for epsilon=8 (corresponds to 2000x likely!).

Edit: [1] this can be problematic, if a piece of information greater than 1024 tokens long gets split into two sentences, then there is no theoretical guarantee across sequences. However this is an implementation detail of this model, I've yet to see the effect of increasing this number to a more reasonable value.

The whole framing of DP is:

Probability that you reveal private info is same whether or not you train on a particular users data.

It is useful in many cases, but google the product company specifically is going to use it for ads.