Finding Thousands of Exposed Ollama Instances Using Shodan

If you deploy a power plug outside your house, is it the fault of the power plug designer if people steal your power?

Put it behind a webserver with basic auth or whatever you fancy, done.

Seriously, this is extremely mild as far as issues go. There's basically no incentive to fix this problem, because I bet even the people who lost a few pennies of electricity would still prefer the convenience of ollama not having auth.

Plus, that's the worst case scenario, in real life even if some black hat found an exposed ollama service, they have no interest in generating tokens for <insert random LLM here at 4 bit quant> at a slow speed of <50tok/sec.

That's not security by obscurity.

If the "uuid looking" part is generated using a csprng and has enough entropy, it has the same security properties as any other secret.

There's other issues with having the secret in the URL.

Fortunately it’s an easy fix. Just front it with nginx or caddy and expect a bearer token (that would be your api key)

Problem is people don’t know that it’s a good solution.

The default install only binds to loopback, so I am sure it is pretty common to just slap OLLAMA_HOST=0.0.0.0 and move on to other things. I know I did at first, but my host isn't publicly routable and I went back the same night and added IPAddressDeny/Allow rules (among other standard/easy hardening).

What's likely happening here is that people are renting VMs and one-line some docker-compose up thing from a tutorial. And because it's a tutorial and people can't be bothered to tunnel their own traffic, most likely those tutorials are binding on 0.0.0.0.

Plenty of ways to footgun yourself with c/p something from a tutorial, even if you somewhat know what you're doing. No need to bring "everyone thinking they can code" into this. This is a tale as old as the Internet.

Another thing is that docker, being the helpful little thing that it is, in its default config will alter your firewall and open up ports even if you have a rule to drop everything you're not specifically using. So, yeah. That's probably what's happening.

The new problem is if the LLMs are connected to tooling.

There's been plenty of examples showing that with subtle changes to the prompt you can jailbreak the LLM to execute tooling in wildly different ways from what was intended.

They're trying to paper over this by having the LLM call regular code just so they can sure all steps of the workflow are actually executed reliably every time.

Even the same prompt can give different results depending on the temperate used. How security teams are able to sign these things off is beyond me.

So at least 1,100.