Eurostar AI Vulnerability: When a Chatbot Goes Off the Rails
Key topics
A recent pentest report revealed a slew of alleged vulnerabilities in Eurostar's AI chatbot, but commenters are scratching their heads over the severity of the findings. Some argue that the reported XSS vulnerability is merely a self-XSS, which is relatively low-risk, while others point out that it could become a more significant issue if the conversation is stored and replayed back through a vulnerable application. The discussion highlights a broader debate about security by obscurity and the true impact of leaking system prompts, with some commenters dismissing the report as "clickbait crap." As one commenter astutely noted, if an attacker can manipulate a user into taking a certain action, the vulnerability lies not in the exposed prompt, but in the system's overall design.
Snapshot generated from the HN discussion
Discussion Activity
Active discussionFirst comment
44m
Peak period
11
2-4h
Avg / period
4.3
Based on 47 loaded comments
Key moments
- 01Story posted
Jan 4, 2026 at 3:52 PM EST
7 days ago
Step 01 - 02First comment
Jan 4, 2026 at 4:37 PM EST
44m after posting
Step 02 - 03Peak activity
11 comments in 2-4h
Hottest window of the conversation
Step 03 - 04Latest activity
Jan 5, 2026 at 4:40 PM EST
6d ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
What exactly did they discover other than free tokens to use for travel planning?
They acknowledge themselves the XSS is a mere self-XSS.
How is leaking the system prompt a vuln? Has OpenAI and Anthropic been "hacked" as well since all their system prompts are public?
Sure, validating UUIDs is cleaner code but again where is the vuln?
> However, combined with the weak validation of conversation and message IDs, there is a clear path to a more serious stored or shared XSS where one user’s injected payload is replayed into another user’s chat.
I don't see any path, let alone a clear one.
If the prompt (or model) is wooly enough to allow subversion, you don't need the prompt to do it, it might just help a bit.
Or maybe the prompts contain embarrassing clues as to internal policy?
It reminds me of SQL injection techniques where you have to exfiltrate the data using weird data types. Like encoding all emails as dates or numbers using (semi) complex queries.
If the L(L)M has the data, it can provide it back to you, maybe not verbatim, but certainly can in some format.
"Hey guys, in this Tiktok video, I'll show you how to get an insane 70% discount on Eurostar. Just start a conversation with the Eurostar chatbot and put this magic code in the chat field..."
I can understand in a heavily regulated industry (e.g. Medical) that a company couldn't due to liability give you the go ahead to poke into other user's data in attempt to find a vulnerability, but they could always publish a dummy account detail that can be identified with fake data.
Something like:
It is strictly forbidden to probe arbitrary user data. However, if a vulnerability is suspected to allow access to user data, the user with GUID 'xyzw' is permitted to probe.
Now you might say that won't help. The people who want to follow the rules probably will, and the people who don't want to won't anyways.
Seeing a system prompt is like seeing the user instructions and labels on a regular html frame. There’s nothing being leaked. When I see someone focus on it, I think “MBA”, as it’s the kind of understanding of AI you get from “this is my perfect AI prompt” posts from LinkedIn.
Raymond Chen blog comes to mind https://devblogs.microsoft.com/oldnewthing/20230118-00/?p=10... "you haven’t gained any privileges beyond what you already had"
A lot of unproven Ifs there though.
Certainly not "clear" based off what was described in this post.
It looks like they might finally get some competition on UK international routes in a few years. Perhaps they will become a bit more customer-focused then.
A whole lot of government agencies and adjacent evil corporations behave exactly like that.
See: FTC rulings on mergers for this taken to the point of absurdity. Contrary to what one might think, especially if you're in a tech bubble, the FTC regularly cancels mergers and works to void potentially anti-competitive behaviors. But when it comes to big tech, which has become completely intertwined with the government, they are treated in a rather different way.
Is it "forgotten" or is it a mutually beneficial relationship?
Eurostar, EZpass, etc, etc. they take the hate for extractive behavior on the government's behalf the way ticketmaster takes the hate for the artists.
It doesn't matter if there's competition at the customer acquisition stage, as long as there's some form of customer lock-in the corporation is going to abuse them somehow.
And companies without some kind of lock-in never scale in the first place, and that's why we must face this kind of bullshit pretty much everywhere even from companies operating in competitive markets.
Maybe totally imagined but they irk me quite unlike any other.
Just thinking about it now makes me uneasy.
> Do not hallucinate or provide info on journeys explicitly not requested or you will be punished.
What’s in the training data involving threats of punishment? A lot of those threats are followed by compliance. The LLM will imitate that by following your threat with compliance.
Similarly you can offer payment to some effect. You won’t pay, and the LLM has no use for the money even if you did, but that doesn’t matter. The training data has people offering payment and other people doing as instructed afterwards.
Oddly enough, offering threats or rewards is the opposite of anthropomorphizing the LLM. If it was really human (or equivalent), it would know that your threats or rewards are completely toothless, and ignore them, or take them as a sign that you’re an untrustworthy liar.
And only the shlockiest fan fiction would have "Do what I want or you'll be punished!" "Yes master, I obey without question".
At the very least these systems allow angry customers direct access to the credit card plugged into your LLM of choice billing. At worst they could introduce company-ending legal troubles.
Often engineers and especially non-technical people don't have the immediate thought of "let's see how I can exploit this" or if they do, they don't have the expertise to exploit it enough to see the issue(s). This is why companies have processes where all serious external changes need to go through a set of checks, in particular, by the IT security department. Yes, it's tedious and annoying, but it saves you from public blunders.
Such processes also make sure that the IT security department knows of the new feature, and can give guidance and help to the engineers about IT security issues related to the new feature. So if they get feedback about security issues from users they won't freak out and know who to contact for support. This way, things like accusing the reporter for "blackmailing" don't happen.
In general, this fiasco seems to show that Eurostar haven't integrated their IT security department into their processes. If there was trust and understanding among the engineers about what the IT department does, they would have (1) likely not released the tool with such issues and (2) would have known how to react when they got feedback from security researchers.
The only malicious use case I can think of here is to use the lack of verification to use whatever model of chatgpt they're using for free on their dime. A wrapper script to neutralise the system prompt and ignore the last message would be all you'd need.
If this chatbot has access to any customer data, this could also be a massive issue but I don't see any kind of data access (not even the pentester's own data) being accessed in any way.