Opentslm: Language Models That Understand Time Series

There are papers on that, such as https://arxiv.org/abs/2410.15319. Time series modeling will not bring about an understanding of causality except in a weak sense https://en.wikipedia.org/wiki/Granger_causality. To truly connect a cause and effect you need a graphical model. And automated causal discovery, the hardest part of which is proposing the nodes of the graph, is a nascent field.

I would if the image analysis library was backed by a VLM. I have not fully read the paper, but couldn't figure 6 have been done by an LLM writing a script that calls libraries for time series feature extraction and writing a hypothesis test or whatever? They will do the heavy lifting and return a likelihood ratio or some statistic that is interpretable to an LLM.

Can you explain how? If I'm understanding the paper right, the timeseries encoding is a Conv1D and the cross-attention layer is constrained to output the token space of a pre-trained LLM. My naive expectation is these constraints would make the model less expressive / fine-tunable to pick up on these types of subtle signals.

But obviously ML is an empirical field, so if you found that a constrained architecture worked well in practice, that's an interesting result in its own right.

[flagged]

It works.

Check it out, they are completely based on Llama and Gemma, outputting text. Models are open-source.

KBD+ is no secret, but what does this have to do with anything? It's just a database optimized for time series data and has nothing to do with AI. It's widely used in the financial business and even for non-financial things like Formula-1 race analysis.

The emergent behavior of LLMs being amazing at accurately predicting tokens in previously unseen conditions might be more powerful than more rigorous machine learning extrapolations.

Especially when you throw noisy subjective context at it.

Predicting the future is valuable. If a model can apply the same underlying world model that it uses to accurately predict OLHC series as it does to produce English language, then you can interrogate and expand on that underlying world model in complex and very useful ways. Being able to prompt it can describe a scenario, or uncover hidden influences that wouldn't be apparent from a simple accurate prediction. Things like that allow sophistication in the tools - instead of an accurate chart with all sorts of complex indicators, you can get English explication and variations on scenarios.

You can't tell a numbers only model "ok, with this data, but now you know all the tomatoes in the world have gone rotten and the market doesn't know it yet, what's the best move?" You can use an LLM model like that, however, and with RL, which allows you to branch and layer strategies dependent on dynamic conditions and private data, for arbitrary outcomes. Deploy such a model at scale and run tens of thousands of simulations, iterating through different scenarios, and you can start to apply confidence metrics and complex multiple-degree-of-separation strategies to exploit arbitrage opportunities.

Any one of the big labs could do something like this, including modeling people, demographic samples, distributions of psychological profiles, cultural and current events, and they'd have a manipulation engine to tell them exactly who, when, and where to invest, candidates to support, messages to push and publish.

The fundamental measures of intelligence are how far into the future a system can predict across which domains. The broader the domains and farther into the future, the more intelligence, and things like this push the boundaries.

We should probably get around to doing a digital bill of rights, but I suspect it's too late already anyway, and we're full steam ahead to snow crash territory.

hypothetically LLM absorbed lots of world knowledge, and it can trace lots of deep correlations between various factors.

This makes intuitive sense to me, because the system you are modeling is wide open and you’re competing against others who have the same information. Achieving much more than 51% accuracy would be extraordinary. But if you get 51% consistently over time, with leverage, you can make a good amount of money.

My experience as well; seemed more accurate while prices were rising.

Yep, fully open-source!

You guys are so funny, when papers like these exist: https://arxiv.org/abs/2404.11757

Numerous research, INCLUDING the OpenTSLM paper has PROVEN they are NOT able to do this out of the box. Did you even check out the results at all? They literally compare OpenTSLM against standard text only baselines. Gemma3-270M performs better than GPT-4o using tokenized time series alone. Thus, I guess you guys are being ironic.

Unfortunately not really, but we've found (and used in production for a year) that Claude 3.5 is perfectly capable of identifying anomalies or other points of interests in very large sets of time series data.

Think of 100-200K worth of tokens formatted like this:

<Entity1>-<Entity2> <Dimension> <ISO 8601 time> <value>

<Entity1>-<Entity2> <Dimension> <ISO 8601 time +1> <value>

<Entity1>-<Entity2> <Dimension> <ISO 8601 time +2> <value>

<Entity1>-<Entity2> <Dimension2> <ISO 8601 time> <value>

<Entity1>-<Entity2> <Dimension2> <ISO 8601 time +1> <value>

The only pre-filtering we do is eliminate "obviously non relevant" data, such as series where the value is completely flat the whole time, but this was done to add more data to the context, not because Claude struggled with it (it doesn't).

Such a blatant bot? Makes you wonder what’s lurking here.

> Limitations: ... Finally, while we report strong results on individual datasets, we have not yet demonstrated generalization to unseen data, an essential step toward general TSLMs.

From the paper itself...

Imagine you asked ChatGPT a question but it could only give you answers from a single blog.

> advanced pattern detection... detect new class of complex patterns

This sounds great and all, but it's wishful thinking. There isn't anything in this supporting that it's able to find any meaningful patterns beyond existing solutions (i.e. standard rules based detection/machine learning as mentioned above).

What they've essentially done is taken a dataset in which each report was "annotated with a report string (generated by cardiologist or automatic interpretation by ECG-device)" [1] and used it with a series of templates (i.e. questions to ask the llm) from the ECG-QA paper [2] to fine-tune a model to achieve 65% accuracy with solely pattern recognition and 85% accuracy with pattern+clinical context (i.e. patient history).

The 42 template questions they used (as mentioned in 4.1 in the paper) can each be evaluated deterministically via code and retrieved via a tool call for any llm to parse. And I argue that the results would be the same, if not better, for a fraction of the cost. Doing calculations like this on time series data is very very quick. A couple ms at most. I don't see why this couldn't be run on the edge.

Plus, Table 9 shows this thing takes a minimum of 7GB of ram usage with a 270m parameter model and ~15-20GB for a 1B model. I don't see how this could be run on the edge considering most phones have 6-8GB of ram.

[1]: https://physionet.org/content/ptb-xl/1.0.3/ [2]: https://arxiv.org/pdf/2306.15681

I had stopped using the sonnet model for anything important after some very big goof ups. 4.5 is definitely significantly better.

I've been totally blown away by opus except on a project I'm working on I discovered a few unexpected weaknesses that have cost quite a bit of time.