Semantic Query Engines with Matthew Russo (MIT)

AI is transforming Database Systems. Perhaps the biggest impact so far has been natural language to query language translations, or Text-to-SQL. However, another massive innovation is brewing.

I am SUPER EXCITED to publish the 131st episode of the Weaviate Podcast with Matthew Russo, a Ph.D. student at MIT!

AI presents new Semantic Operators for our query languages. For example, we are all familiar with the WHERE filter. Now we have AI_WHERE, in which an LLM or another AI model computes the filter value without needing it to be already available in the database!

```sql SELECT * FROM podcasts AI_WHERE “Text-to-SQL” in topics ```

Semantic Filters are just the tip of iceberg, the roster of Semantic Operators further includes Semantic Joins, Map, Rank, Classify, Groupby, and Aggregation!

And it doesn’t stop there! One of the core ideas for Relational Algebra and how its influenced Database Systems is query planning and finding the optimal order to apply filters. For example, let’s say you have two filters, the car is red and the car is a BMW. Now let’s say the dataset only contains 100 BMW, but 50,000 red cars!! Applying the BMW filter first will limit the size of the set for the next filter!

This foundational idea has all sorts of extensions now that LLMs are involved! This opportunity is giving rise to new query engines and declarative optimizers such as Palimpzest, LOTUS, and others!

So many interesting nuggets in this podcast, loved discussing these things with Matthew, and I hope you find it interesting!

YouTube: https://youtu.be/koPBr9W4qU0

Spotify: https://spotifycreators-web.app.link/e/ddUhVMmLoYb

Medium: https://medium.com/@connorshorten300/semantic-query-engines-with-matthew-russo-weaviate-podcast-131-131a42bbc521