Initial design for a serving layer

[antiguru]

Thought experiment for a serving layer in Materialize.

Rendered: https://github.com/antiguru/materialize/blob/serving_layer_design/doc/developer/design/20260210_serving_layer.md

[djahandarie]

Just saw this PR scroll by and wanted to share that we are very excited about this direction! I know it's a draft, but left some random thoughts in case they are useful. (Feel free to ignore them, you don't need to give detailed responses unless it's helpful to your thought process somehow.)

[djahandarie]

We would be very interested in btree-like access patterns; for example, in a huge amount of our use cases, we have some sort of paginated display of index contents, where we want to SELECT ... WHERE X < ... ORDER BY X LIMIT 100 or similar. Maybe it's not so easy to incrementally maintain a data structure which allows for something like this and that's why you made this choice, but just sharing the thought.

[djahandarie]

Is the idea that this would ~fully replace any clusters that currently function as "serving" clusters (like recommended in the docs)? If so this seems great to me, as it keeps the flexibility of the user being able to define their own compute isolation in whatever way is relevant to them, and now it's just more scalable/faster due to the explicit specialization.

[djahandarie]

I suppose I'm a bit confused why something special like this would need to be done. Is this to improve the ergonomics of being able to auto-optimize the performance of any arbitrary prepared statement that might come in (compared to users needing to individually define indexes for their access patterns)? I guess I'd be a little afraid of a new query suddenly causing a big rearrangement which starts getting permanently maintained, and manual definition of indexes via dbt or similar feels safer, but maybe I don't see the vision yet or don't fully understand it.

I think the strawman design would be to just extend CREATE INDEX to have USING newdatastructure, and maybe also support something closer to INCLUDE (col1, col2) in Postgres to be able to control what is actually in the index (as opposed to the current pattern of always including everything and requiring users to make views in order to achieve this) to improve ergonomics. Then a prepared statement would simply be responsible for finding a matching index, and from there it could be served very fast, right?

[djahandarie]

(One vote for as close to a direct connection as possible 😄 We really want to isolate serving between tenants as much as possible, and don't want any cross-tenant DoS situations even if one makes a couple million connections or something crazy. )

[aljoscha]

In that design, is envd still "terminating" network connections or clients go directly to a serving-layer process? I can see either way working, but if clients go directly to a serving-layer process we need a way of syncing catalog changes over to there, and they need to run a check to make sure they're up to date to be strictly serializable.

You'd need roughly https://github.com/MaterializeInc/database-issues/issues/9738, which builds on two other issues.