Merge master into jsdt/ws-v2#4212
Merged
cloutiertyler merged 103 commits intojsdt/ws-v2from Feb 5, 2026
Merged
Conversation
…#3978) # Description of Changes Despecializes direct indices for `u64` keys that have too large values and convert them into a B-Tree index instead. Otherwise, we are exposed to OOM aborts. Also hardens typed index tests to cover more combinations. This is how the bug was caught. Some index code is also simplified. # API and ABI breaking changes None # Expected complexity level and risk 2? # Testing Proptests were generalized to catch the bug and to then verify that it was fixed.
#4024) # Description of Changes We've received a number of community pull requests which add `impl FilterableValue` for new column types, without implementing the features which `FilterableValue` is intended to describe. This PR extends the doc comments on `FilterableValue` to explicitly write what the requirements are to add a new implementation. # API and ABI breaking changes N/a # Expected complexity level and risk N/a # Testing N/a
This also has a few minor changes to fix build errors for the `test-app`. # Description of Changes Updates the typescript-test CI job to build some packages that weren't being built before. This also updates the root-level `pnpm generate/format/lint/build` commands to also apply to templates. # Expected complexity level and risk 1 --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
…d `Local` out of `SpacetiemDB.Runtime` (#3996) # Description of Changes This PR fixes a C# SDK regression where using `Bound` in index filters could trigger an ambiguous reference compiler error for Local after upgrading to `v1.11.2`, as reported in [#3995](#3995). It also fixes a related warning-spam regression (`CS0436`) where user projects could see `Local` type conflicts between generated module code and the `SpacetimeDB.Runtime` assembly. * Introduced a public `SpacetimeDB.Bound` type so users no longer need to import `SpacetimeDB.Internal` to use bounds in index filters. * Kept `SpacetimeDB.Internal.Bound` for compatibility, but added implicit conversions between `SpacetimeDB.Internal.Bound` and `SpacetimeDB.Bound`. * Updated the C# code generator to emit fully-qualified `global::SpacetimeDB.Bound` in generated index filter signatures, avoiding `SpacetimeDB.Internal` in public-facing APIs and preventing name collisions (e.g., `Local`). * Updated internal runtime bounds helpers (`BTreeIndexBounds<...>`) to explicitly use `SpacetimeDB.Bound` when constructing range-scan arguments. * Updated Codegen snapshot fixtures to match the new generated output (type name + formatting). * Fixed codegen output for `ITableView` `static abstract` member implementations to generate `public static` methods (required for the generated code to compile). It also fixes a related warning-spam regression (CS0436) where user projects could see Local type conflicts between generated module code and the SpacetimeDB.Runtime assembly. Additional fix (related to the `Local` reports): * Removed the runtime assembly’s ownership of `SpacetimeDB.Local` (introduced more recently than the generated module `Local`) to prevent `CS0436` duplicate-type warnings. Basically, the runtime’s concrete `Local`/`ProcedureTxContext` helpers were renamed and made internal so the code generator remains the sole owner of module-level `SpacetimeDB.Local`. Regression coverage: * Added generator regression assertions to ensure generated code does not reference `global::SpacetimeDB.Internal.Bound<...>` and does reference `global::SpacetimeDB.Bound<...>`. * Added a runtime API regression assertion that `SpacetimeDB.Bound` exists and is public in the runtime reference. * Added a regression assertion that `SpacetimeDB.Runtime` does not define codegen-owned types (e.g. `SpacetimeDB.Local`, `ProcedureContext`, etc.) to prevent future `CS0436` conflicts. * Added a “simulated downstream user file” compile check ensuring no `CS0436` diagnostics occur when user code references `SpacetimeDB.Local`. # API and ABI breaking changes None. * No schema or wire-format changes. * The changes are limited to C# type exposure / naming and codegen output. * `SpacetimeDB.Internal.Bound` remains usable via implicit conversions (backwards compatible for existing code). # Expected complexity level and risk 2 - Low * Changes are isolated to C# runtime type exposure, codegen type references, and snapshot updates. * No runtime behavior changes to index scan encoding/decoding; only avoids requiring SpacetimeDB.Internal in user code. # Testing - [X] Ran:`dotnet test crates/bindings-csharp/Codegen.Tests/Codegen.Tests.csproj` - [X] Ran regression tests locally.
# Description of Changes - Updated Unreal codegen to work with ::Result to match how ::Option works to create a new struct for each type pair identified - Added new integration tests to confirm Ok state for all new types added to sdk-tests - Added new UE_SPACETIMEDB_RESULT macro to capture de/serialization for the new structs # API and ABI breaking changes N/A # Expected complexity level and risk 2 - Added new type generated from Result which creates new structs for each one identified # Testing - [x] Updated the integration tests to include the new Result types --------- Co-authored-by: Mario Alejandro Montoya Cortes <mamcx@elmalabarista.com>
…ngle `class` (#3964) # Description of Changes This PR adds a regression test covering nullable reference-type view returns in C# modules (e.g. Account? where Account is a class), as reported in [#3962](#3962). ```csharp public static Account? MyAccount(ViewContext ctx) ``` * Updated the C# regression-test server module to include a reference-type table and views that exercise the RefOption path: * A new public reference-type table row: ```csharp [SpacetimeDB.Table(Name = "account", Public = true)] public partial class Account { ... } ``` * A public at-most-one view that returns a nullable reference type (Account?) via Find(...): ```csharp [SpacetimeDB.View(Name = "my_account", Public = true)] public static Account? MyAccount(ViewContext ctx) { return ctx.Db.account.Identity.Find(ctx.Sender) as Account; } ``` * A second public view that returns null to ensure the “empty result” case is exercised: ```csharp [SpacetimeDB.View(Name = "my_account_missing", Public = true)] public static Account? MyAccountMissing(ViewContext ctx) => null; ``` * Updated ClientConnected to ensure an Account row exists for the connecting identity so the “one row” case is deterministic. * Updated the C# regression-test client to: * Subscribe to the new views: * `SELECT * FROM my_account` * `SELECT * FROM my_account_missing` * Assert correct semantics for nullable reference-type view returns: * MyAccount.Count == 1 * MyAccountMissing.Count == 0 * Updated the regression-test server project to use local C# runtime/codegen project references so the regression module exercises the in-repo generator/runtime behavior (instead of the published SpacetimeDB.Runtime package). # API and ABI breaking changes None. * No changes to public module schema/wire format semantics beyond adding regression-test-only tables/views. * No behavior changes outside the C# regression test module + harness. # Expected complexity level and risk 2 - Low * Changes are isolated to regression tests and project wiring. * The scenario specifically guards the nullable reference-type “Option-like view return” path against regressions. # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> - [X] Ran C# regression tests with no failures in new View tests Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> - Version bump to `v1.11.3` for just the CLI and rust # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk 1 <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> - [x] CLI version has been updated - [x] Version + date in the license file has been updated
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> - Small fix for checking to see if a port is available on some given interface. updated: The original implementation here used `bind` to try to discover if a port is currently in use. This isn't reliable due to platform differences - especially on windows where it's apparently acceptable to have a service running on both `0.0.0.0:3000` and `127.0.0.1:3000`. This would cause bind to return successfully when we wanted it to fail. Also: binding on an ipv6 interface when a machine doesn't have ipv6 enabled caused random errors and it was too unreliable to be useful. This new implementation uses `get_socket_info` which returns info on all sockets in use on the system. We can then look through this list to find services which conflict with our requested port. updated 1/14: This PR now includes a fix for flaky CLI tests. Originally we were using `find_free_port` to pick a random free port, but that was causing a race condition which resulted in test flakes. This PR fixes this issue by using `127.0.0.1:0` as the listen addr so the kernel will automatically pick a free port for us. # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> 1 - this is a pretty isolated check, unlikely to introduce larger issues. # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> I tested on macos, windows and linux: ``` ALLOW docusaurus is already running on 127.0.0.1:3000 SpacetimeDB then tries to start on 192.168.1.10:3000 ALLOW docusaurus is already running on ::1:3000 SpacetimeDB then tries to start on 192.168.1.10:3000 DENY docusaurus is already running on ::1:3000 SpacetimeDB then tries to start on 127.0.0.1:3000 DENY docusaurus is already running on 0:0:0:0:0:0:0:0:3000 SpacetimeDB then tries to start on 0.0.0.0:3000 DENY docusaurus is already running on 0:0:0:0:0:0:0:0:3000 SpacetimeDB then tries to start on 127.0.0.1:3000 DENY docusaurus is already running on 0:0:0:0:0:0:0:0:3000 SpacetimeDB then tries to start on 192.168.1.10:3000 DENY docusaurus is already running on 127.0.0.1:3000 SpacetimeDB then tries to start on 0:0:0:0:0:0:0:0:3000 DENY docusaurus is already running on 192.168.1.10:3000 SpacetimeDB then tries to start on 0:0:0:0:0:0:0:0:3000 ```
Confirmed reads are arguably something users should be aware of. # Expected complexity level and risk 1 # Testing Documentation update. --------- Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes Anther knob for benchmarking only without job core pinning but keeping other core pinning (tokio background, rayon, etc.). # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing No semantic changes.
# Description of Changes Update the path of dirName to match the correct path. # API and ABI breaking changes None. # Expected complexity level and risk 0.0001 # Testing Tested locally by running Docusaurus
# Description of Changes Just adds public accessors for some existing internal functionality, to enable a change in another repo. Review starting from there. # API and ABI breaking changes These crates are marked unstable. # Expected complexity level and risk 0.5 # Testing See other PR. --------- Co-authored-by: Kim Altintop <kim@eagain.io>
# Description of Changes In `eval_updates_sequential`, if here are no queries, don't eval updates at all. Found via flamegraph. Makes `eval_updates_sequential` take 3.3% of `commit_and_broadcast_event` instead of 8%. # API and ABI breaking changes None # Expected complexity level and risk 1
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> Fix LLM benachmarks in rust since they used Result instead of ResultRow in the request to the LLM making it always fail. 1. see the answer's file there it is ResultRow. 2. Result is a keyword so it will always fail # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> # Expected complexity level and risk <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> 1 # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> Run /update-llm-benchmark to see if more passes. --------- Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Notifications were sent shortly before dropping the actor (and thus, the lockfile), which could cause the `close` future to return while the lock is still held. This can lead to a race if the database is re-opened immediately, such as in `TestDB::reopen()`, causing test flakes. # Expected complexity level and risk 1 # Testing Should result in "Database is already opened" test flakes to go away.
This is the second step to make in-memory-only databases not touch the disk at all. While at it, also make it so file-backed module logs are streamed in constant memory where possible. Depends-on: #3912 # Expected complexity level and risk 2 # Testing Added some unit-level tests. --------- Signed-off-by: Kim Altintop <kim@eagain.io> Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes Same as #3972. Merged the original PR into wrong branch instead of mater :) --------- Signed-off-by: Shubham Mishra <shivam828787@gmail.com> Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
# Description of Changes I believe that local users do not have API tokens for OpenAI, so the existing hint was not helpful. Apparently the correct path is to post `/update-llm-benchmark` on the PR and let the CI take care of it. # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing None --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes Major documentation overhaul focusing on tables, column types, and indexes. **Quickstart Guides:** - Updated React, TypeScript, Rust, and C# quickstarts with table/reducer examples - Fixed CLI syntax (positional `--database` argument) - Improved template consistency across languages **Tables Documentation:** - Added "Why Tables" section explaining table-oriented design philosophy (tables as fundamental unit, system tables, data-oriented design principles) - Added "Physical and Logical Independence" section explaining how subscription queries use the relational model independently of physical storage - Added brief sections linking to related pages (Visibility, Constraints, Schedule Tables) - Renamed "Scheduled Tables" to "Schedule Tables" throughout (tables store schedules; reducers are scheduled) **Column Types:** - Split into dedicated page with unified type reference table - Added "Representing Collections" section (Vec/Array vs table tradeoffs) - Added "Binary Data and Files" section for Vec<u8> storage patterns - Added "Type Performance" section (smaller types, fixed-size types, column ordering for alignment) - Added complete example struct demonstrating all type categories - Renamed "Structured" category to "Composite" **Indexes:** - Complete rewrite with textbook-style documentation - Added "When to Use Indexes" guidance - Documented single-column and multi-column index syntax (field-level and table-level) - Comprehensive range query examples with correct TypeScript `Range` class syntax - Explained multi-column index prefix matching semantics - Added index-accelerated deletion examples - Included index design guidelines **Styling:** - Added CSS for table border radius and row separators - Created Check component for green checkmarks in tables # API and ABI breaking changes None. Documentation only. # Expected complexity level and risk 1 - Documentation changes only, no code changes. # Testing - [ ] Verify docs build without errors - [ ] Review rendered pages for formatting issues - [ ] Confirm code examples are syntactically correct --------- Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com> Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> - This will eliminate a lot of the test flakes with the unity testsuite. A huge amount of the test flakes here were that we were trying to acquire more than 2 unity license seats at a time which is not allowed. This will have the largest impact when the runners are busy. The downside here: This will cause jobs to wait which is what we want right now. Jobs may wait to queue for several minutes while other jobs finish but I don't expect this to have a big impact on the total time a PR will spend in the merge queue. # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk 1 <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> - [x] Unity CI passes even if multiple jobs are running at the same time
# Description of Changes This is a try to fix the Update LLM benchmarks workflow # Expected complexity level and risk 1
# Description of Changes Add Auth0 tutorial <img width="1612" height="1350" alt="image" src="https://github.com/user-attachments/assets/077c156c-ec01-4fb5-aed2-ed728eac049c" /> <img width="1612" height="1350" alt="image" src="https://github.com/user-attachments/assets/075f42e5-6cb1-4766-bd68-169c6ed70be9" /> # API and ABI breaking changes None. # Expected complexity level and risk 1 # Testing - Followed rendered tutorial on the locally running website --------- Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes This pr reexports `PointIndex` and `PointIndexReadOnly` because currently the new `Hash` indices dont compile when used in projects. This is because the table macro cant find those two exports. Maybe @Centril could take a look? :> # API and ABI breaking changes None # Expected complexity level and risk 1. Only fixes a compiler error. # Testing - [x] Project compiles now. Before: <img width="1454" height="337" alt="image" src="https://github.com/user-attachments/assets/42df164c-dd09-42b9-97df-a1df719dc3b4" /> After: <img width="1404" height="172" alt="image" src="https://github.com/user-attachments/assets/0d11158c-daae-44b6-b001-ccccfe0940fd" />
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> in the llm tests we run to see if docs update break llms. we are making sql queries to tables that we did not set to public what means the sql query will fail # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> # Expected complexity level and risk 1 <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> Run /update-llm-benchmark and see if it fixes the sql query issues. Signed-off-by: Boegie19 <34578426+Boegie19@users.noreply.github.com>
# Description of Changes This moves the query builder code out of the `server` package and into `lib` so it can be shared by the client and server. I put the query builder in the `index.ts` of module bindings as a `query` object that can be imported. The typescript `test-app` has an example of using it with the subscription builder. This is branched off of `https://github.com/clockworklabs/SpacetimeDB/pull/3980`. # API and ABI breaking changes This extends the client subscription builder API to allow `string | RowTypedQuery<any, any>`, so existing client code should be fine. # Expected complexity level and risk 1.5. This is low risk. # Testing I manually tested that the test app still works locally.
# Description of Changes Add Clerk tutorial # API and ABI breaking changes None. # Expected complexity level and risk 1 # Testing Tested locally --------- Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes Dpends on and includes: #4011 Will remove commits whenever that merges 👍 This impacts the `useTable()` functions `where`clause. We add a new possibility for the `Value` type which is the new `Uuid` type. Also my formatter is bugging again :( whenever i `pnpm format` it does things differently than the guidelines? Would be cool for that reason if someone can run format for me maybe bfops like the last time :> # API and ABI breaking changes None # Expected complexity level and risk 1. Only lifts the filter for the where clause a bit to further support `Uuid` alongside bool, string, and number. # Testing - [x] Getting syntax highlithing for the new availbale row - [x] Getting row back from subscription when supplying a uuid - [x] Fix edge case with string --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
…smatches (#4059) # Description of Changes Adds TypeScript as a third language for LLM benchmark tests alongside Rust and C#, and fixes table naming convention mismatches. **TypeScript Support:** - Added `Lang::TypeScript` variant with camelCase naming conventions - Created TypeScript project template (`templates/typescript/server/`) with package.json, tsconfig.json, and index.ts - Added TypeScript publisher that uses `spacetime build` and `spacetime publish` - Created 22 TypeScript task prompts and golden answer files for all benchmark tests - Updated prompt discovery to find `tasks/typescript.txt` files **Table Naming Fix:** - Standardized on singular table names across all languages: - Rust: `user` (snake_case singular) - C#: `User` (PascalCase singular) - TypeScript: `user` (camelCase singular) - Updated `table_name()` helper to convert singular names to appropriate case per language - Updated all spec.rs files to use `table_name("user", lang)` instead of hardcoded `"users"` **CI/Hashing Improvements:** - Added `compute_processed_context_hash()` for language-specific hash computation after tab filtering - Updated CI check to verify both `rustdoc_json` and `docs` modes for Rust - Fixed `--hash-only` mode to skip golden builds # API and ABI breaking changes None - these are internal benchmark tooling changes only. # Expected complexity level and risk **Complexity: 2** The changes add a new language following existing patterns for Rust and C#. The table naming fixes are straightforward find-and-replace style updates. Low risk since this only affects the benchmark tooling, not the core SpacetimeDB codebase. # Testing - [x] `cargo build -p xtask-llm-benchmark` compiles successfully - [x] All 22 TypeScript golden modules build and publish successfully - [x] Rust and C# benchmarks unaffected by changes --------- Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Empty commit to fix llm benchmark. --------- Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes The implementation of a solution to #3044 , this adds an `Abort` function to the `WebSocket`, which runs if `Disconnect` is called when the `WebSocket` is not connected. # API and ABI breaking changes Not API breaking. # Expected complexity level and risk 1 # Testing - [X] Test locally with a C# CLI test client. **Note**: Before change (either on Rust of C# server), server would see 4 `Debug` log entries about connecting, but not see the `Info` log about the client connection ending like would normally be seen in a disconnect. After change, server shows no log entries at all, because connection is properly aborted. - [x] Test locally with a C# WebGL test client. --------- Signed-off-by: rekhoff <r.ekhoff@clockworklabs.io> Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes use own LogSpacetimeDb_Connection log cateogory to configure & control log level # API and ABI breaking changes - # Expected complexity level and risk 1 # Testing - Co-authored-by: Jason Larabie <jason@clockworklabs.io>
# Description of Changes Update ai rules general mdc file to explain deployment and maincloud # API and ABI breaking changes No # Expected complexity level and risk 1 # Testing Ran with claudecode
# Description of Changes Replaced the specific versions in `templates/basic-rs` with relative paths into the repo. This was split out of #4102. # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing - [x] Upgrade version check still passes --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes This has been popping up in more places, so I've added this in more places. # API and ABI breaking changes None. CI only. # Expected complexity level and risk 1 # Testing :shrug: --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes This change implements the module bindings changes to Views that where updated with core in #3685 and the Rust module bindings implementation in #3819. Updates the C# module bindings to use new header-first view ABI (`ViewResultHeader`) and updates the return codes for the `__call_view__` and `__call_view_anon__` module exports. This is a prerequsite for `Query` builder support being added to C# modules. # API and ABI breaking changes Not breaking. Existing modules will continue to use the old ABI. New modules will use the new ABI. However previous host versions will not support modules built using this version of the bindings. # Expected complexity level and risk 2 # Testing This is an internal refactor. All existing tests should continue to pass. All existing tests should continue to pass. The only tests that needed updating were the C# codegen snapshot tests (`Codegen.Tests`) because the generated view dispatcher bodies changed (they now prefix `ViewResultHeader.RowData` before the existing row payload).
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> This workflow change migrates us from docker container runners to the new system which uses virtual machines via qemu. This solves a lot of complexities around docker-in-docker and also allows us to easily run windows VMs in the future. # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk 0 - this only updates our workflow and has no impact on the code. <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> - [x] CI is passing --------- Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> Copied from: #4084 We originally reverted this because it was causing testsuite flakes in private. Now we have solved the issue that was causing the flakes so this should be safe to merge. Version upgrade to `v1.12.0`. # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk 1 - this is just a version upgrade <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> The testsuite failures are fixed by #4120 - [x] License has been properly updated including version number and date - [x] CI passes --------- Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes This got a little bigger than I had hoped, but I think it's still pretty manageable. This PR partially reverts back to before #3263: cores are re-balanced with a `watch::Receiver<CoreId>` that each database thread will listen for updates on in order to repin itself, and multiple OS threads (each matched to a database) can be pinned to one core. As I understand it, that second part is something Phoebe was trying to avoid, but given that there's no way to asyncify a JS module, it's kind of necessary. JS is single-threaded, and uses cooperative rather than preemptive multitasking (callbacks/async, not green threads). That means that if a JS function has an infinite loop, no other event handlers would be able to run unless that loop were to exit. Coupled with the fact that we can't `Send` a v8 isolate across threads, it makes more sense to keep the module host on one thread and repin that thread as needed. An alternative option, as was brought up, would be to deconstruct and reconstruct the module onto a different thread when needed, since load-balancing won't be happening often anyway. # Expected complexity level and risk 3 - reworks the threadpool that databases run on, and so could lead to deadlocks or other concurrency bugs. However, that seems unlikely, since this separates databases each onto their own thread, and as such decreases the likelihood of them interacting poorly with each other. # Testing Not sure if there's anything specific I should do, since this doesn't change behavior. - [ ] <!-- maybe a test you want a reviewer to do, so they can check it off when they're satisfied. --> --------- Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
# Description of Changes This patches a regression introduced in [#4033](#4033) where cargo ci dlls stopped copying the Unity .meta files that live outside the package skeleton tree: 1. `overlay_unity_meta_skeleton` now copies any `sdks/csharp/unity-meta-skeleton~/spacetimedb.<pkg>.meta` file into `sdks/csharp/packages/` before overlaying nested content. 2. Added support for a `version.meta` template inside each skeleton package; it’s renamed to match the single restored version directory (e.g. `1.11.2.meta`). 3. Added the missing `version.meta` templates for both `spacetimedb.bsatn.runtime` and `spacetimedb.runtime`, based on the historical GUIDs Unity already knows about. Together this restores the `spacetimedb.bsatn.runtime.meta` and `<version>.meta` files that Unity requires to keep those folders visible when developers run `cargo ci dlls` on a clean checkout. # API and ABI breaking changes None. This only affects the CI helper responsible for syncing Unity metadata. # Expected complexity level and risk 2 — localized changes to the CI helper and skeleton assets. Primary risk is forgetting a template or mis-copying a GUID; the code paths themselves are straightforward. # Testing - [X] Ran `cargo check -p ci` - [X] Ran `cargo ci dlls` on a clean tree, verifying that: * `sdks/csharp/packages/spacetimedb.bsatn.runtime.meta` exists * The restored version directory (e.g. `sdks/csharp/packages/spacetimedb.bsatn.runtime/1.11.2.meta`) exists - [X] Locally launched Unity with a SpacetimeDB project and had no errors/issues.
# Description of Changes This should optimize better than walking the AlgebraicType tree on every call. I'll do some profiling to confirm, but I am already using it in my performance experiments. # Expected complexity level and risk 2 - fully rewrites the bsatn encoder/decoder, but it's a fairly simple format. # Testing - [x] Automated testing - [x] Profiling
# Description of Changes Tiny addition to the llm context to explain that maincloud publishing is free of charge # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing Ran claude code with updated context and it performed better in the publishing step
# Description of Changes This particular line lost during rebasing of prev PRs. Now causing 2.0 branch to fail - #4115 # API and ABI breaking changes NA # Expected complexity level and risk 0
# Description of Changes This moves a bunch of stuff from `lib` back into `server` and `sdk`, and removes all but one global variable from the server sdk in preparation for export-based reducer definition. # Expected complexity level and risk 2 - a pretty big refactor, but it's mostly just code movement. # Testing - [x] Refactor, so automated tests are sufficient.
# Description of Changes The script was running `cd` but using the STDB path as passed, so if it was run from e.g. the root directory, it would generate incorrect paths. # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing - [x] Running `sdks/csharp/tools~/write-nuget-config.sh .` from the root directory generates a `NuGet.Config` that uses correct paths Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes Ignoring nuget config files regardless of casing, as well as in the root directory since we generate them there now. # API and ABI breaking changes None. # Expected complexity level and risk 1 # Testing - [x] Files generated by `write-nuget-config.sh` are now properly ignored --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Description of Changes Small rewording of the Zen document, fixed some typos. --------- Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
# Description of Changes New tagline Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com> Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes <!-- Please describe your change, mention any related tickets, and so on here. --> - This was blocking the release and is now blocking a CI check that @bfops added in private. # API and ABI breaking changes <!-- If this is an API or ABI breaking change, please apply the corresponding GitHub label. --> None # Expected complexity level and risk 1 <!-- How complicated do you think these changes are? Grade on a scale from 1 to 5, where 1 is a trivial change, and 5 is a deep-reaching and complex change. This complexity rating applies not only to the complexity apparent in the diff, but also to its interactions with existing and future code. If you answered more than a 2, explain what is complex about the PR, and what other components it interacts with in potentially concerning ways. --> # Testing <!-- Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected! --> - [x] CI is passing
# Description of Changes And more precisely pin the Cargo dep, along with a comment. # API and ABI breaking changes N/a # Expected complexity level and risk 1 # Testing - [x] `nix build` now exits 0 and produces runnable binaries locally. (`nix flake check` is still broken.)
# Description of Changes The goal of this PR is to: - Make it easy to switch our backing implementation of identifiers, as seen in the commit changing to `LeanString`. - Improve type safety of our identifiers by having every identifier start with `RawIdentifier` and then `Identifier` is just a wrapper with validation on construction. `TableName` and `ReducerName` are then just wrappers around `Identifier`. - Reduce allocation in `InstanceEnv` by using the now clone-on-write O(1) + SSO optimized `Identifier`. - Reduce allocations in the query engine as a consequence of improving `RawIdentifier` and `Identifier`. - Have `&'static str` strings be further optimized by avoiding to allocate even for long ones. This is supported by `impl From<&'static str> for RawIdentifier` as well as switching to `LeanString`. The PR results in a roughly 2k TPS improvement over master for WASM + hash indices. This should also help V8, as this is VM-agnostic. # API and ABI breaking changes None # Expected complexity level and risk 3? Mostly just small changes in many places, but in a sea of small changes, mistakes can happen. # Testing Covered by existing tests.
# Description of Changes
This PR translates all of our Python smoketests into Rust tests which
can be run from `cargo run`
## Motivation
The purpose of this fivefold:
1. All developers on the team are familiar with Rust
2. It simplifies our devops because we can drop Python as a dependency
to run the tests
3. You can now run all tests in the repo through the single `cargo test`
interface
4. Because we use the `SpacetimeDbGuard` and `cargo test`/`cargo
nextest` we can easily parallelize the smoke tests
5. The smoketests can now use machinery imported from SpacetimeDB crates
(e.g. `bsatn` etc.)
IMPORTANT NOTE!
There are several ways to implement the smoke tests in Rust (none are
great):
1. A separate xtask specifically for the smoke tests
- This doesn't solve the problem of the CLI tests which also use the
`guard` crate
- Idiosyncratic way to run the smoke tests as opposed to cargo test
- Does NOT resolve the cargo within cargo problem because we still have
to build the test modules with cargo
2. A `build.rs` script in `guard` which first builds the executables as
a compile step for compiling guard
- Deadlocks on a cargo lock file conflict (Outer cargo compiles guard →
runs build.rs, inner cargo tries to acquire the build directory lock,
outer cargo holds the directory lock, deadlock)
- If you fix the deadlock by using different target dirs, it still looks
stuck on building guard because it's actually compiling all of
spacetimedb-standalone and spacetimedb-cli.
- Still technically runs cargo inside of cargo.
3. Add `spacetimedb-cli` and `spacetimedb-standalone` as an artifact
dependency of the guard crate
- Has good and clear output but requires +nightly when running the
smoketests and CLI tests, otherwise won't do the right thing. See
rust-lang/cargo#9096
4. Compile the executables at runtime during the tests themselves where
the first test takes a lock while the executables are building using
cargo within cargo
- Makes the tests look like they're taking a long time when they're just
waiting for the build to complete
- Requires relatively complex locking machinery across
binaries/tests/processes
5. A two step solution where the developer has to build the binaries
before calling the smoke tests
- Very error prone
None of these are good. `xtask` is not bad, but doesn't enable us to run
other integration tests in other crates (e.g. the CLI)
(3) is the correct solution and has the best user experience, but it
requires nightly and I don't want to introduce that for all of our
tests.
I have chosen to do a combination of (1) and (4). You will now run the
smoketests with `cargo smoketest`. If you run `cargo test --all` (or use
`guard`) without doing `cargo smoketest` it will fall back to (4) which
compiles the executables at runtime. Running `cargo build` is the **only
way** to ensure that the executables are not stale because of the
internal fingerprint checking. Everything else is fragile not robust.
NOTE! There is no way to avoid cargo within cargo and have the smoke
tests be run as cargo tests because the modules under test must be
compiled with cargo.
# API and ABI breaking changes
Note that this is a BREAKING CHANGE to `cargo test --all`. The
smoketests are now part of `cargo test --all` unless you specifically
exclude them.
# Expected complexity level and risk
3, this is partially AI translated. We need to carefully review to
ensure the semantics have not regressed.
# Testing
<!-- Describe any testing you've done, and any testing you'd like your
reviewers to do,
so that you're confident that all the changes work as expected! -->
- [ ] <!-- maybe a test you want to do -->
- [ ] <!-- maybe a test you want a reviewer to do, so they can check it
off when they're satisfied. -->
---------
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
# Description of Changes Fixes #2298. The general idea is to avoid moving around types that are 400+ bytes or so only to eventually pass them by reference. Just pass them by ref as soon as possible instead. # API and ABI breaking changes None # Expected complexity level and risk 1 # Testing Covered by existing tests.
# Description of Changes - Codegen to skip creating reducer and procedure file if `visibility == FunctionVisiblity::Internal` - `UpdateHost` to check for visibility before excuting reducers and procedures. # API and ABI breaking changes NA, master is not exposing new raw module version yet. # Expected complexity level and risk 1. While the current patch is simple, could there be other places where visibility should also be applied? # Testing Smoketest can be in `2.0-breaking-changes` branch once we merge this. Exising test should cover for regression
# Description of Changes We were using `cargo install` but doing our hacky v8 dance for debug build outputs. `cargo install` runs in release mode though. This came up in #4205. # API and ABI breaking changes None. CI only # Expected complexity level and risk 1 # Testing :shrug: --------- Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
# Conflicts: # crates/core/src/client/message_handlers.rs # crates/core/src/client/messages.rs # crates/core/src/subscription/execution_unit.rs # crates/core/src/subscription/mod.rs # crates/core/src/subscription/module_subscription_actor.rs # crates/core/src/subscription/module_subscription_manager.rs # crates/core/src/subscription/query.rs # sdks/rust/tests/connect_disconnect_client/src/module_bindings/mod.rs # sdks/rust/tests/procedure-client/src/module_bindings/mod.rs # sdks/rust/tests/test-client/src/module_bindings/mod.rs # sdks/rust/tests/view-client/src/module_bindings/mod.rs
- Create message_handlers_v1.rs and message_handlers_v2.rs (WIP split from the branch that was missing the actual files) - Fix RowListLen import path (moved to websocket::common) - Fix FormatSwitch/BsatnFormat etc imports (moved to websocket::v1) - Add async to closures in call_view_add_v2_subscription - Add Ord derive to TableName (needed for BTreeMap key in v2 updates)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Merge in
master