Skip to content

feat(runtime): schedule object fields concurrently#44

Open
vito wants to merge 1 commit into
mainfrom
concurrency
Open

feat(runtime): schedule object fields concurrently#44
vito wants to merge 1 commit into
mainfrom
concurrency

Conversation

@vito
Copy link
Copy Markdown
Owner

@vito vito commented Apr 27, 2026

I kicked around a few ideas for concurrency, and I think this one strikes the sweet spot for Dang's focus.

We really just need a way to schedule work in parallel and group the results and/or fail-fast when any of them fail. In Go you might model this with goroutines, wait groups, context cancellation, and storing results in a map using locks to synchronize writes (or sync.Map but you get the point). But that's all low-level plumbing: tons of boilerplate to write, and tons of independent language features that would dramatically increase Dang's scope.

This PR instead enhances a single pre-existing feature: object literals - quick reminder example of those:

dang> {{foo: {{bar: 1}}, fizz: foo.bar}}
=> module {fizz: Int!, foo: {bar: Int!}!}
dang> toJSON({{foo: {{bar: 1}}, fizz: foo.bar}})
=> {"fizz":1,"foo":{"bar":1}}

Currently object literals instantiate a new scope and evaluate + assigns slots within it, serially.

With this PR, they are instead evaluated in DAG order. Independent slots are evaluted in parallel, and inter-dependent slots are evaluated in dependency order.

This seems pretty elegant to me, because it means {{}} literals that make API queries are now just as parallel as GraphQL multi-field selection syntax - i.e. {{bar: foo.bar, baz: foo.baz}} is just as parallel as foo.{bar, baz} (except it sends two requests instead of one).

Take this snippet, from one of Dang's tests:

pub complex_query = {{
  server: serverInfo.{version, platform},
  users: users.{name, emails},
  posts: posts.{title, author.{name}},
  titles: postTitles
}}

Previously this would send 3 queries one after another. Now it sends them all at once.

With the DAG evaluation order, this also means you can hypothetically define entire pipelines as single objects, and let Dang figure out how to evaluate everything optimally:

{{
  images: {{
    linux: build("linux")
    windows: build("windows")
    darwin: build("darwin")
  }}
  test: test
  integration: integration(images.linux)
}}
# evaluates:
# 1. build("linux"), build("windows"), build("darwin"), test
# 2. integration(images.linux)

Failing fast (or not)

Evaluation fails as soon as anything fails, i.e. it fails fast. I think this is the best default, since the alternative would mean returning a mixed report of successful results and failures, and seems like it could be represented by composing with something like try to have fields typed as something like Either foo Error.

Evaluate object literals as dependency graphs so independent fields can
run in parallel while dependent fields wait for their prerequisites.
Detect cyclic field dependencies during inference and keep publication
deterministic.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant