Skip to content

Conversation

@wjmelements
Copy link
Contributor

@wjmelements wjmelements commented Oct 30, 2025

Supercedes #325
Toward #312
Reviewers @rvagg @hugomrdias

As this is not yet the blessed path, I don't update the examples from createContext to createContexts.

Changes

  • CreateContextsOptions
  • createWithSelectedProvider
  • createContexts
  • mockServiceProviderRegistry helper
  • tests

@github-project-automation github-project-automation bot moved this to 📌 Triage in FOC Oct 30, 2025
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Oct 30, 2025

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
synapse-dev 877f4ed Commit Preview URL

Branch Preview URL
Nov 04 2025, 12:32 AM

@wjmelements wjmelements added the enhancement New feature or request label Oct 30, 2025
@wjmelements wjmelements mentioned this pull request Oct 30, 2025
@wjmelements wjmelements linked an issue Oct 30, 2025 that may be closed by this pull request
@wjmelements wjmelements changed the title [WIP] createContexts createContexts Oct 30, 2025
@wjmelements wjmelements marked this pull request as ready for review October 30, 2025 20:35
@wjmelements wjmelements mentioned this pull request Oct 30, 2025
@rjan90 rjan90 moved this from 📌 Triage to 🔎 Awaiting review in FOC Nov 3, 2025
@rvagg rvagg changed the title createContexts feat: createContexts Nov 3, 2025
Copy link
Member

@hugomrdias hugomrdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can this be more simple like:

user knows what he wants

type ProviderDatasetPair {
  providerId: bigint
  datasetId: bigint | undefined // create new dataset in this provider
}

createContexts(pairs: ProviderDatasetPair[], options: createDatasetOptions)

smart selection

type Options {
  excludeProviderIds?: number []
  withCDN?: boolean
  withIpni?: boolean
  dev?: boolean
  forceCreateDataSet?: boolean
  uploadBatchSize?: number
  metadata?: Record<string, string>
}

createSmartContexts(options: Options) // its auto and smart

Comment on lines 196 to 197
for (const dataSetId of new Set(options.dataSetIds)) {
const resolution = await StorageContext.resolveByDataSetId(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just force options?.dataSetIds.length to equal options.count ?

Comment on lines 196 to 197
for (const dataSetId of new Set(options.dataSetIds)) {
const resolution = await StorageContext.resolveByDataSetId(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

humm because its selecting providers...

@github-project-automation github-project-automation bot moved this from 🔎 Awaiting review to ⌨️ In Progress in FOC Nov 3, 2025
@wjmelements
Copy link
Contributor Author

datasetId: bigint | undefined // create new dataset in this provider

This isn't our behavior for createContext. createContext will prefer to use existing data sets when they match the criteria, even if providerId or providerAddress is provided. Smart selection also does that, but for all providers.

@hugomrdias
Copy link
Member

datasetId: bigint | undefined // create new dataset in this provider

This isn't our behavior for createContext. createContext will prefer to use existing data sets when they match the criteria, even if providerId or providerAddress is provided. Smart selection also does that, but for all providers.

sure we can the same and not force the create dataset if its undefined

@wjmelements
Copy link
Contributor Author

type ProviderDatasetPair {
  providerId: bigint
  datasetId: bigint | undefined // create new dataset in this provider
}

I've thought about this a bit, and while the priority is more implicit, I think the interface is slightly better without this intermediate ProviderDatasetPair. Compare:

createContexts({
    dataSetIds,
})
createContexts({
    providerIds,
})
// vs ProviderDatasetPair
createContexts({
    selections: dataSetIds.map((dataSetId) => { dataSetId }),
})
createContexts({
    selections: providerIds.map((providerId) => { providerId }),
})

I would also be interested in @rvagg's opinion on this. It's not too important to me but after we make this interface public it will be painful to change it.

createSmartContexts

I don't think we need multiple methods since both will fallback to smart selection. It's sufficient for me that the specification parameters are optional.

@wjmelements wjmelements requested a review from rvagg November 3, 2025 22:05
@rvagg
Copy link
Collaborator

rvagg commented Nov 4, 2025

Sorry, I'm not a fan of ProviderDatasetPair - mainly because I think they are joining two separate user behaviours:

  1. I want to make multiple contexts from my list of pre-selected providers - I may or may not have data sets for these, you figure it out
  2. I have existing data sets and I just want contexts for those - don't make me remember the providers, data set ID is globally unique, that's all I should need to keep, you just need to figure it out, don't need to create them, just set it up for me

That second path I think is the most interesting one and the one we want to make sure works - perhaps data set IDs get persisted in a cookie for a dapp user, every time they come back they always just get those same data sets that were created for them the first time they used it.

Every other bit of optionality is advanced use I think.

@wjmelements
Copy link
Contributor Author

Unable to merge
Screenshot 2025-11-03 at 9 09 21 PM

@rvagg
Copy link
Collaborator

rvagg commented Nov 4, 2025

I'm fine with override-and-fix since we have release-please standing in the way of releases. If @hugomrdias would like to propose an alternative diff then let's get that in before we cut a release. Sorry Hugo, let's keep moving and fix stuff later if we discover the API isn't all that great.

@rvagg rvagg merged commit 6fe64da into master Nov 4, 2025
12 checks passed
@rvagg rvagg deleted the wjmelements/create-contexts branch November 4, 2025 03:14
@github-project-automation github-project-automation bot moved this from ⌨️ In Progress to 🎉 Done in FOC Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: 🎉 Done

Development

Successfully merging this pull request may close these issues.

Default Replication

5 participants