feat: show remaining CDN & cache-miss egress #538

bajtos · 2026-01-07T11:12:47Z

Update the demo app to show remaining egress allowances for each data set where the FilBeam service is enabled.

Screenshot showing the new version in practice:

version 2 using FolderSync icon

version 1 using RefreshCw icon

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

cloudflare-workers-and-pages · 2026-01-07T11:14:09Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	synapse-dev	`5905842`	Commit Preview URL Branch Preview URL	Jan 20 2026, 01:06 PM

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx

packages/synapse-react/src/warm-storage/use-data-sets.ts

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

packages/synapse-react/src/warm-storage/use-data-sets.ts

rvagg · 2026-01-08T08:04:27Z

This seems fine to me and I appreciate the AGENTS.md edits but I'm going to leave it to @hugomrdias to review

apps/synapse-playground/src/lib/utils.ts

- Change format to "Egress remaining: X GiB delivery · Y GiB cache-miss" - Remove FolderSync icon, keep Globe icon with tooltip "This data set is using CDN" - Use binary units (KiB, MiB, GiB, TiB) instead of decimal units Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos · 2026-01-13T10:13:12Z

I addressed the review comments, this PR is ready for another round of reviews. 🙏🏻

Add unit tests for getStatsBaseUrl and validateStatsResponse functions. Export validateStatsResponse to enable direct testing without HTTP mocking. Also update AGENTS.md with testing guidelines: - Use assert.deepStrictEqual for object comparisons - Use parameterized tests with descriptive names Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

just in case Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos · 2026-01-14T12:18:23Z

@hugomrdias I addressed your comments, PTAL again.

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx

apps/synapse-playground/src/lib/utils.ts

packages/synapse-react/src/warm-storage/use-egress-quota.ts

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx

hugomrdias · 2026-01-19T15:56:32Z

packages/synapse-core/src/filbeam/stats.ts

+function parseBigInt(value: string, fieldName: string): bigint {
+  // Check if the string is a valid integer format (optional minus sign followed by digits)
+  if (!/^-?\d+$/.test(value)) {
+    throw new GetDataSetStatsError('Invalid response format', `${fieldName} is not a valid integer: "${value}"`)
+  }
+  return BigInt(value)
+}
+
+/**
+ * Validates the response from FilBeam stats API and returns DataSetStats
+ */
+export function validateStatsResponse(data: unknown): DataSetStats {
+  if (typeof data !== 'object' || data === null) {
+    throw new GetDataSetStatsError('Invalid response format', 'Response is not an object')
+  }
+
+  const response = data as Record<string, unknown>
+
+  if (typeof response.cdnEgressQuota !== 'string') {
+    throw new GetDataSetStatsError('Invalid response format', 'cdnEgressQuota must be a string')
+  }
+
+  if (typeof response.cacheMissEgressQuota !== 'string') {
+    throw new GetDataSetStatsError('Invalid response format', 'cacheMissEgressQuota must be a string')
+  }
+
+  return {
+    cdnEgressQuota: parseBigInt(response.cdnEgressQuota, 'cdnEgressQuota'),
+    cacheMissEgressQuota: parseBigInt(response.cacheMissEgressQuota, 'cacheMissEgressQuota'),
+  }
+}


not a fan of validating http apis responses

Well, I guess it's okay to not validate HTTP API responses in browsers, where it's easy to reload the page on errors. Since synapse-core is targeting both browsers and long-living Node.js processes, I feel there is enough value in validating API responses.

How do @rvagg and @juliangruber feel about this?

I don't have a strong opinion. If there is a consensus to drop the validation step, then I will change the response handling in getDataSetStats() to this:

return { cdnEgressQuota: BigInt(response.result.cdnEgressQuota), cacheMissEgressQuota: BigInt(response.result.cacheMissEgressQuota), }

The caveat is that both BigInt(undefined) and BigInt('foo') throw a SyntaxError error. If the FilBeam Stats API ever returns an invalid response, the user will see a cryptic error Uncaught TypeError: Cannot convert undefined to a BigInt, and the only reference to FilBeam Stats will be in the stack trace. I suppose that's not entirely bad 🤷🏻

See also https://github.com/FilOzone/synapse-sdk/pull/538/changes#r2708317943

hugomrdias · 2026-01-19T15:58:58Z

packages/synapse-core/src/filbeam/stats.ts

+  const response = await request.json.get<unknown>(url)
+
+  if (response.error) {
+    if (HttpError.is(response.error)) {
+      const status = response.error.response.status
+      if (status === 404) {
+        throw new GetDataSetStatsError(`Data set not found: ${options.dataSetId}`)
+      }
+      const errorText = await response.error.response.text().catch(() => 'Unknown error')
+      throw new GetDataSetStatsError(
+        `Failed to fetch data set stats`,
+        `HTTP ${status} ${response.error.response.statusText}: ${errorText}`
+      )
+    }
+    throw response.error
+  }


if your api is not JSON everywhere, dont use request.json just request.get

I see your point.

The thing is, even if our service always returns JSON everywhere, I don't know if other components like Cloudflare proxy and any proxy the user may have configured are also always returning JSON responses on unexpected errors.

I don't have a strong opinion, this code was generated by Claude Code, I am happy to adjust it to make it consistent with the rest of the codebase.

How do you suggest handling response.error in this function? Should I just wrap it in new GetDataSetStatsError? Would you like to preserve the special handling of 404?

hugomrdias · 2026-01-19T15:59:53Z

packages/synapse-sdk/src/filbeam/service.ts

in this file either import from core or remove completely because its already available in core

I agree it will be best to share the same underlying implementation.

The difficulty: synapse-sdk and synapse-core use different API for making HTTP requests and different error classes.

Let me try to find a solution that preserves the existing synapse-sdk API and uses synapse-core under the hood.

we are changing the sdk to reuse core as much as possible, sdk will use http and errors from core very soon

Addressed in f2bc852 and 3914329

The lens I'm trying to use for core vs sdk is:

core is a stateless functional library with pure functions.

sdk is stateful service classes, DX-focused particularly on the golden-path (don't make me care about your junk).

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx

Move CDN info (Globe icon + tooltip + egress quota display) into a dedicated CdnDetails component, simplifying data-sets-section.tsx. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

Rename query key from `synapse-filbeam-egress-quota` to `synapse-filbeam-egress-quotas` for consistency with other query keys that use plural when the response contains multiple values. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

…apse-core Simplify FilBeamService by delegating getDataSetStats to synapse-core's implementation, eliminating code duplication. Add requestGetJson option to synapse-core for testability. Update SynapseError to prioritize explicit details over cause.message. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

Move detailed HTTP error handling and fetch tests from synapse-sdk to synapse-core, keeping only smoke tests in synapse-sdk to verify the delegation works correctly. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

Move helper functions (getStatsBaseUrl, parseBigInt, validateStatsResponse) after the main getDataSetStats function for better code organization. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos · 2026-01-20T13:16:39Z

packages/synapse-core/src/filbeam/stats.ts

+  const response = await requestGetJson<unknown>(url)
+
+  if (response.error) {
+    if (HttpError.is(response.error)) {
+      const status = response.error.response.status
+      if (status === 404) {
+        throw new GetDataSetStatsError(`Data set not found: ${options.dataSetId}`, {
+          cause: response.error,
+        })
+      }
+      const errorText = await response.error.response.text().catch(() => 'Unknown error')
+      throw new GetDataSetStatsError(`Failed to fetch data set stats`, {
+        details: `HTTP ${status} ${response.error.response.statusText}: ${errorText}`,
+        cause: response.error,
+      })
+    }
+    throw new GetDataSetStatsError('Unexpected error', { cause: response.error })
+  }
+
+  return validateStatsResponse(response.result)


Following up on #538 (comment) and #538 (comment).

If we apply both suggestions, then we can simplify this block as follows - is that what we want?

const response = await requestGetJson<unknown>(url) if (response.error) { throw new GetDataSetStatsError( 'Cannot get DataSet stats from FilBeam API.', { cause: response.error } ) } return { cdnEgressQuota: BigInt(response.result.cdnEgressQuota), cacheMissEgressQuota: BigInt(response.result.cacheMissEgressQuota), }

Copilot

Pull request overview

This PR adds the ability to display remaining CDN and cache-miss egress quotas for FilBeam-enabled data sets in the demo application. The implementation refactors FilBeam stats functionality from synapse-sdk into a new synapse-core module that can be shared between synapse-sdk and synapse-react packages.

Changes:

Moved FilBeam stats API logic from synapse-sdk to synapse-core with improved error handling and validation
Added a new React hook useEgressQuota in synapse-react for fetching egress quota data
Updated the playground app to display remaining egress allowances for data sets with FilBeam enabled

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
packages/synapse-core/src/filbeam/stats.ts	New core implementation for fetching and validating FilBeam data set stats
packages/synapse-core/src/errors/filbeam.ts	New error class for FilBeam stats operations
packages/synapse-core/test/filbeam.test.ts	Comprehensive tests for FilBeam stats functionality
packages/synapse-core/src/filbeam/index.ts	Export module for FilBeam functionality
packages/synapse-core/src/errors/index.ts	Added export for FilBeam errors
packages/synapse-core/src/errors/base.ts	Improved error message handling to prioritize explicit details
packages/synapse-core/package.json	Added exports for filbeam module
packages/synapse-sdk/src/filbeam/service.ts	Refactored to delegate to synapse-core implementation
packages/synapse-sdk/src/test/filbeam-service.test.ts	Updated to smoke tests that verify delegation to synapse-core
packages/synapse-react/src/warm-storage/use-egress-quota.ts	New React hook for querying egress quotas
packages/synapse-react/src/warm-storage/index.ts	Added export for useEgressQuota hook
apps/synapse-playground/src/components/warm-storage/cdn-details.tsx	New component to display egress quota information
apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx	Updated to use CdnDetails component
apps/synapse-playground/src/lib/utils.ts	Added formatBytes utility function
AGENTS.md	Added documentation about monorepo structure and testing practices

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

apps/synapse-playground/src/lib/utils.ts

Copilot · 2026-01-20T13:16:53Z

packages/synapse-core/src/filbeam/stats.ts

+export function getStatsBaseUrl(chainId: number): string {
+  return chainId === 314 ? 'https://stats.filbeam.com' : 'https://calibration.stats.filbeam.com'


The getStatsBaseUrl function defaults all unknown chain IDs to the calibration URL. When devnet (31415926) is passed, it will incorrectly use the calibration URL instead of a devnet-specific URL or throwing an error. This could lead to silent failures when using devnet.

apps/synapse-playground/src/components/warm-storage/cdn-details.tsx

packages/synapse-core/src/filbeam/stats.ts

Copilot · 2026-01-20T13:16:54Z

packages/synapse-sdk/src/test/filbeam-service.test.ts

+
 describe('FilBeamService', () => {
  describe('network type validation', () => {
    it('should throw error if network type not mainnet or calibration', () => {


The test description states "should throw error if network type not mainnet or calibration" but the FilBeamService now supports 'devnet' as well (as shown in service.ts line 51). The test description should be updated to reflect this, or a test for devnet support should be added.

This is a great call-out. FilBeam does not support "devnet". What should the function getDataSetStats() do in such a case?

We can throw an error for unsupported networks.

We can return 0 remaining egress (which is kind of true?) This will work well in the UI.

We can return undefined remaining egress to indicate that FilBeam does not support that network. The UI can handle this case and display n/a:
Egress remaining: n/a delivery · n/a cache-miss

I propose keeping this simple and throwing an error for unsupported networks like "devnet".

Thoughts?

Copilot · 2026-01-20T13:16:54Z

packages/synapse-core/src/filbeam/stats.ts

+}
+
+export interface GetDataSetStatsOptions {
+  /** The chain ID (314 for mainnet, 314159 for calibration) */


The documentation comment lists only chain IDs 314 and 314159, but the codebase supports devnet (31415926) as well. The comment should include all supported chain IDs for completeness.

Suggested change

/** The chain ID (314 for mainnet, 314159 for calibration) */

/** The chain ID (314 for mainnet, 314159 for calibration, 31415926 for devnet) */

not a bad suggestion

It's a great suggestion! But: FilBeam does not support devnet 🤔

I think perhaps the whole style of this file is goiong to need to change to adapt to the style in #555 and #566 which also includes chain definition stuff, check out #566 in particular for the new synapse-core method replacements and the way @hugomrdias is doing namespaces and function arguments

apps/synapse-playground/src/lib/utils.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

AGENTS.md

Co-authored-by: Rod Vagg <rod@vagg.org>

bajtos added 2 commits January 7, 2026 12:06

feat: show remaining CDN & cache-miss egress

fd491d0

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

chore: document synapse-react in AGENTS.md

9aa16b7

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos requested review from juliangruber and pyropy January 7, 2026 11:12

bajtos requested review from hugomrdias and rvagg as code owners January 7, 2026 11:12

github-project-automation bot added this to FOC Jan 7, 2026

github-project-automation bot moved this to 📌 Triage in FOC Jan 7, 2026

juliangruber approved these changes Jan 7, 2026

View reviewed changes

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx Outdated Show resolved Hide resolved

packages/synapse-react/src/warm-storage/use-data-sets.ts Outdated Show resolved Hide resolved

bajtos added 3 commits January 7, 2026 12:25

fixup! cache-miss icon RefreshCW -> FolderSync

2132a05

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

fixup! npm run lint:fix

f029bc4

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

fixup! code cleanup + better error logging

b1f0f04

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos mentioned this pull request Jan 7, 2026

feat: access stats from browsers filbeam/worker#539

Merged

rjan90 moved this from 📌 Triage to 🔎 Awaiting review in FOC Jan 7, 2026

rjan90 added this to the M4: Filecoin Service Liftoff milestone Jan 7, 2026

bajtos mentioned this pull request Jan 7, 2026

fix: change FilBeam URL to stats.filbeam.com #539

Merged

fixup! forgotten err

10c3811

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos commented Jan 7, 2026

View reviewed changes

packages/synapse-react/src/warm-storage/use-data-sets.ts Outdated Show resolved Hide resolved

Merge branch 'master' into feat-egress-allowance

c0bbd0e

rjan90 moved this from 🔎 Awaiting review to ✔️ Approved by reviewer in FOC Jan 7, 2026

rjan90 assigned bajtos Jan 7, 2026

rvagg removed their request for review January 8, 2026 08:04

BigLep moved this from ✔️ Approved by reviewer to 🔎 Awaiting review in FOC Jan 8, 2026

pyropy reviewed Jan 12, 2026

View reviewed changes

apps/synapse-playground/src/lib/utils.ts Outdated Show resolved Hide resolved

bajtos requested review from juliangruber and pyropy January 13, 2026 10:12

bajtos and others added 2 commits January 14, 2026 12:19

fix: add PiB and EiB to formatBytes helper

725f20b

just in case Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos requested a review from hugomrdias January 14, 2026 12:17

bajtos moved this from ⌨️ In Progress to 🔎 Awaiting review in FOC Jan 14, 2026

juliangruber approved these changes Jan 14, 2026

View reviewed changes

bajtos commented Jan 14, 2026

View reviewed changes

apps/synapse-playground/src/components/warm-storage/data-sets-section.tsx Outdated Show resolved Hide resolved

apps/synapse-playground/src/lib/utils.ts Show resolved Hide resolved

packages/synapse-react/src/warm-storage/use-egress-quota.ts Outdated Show resolved Hide resolved

docs: clarify CDN & cache-miss quotas in API docs

30952c2

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

rjan90 mentioned this pull request Jan 19, 2026

Cut Synapse-SDK 0.37.0 #564

Open

hugomrdias requested changes Jan 19, 2026

View reviewed changes

github-project-automation bot moved this from 🔎 Awaiting review to ⌨️ In Progress in FOC Jan 19, 2026

bajtos and others added 6 commits January 20, 2026 12:53

Merge branch 'master' into feat-egress-allowance

abd0d39

refactor(playground): extract CdnDetails component

47d0a25

Move CDN info (Globe icon + tooltip + egress quota display) into a dedicated CdnDetails component, simplifying data-sets-section.tsx. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos requested review from Copilot, hugomrdias and rvagg January 20, 2026 13:12

Copilot started reviewing on behalf of bajtos January 20, 2026 13:13 View session

bajtos commented Jan 20, 2026

View reviewed changes

Copilot AI reviewed Jan 20, 2026

View reviewed changes

bajtos and others added 2 commits January 20, 2026 16:29

Fix formatBytes to support negative values

6f7f718

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

fix: harden the implementation of formatBytes

0063a33

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>

bajtos moved this from ⌨️ In Progress to 🔎 Awaiting review in FOC Jan 20, 2026

rvagg reviewed Jan 21, 2026

View reviewed changes

AGENTS.md Outdated Show resolved Hide resolved

rvagg reviewed Jan 21, 2026

View reviewed changes

AGENTS.md Outdated Show resolved Hide resolved

bajtos and others added 2 commits January 21, 2026 11:28

Update AGENTS.md

1bd4f2c

Co-authored-by: Rod Vagg <rod@vagg.org>

Update AGENTS.md

53c49e5

Co-authored-by: Rod Vagg <rod@vagg.org>

		export function getStatsBaseUrl(chainId: number): string {
		return chainId === 314 ? 'https://stats.filbeam.com' : 'https://calibration.stats.filbeam.com'

	/** The chain ID (314 for mainnet, 314159 for calibration) */
	/** The chain ID (314 for mainnet, 314159 for calibration, 31415926 for devnet) */

feat: show remaining CDN & cache-miss egress #538

Are you sure you want to change the base?

feat: show remaining CDN & cache-miss egress #538

Uh oh!

Conversation

bajtos commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cloudflare-workers-and-pages bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rvagg commented Jan 8, 2026

Uh oh!

Uh oh!

bajtos commented Jan 13, 2026

Uh oh!

bajtos commented Jan 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bajtos Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

bajtos commented Jan 7, 2026 •

edited

Loading

cloudflare-workers-and-pages bot commented Jan 7, 2026 •

edited

Loading

bajtos Jan 20, 2026 •

edited

Loading