feat(cli): add --car flag to bulletin upload by BigTava · Pull Request #36 · paritytech/dotns-sdk

BigTava · 2026-03-17T17:00:25Z

Description

Adds a --use-car flag to dotns bulletin upload for directory uploads. When set, the CLI:

Merkleizes the directory using the local IPFS CLI (ipfs add -r --cid-version=1 --raw-leaves)
Exports a CAR file (ipfs dag export)
Uploads the CAR as a single chunked file to Bulletin

This replaces the three separate Setup IPFS Kubo / Initialize IPFS / Merkleize into CAR file steps that were previously only available in the deploy workflow (PR #27). Now callers can trigger the same behaviour directly from the CLI:

dotns bulletin upload ./dist --use-car --parallel

The IPFS repo is initialised automatically (ipfs init) if not already present. The flag is a no-op for file uploads (errors with a clear message).

Why CAR upload?

The default mode merkleizes in-memory and stores each IPFS block individually. CAR-based upload writes fewer, larger transactions, which can be faster for large sites and is required for compatibility with gateways that expect a packed DAG (e.g. the Polkadot Triangle gateway).

Type

Package

@dotns/cli
Root/monorepo
Documentation

Related Issues

Closes #28 (no progress feedback during upload — CAR upload makes this more noticeable)

Fixes

N/A

Checklist

Code

Follows project style
bun run lint passes
bun run format passes
bun run typecheck passes

Documentation

README updated if needed
Types updated if needed

Breaking Changes

No breaking changes
Breaking changes documented below

Testing

How to test:

Upload a directory without the flag — verify existing block-by-block behaviour is unchanged
```
dotns bulletin upload ./examples/deploy/site
```
Upload a directory with --use-car — verify IPFS CLI is invoked, a temp CAR file is created, and upload succeeds
```
dotns bulletin upload ./examples/deploy/site --use-car --parallel
```

Upload a single file with --use-car — verify it fails with a clear error message

dotns bulletin upload ./examples/deploy/site/index.html --use-car
# Expected: Error: --use-car is only supported for directory uploads

Verify the returned CID matches the IPFS CID of the directory:

ipfs add -Q -r --cid-version=1 --raw-leaves ./examples/deploy/site
# Should match the CID printed by dotns bulletin upload --use-car

Notes

Requires IPFS Kubo to be installed and on $PATH (e.g. via brew install ipfs or the deploy workflow's ipfs/download-ipfs-distribution-action)
The temp CAR file is written to os.tmpdir() and deleted after upload
The workflow (.github/workflows/deploy.yml) use-car: true input now delegates merkleization to this CLI flag instead of the separate shell steps — see PR feat: add use-car option to deploy workflow #27
An example workflow using use-car is included at examples/deploy/site/workflows/car-upload.yml

closes https://github.com/paritytech/dotns/issues/95

github-actions · 2026-03-17T17:00:38Z

CI Summary

Check	Result
Lint	Passed
Format	Passed
Typecheck	Passed
Build	Passed
Release	Passed
Deploy Example	Passed
PR Title	Passed
Labels	Passed

Release - Passed

Test this PR

Download artifact (GitHub CLI required):

gh run download 23238233460 -n cli-preview-0.0.0-pr.36 -R paritytech/dotns-sdk

Install globally:

Package Manager	Command
npm	`npm install -g ./dotns-cli-0.0.0-pr.36.tgz`
yarn	`yarn global add ./dotns-cli-0.0.0-pr.36.tgz`
bun (macOS/Linux)	`bun add -g "$(pwd)/dotns-cli-0.0.0-pr.36.tgz"`
bun (Windows)	`bun add -g "$PWD\dotns-cli-0.0.0-pr.36.tgz"`

Verify:

dotns --help

Deploy Example — Passed

Stage	Status
✓ Site validation	Site validated
✓ Deploy	Deployed (cached)

Property	Value
Domain	`pr36.dotns-example-site.dot`
CID	`bafybeibkahy65wntair6iba7fal3oky4fnznifzuvnb4sz66qmoldzk7ti` (cached)
URL	https://pr36.dotns-example-site.paseo.li

View run

Labels

pkg: cli, scope: bulletin, type: docs, type: test

Adds `--use-car` to `dotns bulletin upload` so the CLI can merkleize a directory with the IPFS CLI, export a CAR file, and upload it to Bulletin — replacing the three separate workflow steps (Setup IPFS Kubo, Initialize IPFS, Merkleize into CAR file).

andrew-ifrita

Left some comments regarding only the code changes. Most are small nits. A small refactor would be nice where mentioned just to help with testability but that isn't required.

The main missing piece is that the --use-car path seems to be missing in the test suite.

On the more subjective side, --use-car may not be the most intuitive, athough I have mixed feelings about the suggestions off the top of my head

--archive (-as-car?)
--bundle (-as-car?)
--bundled
--car (may be the best if we can assume familiarity with IPFS, but may not be the best assumption to make)

andrew-ifrita · 2026-03-17T18:34:17Z

@BigTava Outside of the code specific changes, the CAR stuff makes complete sense in transit, but do you realize that we are referencing the CAR bytes directly with the CID instead of the unpacked root CID? I would normally assume this is a mistake, but I remember some old CAR handling code in the past.

My assumption is that we just want to do something like this with out script

ipfs dag import archive.car

I don't have a ton of ipfs via js experience but claude suggests

import { CarReader } from '@ipld/car';
                                                                                      
// ... after exportCarFile(ipfsCid, tempCarPath) ...                                  
const carBytes = await filesystem.readFile(tempCarPath);                              
const reader = await CarReader.fromBytes(new Uint8Array(carBytes));                   
const roots = await reader.getRoots();                                                
const rootCid = roots[0].toString(); // should match ipfsCid                          
                                                                                      
const blocks = [];                                                                    
for await (const block of reader.blocks()) {                                          
  blocks.push(block);                                                                 
}                                                                                     
                                                                                      
// Upload each block individually (reuse existing wave logic from storeDirectory)     
for (const block of blocks) {
  await storeBlockToBulletin({                                                        
    rpc: bulletinRpc,                                                                 
    signer,                                                                           
    contentBytes: new Uint8Array(block.bytes),                                        
    contentCid: block.cid.toString(),                                                 
    codecValue: block.cid.code,                                                       
    hashCodeValue: block.cid.multihash.code,                                          
    client: sharedClient,                                                             
  });                                                                                 
}                                                                                     
                
return { cid: rootCid, ipfsCid: rootCid, size: totalSize };

Something like this will give you the benefits of CAR when sending without any special handling of Content Addressable aRchives for reading.

andrew-ifrita · 2026-03-17T18:37:43Z

+                  storageCid = await uploadSingleBlock(bulletinRpc, context.signer, carBytes);
+                }
+
+                return { cid: storageCid, ipfsCid, size: carBytes.length };


This will return the cid of the car blob which is non-standard for anyone accessing the data as CAR is meant for transport.

We likely want something like

const reader = await CarReader.fromBytes(new Uint8Array(carBytes)); const roots = await reader.getRoots(); const rootCid = roots[0].toString(); // should match ipfsCid assert.isEqual(ipfsCid, rootCid); return { cid: rootCid, ipfsCid: rootCid, size: totalSize };

edit: better context in this comment #36 (comment)

The CAR bytes are stored as chunked content with a DAG-PB root via uploadChunkedBlocks(). The cid field returns the DAG-PB root (the Bulletin storage handle the gateway resolves), and ipfsCid carries the IPFS directory root for reference.

The CAR bytes are stored as chunked content with a DAG-PB root

This is what I see as the problem. The CAR is meant for transport. I'll throw an example together. I may just be misreading the code and the only issue is the CID that is output

ipfsCid carries the IPFS directory root for reference.

This part is a little misleading in the current code in master tbh. ipfsCid is only used in getPreviewUrl() by the looks of it. Which is probably the root of the issue

Aside: We should probably consolidate ipfsCid and cid because currently (outside this PR) every cid == ipfsCid. That is outside the scope here and I can follow up with it after. (edit: added issue here https://github.com/paritytech/dotns/issues/97)

I don't know what you were doing before. As I far as I went, querying gateway/<cid> works. gateway/<storageCid> doesn't work.

My example is failing due to possible bulletin chain issues

dotns-sdk/packages/cli/src/bulletin$ bun run dev -- bulletin upload ~/dev/andrew/moonwalker/ --car --parallel $ bun run src/cli/index.ts bulletin upload /home/andrew/dev/andrew/moonwalker/ --car --parallel ════════════════════════════════════════ dotns developer CLI ════════════════════════════════════════ ✔ Directory validated • Connecting RPC wss://paseo-bulletin-rpc.polkadot.io ✓ Connecting RPC wss://paseo-bulletin-rpc.polkadot.io • Resolving account ✓ Resolving account • Loading keypair ✓ Loading keypair 📋 Configuration RPC: wss://paseo-bulletin-rpc.polkadot.io Chain: Bulletin Substrate: 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV Auth: default Account: default Balance: 0.000000000000 ▶ Bulletin Upload path: /home/andrew/dev/andrew/moonwalker rpc: wss://paseo-bulletin-rpc.polkadot.io mode: car (ipfs cli) ✔ Merkleized: bafybeictaikgnxa6aiumef2rfheygvdz2dgmh7sb6f7dumkizodzqdgbua ✔ CAR exported: 3.23 MB chunks: 2 ✖ Upload failed ✗ Error: Descriptor for tx TransactionStorage.store_with_cid_config does not exist error: script "dev" exited with code 1

which makes this hard to test out at the moment. I don't believe this is related to this PR

edit: 🤦 I forgot

bun run prepare

which explains the issues.

Ok here is an example (sorry for delays, had some calls I had to deal with)

bun run dev -- bulletin upload ~/dev/andrew/moonwalker/ --car --parallel $ bun run src/cli/index.ts bulletin upload /home/andrew/dev/andrew/moonwalker/ --car --parallel ════════════════════════════════════════ dotns developer CLI ════════════════════════════════════════ ✔ Directory validated • Connecting RPC wss://paseo-bulletin-rpc.polkadot.io ✓ Connecting RPC wss://paseo-bulletin-rpc.polkadot.io • Resolving account ✓ Resolving account • Loading keypair ✓ Loading keypair 📋 Configuration RPC: wss://paseo-bulletin-rpc.polkadot.io Chain: Bulletin Substrate: 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV Auth: default Account: default Balance: 0.000000000000 ▶ Bulletin Upload path: /home/andrew/dev/andrew/moonwalker rpc: wss://paseo-bulletin-rpc.polkadot.io mode: car (ipfs cli) ✔ Merkleized: bafybeictaikgnxa6aiumef2rfheygvdz2dgmh7sb6f7dumkizodzqdgbua ✔ CAR exported: 3.23 MB chunks: 2 ✔ Stored cid: bafybeibwo72gsgp4qzy7p2gypyxu2oofhnnikcmg74wpys7tchie2ylhqm ipfs-cid: bafybeictaikgnxa6aiumef2rfheygvdz2dgmh7sb6f7dumkizodzqdgbua preview: http://dotns.paseo.li/#/preview/YmFmeWJlaWN0YWlrZ254YTZhaXVtZWYycmZoZXlndmR6MmRnbWg3c2I2ZjdkdW1raXpvZHpxZGdidWE ✓ Upload Complete

Resulting in 2 output CIDs

cid: bafybeibwo72gsgp4qzy7p2gypyxu2oofhnnikcmg74wpys7tchie2ylhqm
ipfs-cid: bafybeictaikgnxa6aiumef2rfheygvdz2dgmh7sb6f7dumkizodzqdgbua

So it works, but cid is effectively worthless. I would simply replace

return { cid: storageCid, ipfsCid, size: carBytes.length };

with

return { cid: ipfsCid, ipfsCid, size: carBytes.length };

and we should be good.

outside the scope of the PR, a nice touch would also be only outputting the cid when piping, allowing easy usage in bash

I think that the ipfsCid only works when you first fetch the storageCid. Only then the ipfsCid is cached in the gateway.

I pushed a PR to merge into this one: #43

Hopefully that clears things up a little

Did you change anything in the build so the CID is different? Also, did you try to fetch directly from the bulletin nodes via Helia? Did you try to fetch from the only ipfs gateway that matters - https://paseo-ipfs.polkadot.io/ipfs? Also this is not working for me https://dotns.paseo.li/#/preview/YmFmeWJlaWdreGs1eG92cmlzdzVya2NvZ3J3ZzJ2M3FzcHlscnlsem9uYnl0Y20zeHhkdnMzeGJud2k.

I think that the ipfsCid only works when you first fetch the storageCid. Only then the ipfsCid is cached in the gateway.

With the linked changes in #43, we are storing each block individually with its original CID, that two-step process shouldn't be necessary anymore. The ipfsCid blocks are stored directly on Bulletin.

sphamjoli · 2026-03-17T20:32:47Z

  transactions?: string;
  /** Number of bytes to authorize */
  bytes?: string;
+  /** Merkleize with IPFS CLI and upload as a CAR file instead of individual blocks */


Great start, We missing helper tests and integration tests here

- Extract CAR upload logic into storeCar() in commands/bulletin.ts, mirroring deploy.js storeChunkedContent: exports a CAR, stores the bytes as chunked content with a DAG-PB root via uploadChunkedBlocks - Use randomUUID() for the temp CAR filename instead of Date.now() - Add safety comment on execSync calls in ensureIpfsInitialized() - Simplify cli/commands/bulletin.ts: useCar path is now a single call to storeCar(); removes previously inlined ora/fs/os/path/ipfs imports

…o --car - ensureIpfsInitialized() now returns the resolved binary path so callers can pass it into merkleizeWithIpfs() and exportCarFile(), avoiding repeated findIpfsBinaryPath() lookups and improving testability - merkleizeWithIpfs, merkleizeSingleFileWithIpfs, and exportCarFile each accept an optional ipfsBinaryPath parameter (falls back to findIpfsBinaryPath() for standalone callers) - storeCar() resolves the path once via ensureIpfsInitialized() and passes it through both subsequent IPFS calls - Rename --use-car flag to --car (matches --parallel, --force-chunked naming convention and is more concise for IPFS-familiar users)

- Unit: assert --car appears in bulletin upload --help output - Integration: upload directory with --car, verify mode line and ipfs-cid field in terminal output - Integration: --car with --json, verify cid (DAG-PB root) and ipfsCid (IPFS directory root) are both present - Integration: --car on a file path surfaces a clear error message - Extend JsonUploadResult type with optional ipfsCid field

BigTava · 2026-03-18T09:37:25Z

@sphamjoli @andrew-ifrita Thanks for your comments.

BigTava · 2026-03-19T15:36:57Z

@andrew-ifrita what is the status?

andrew-ifrita · 2026-03-24T14:04:03Z

I worked out my train of thought in #43. Here is my resulting opinion #43 (comment)

If a user wants to upload a car directly, they can do it with the existing functionality after first generating the CAR and then uploading that blob. We shouldn't help the user shoot themselves in the foot by uploading a CAR blob in a way that goes against standard IPFS usage and practices.

If a user wants, nothing is preventing something like

ipfs export ./project | dotns bulletin upload

So this feature shouldn't be implemented on the CLI side and I would label this as a "good enough" solution that we are trying to avoid. A proper solution would include adding CAR upload support to Bulletin. And even a useful intermediate solution would be allowing batched transaction on bulletin, but this last option may not be feasible for technical reason (I am not a bulletin expert by any means).

BigTava requested review from andrew-ifrita, corey-hathaway, filvecchiato, sphamjoli and waytwotall as code owners March 17, 2026 17:00

github-actions Bot added pkg: cli scope: bulletin type: docs labels Mar 17, 2026

BigTava force-pushed the feat/deploy-use-car branch from 49d6456 to 776a309 Compare March 17, 2026 17:04

style: fix prettier formatting

43fd720

andrew-ifrita requested changes Mar 17, 2026

View reviewed changes

Comment thread packages/cli/src/bulletin/ipfs.ts

Comment thread packages/cli/src/bulletin/ipfs.ts

Comment thread packages/cli/src/cli/commands/bulletin.ts Outdated

Comment thread packages/cli/src/cli/commands/bulletin.ts

andrew-ifrita reviewed Mar 17, 2026

View reviewed changes

sphamjoli requested changes Mar 17, 2026

View reviewed changes

BigTava added 3 commits March 17, 2026 23:28

github-actions Bot added the type: test label Mar 18, 2026

BigTava added 2 commits March 18, 2026 00:23

chore: remove accidentally committed IPFS binary

a6cbaca

style: fix prettier formatting

1ad7806

leonardocustodio mentioned this pull request Mar 18, 2026

[UI] Add support for SPA/CAR uploads to the console-ui? paritytech/polkadot-bulletin-chain#336

Open

2 tasks

andrew-ifrita changed the title ~~feat(cli): add --use-car flag to bulletin upload~~ feat(cli): add --car flag to bulletin upload Mar 24, 2026

andrew-ifrita mentioned this pull request Mar 24, 2026

feat: Use CAR only in transit #43

Closed

sphamjoli closed this Mar 25, 2026

andrew-ifrita mentioned this pull request Mar 25, 2026

[Feature request] ipfs import functionality when uploading CARs paritytech/polkadot-bulletin-chain#353

Open

Conversation

BigTava commented Mar 17, 2026 • edited by andrew-ifrita Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Why CAR upload?

Type

Package

Related Issues

Fixes

Checklist

Code

Documentation

Breaking Changes

Testing

Notes

Uh oh!

github-actions Bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Summary

Uh oh!

andrew-ifrita left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrew-ifrita commented Mar 17, 2026

Uh oh!

andrew-ifrita Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BigTava Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrew-ifrita Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BigTava Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

andrew-ifrita Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrew-ifrita Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BigTava Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

andrew-ifrita Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

BigTava Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrew-ifrita Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sphamjoli Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BigTava Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BigTava commented Mar 18, 2026

Uh oh!

BigTava commented Mar 19, 2026

Uh oh!

andrew-ifrita commented Mar 24, 2026

BigTava commented Mar 17, 2026 •

edited by andrew-ifrita

Loading

github-actions Bot commented Mar 17, 2026 •

edited

Loading

andrew-ifrita left a comment •

edited

Loading

andrew-ifrita Mar 17, 2026 •

edited

Loading

BigTava Mar 17, 2026 •

edited

Loading

andrew-ifrita Mar 18, 2026 •

edited

Loading

andrew-ifrita Mar 18, 2026 •

edited

Loading

andrew-ifrita Mar 18, 2026 •

edited

Loading

BigTava Mar 24, 2026 •

edited

Loading

andrew-ifrita Mar 24, 2026 •

edited

Loading

sphamjoli Mar 17, 2026 •

edited

Loading

BigTava Mar 17, 2026 •

edited

Loading