-
Notifications
You must be signed in to change notification settings - Fork 229
Introduce TomlFile abstraction for TOML file I/O #6943
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+481
−6
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...ages/cli-kit/src/public/node/toml.test.ts → ...li-kit/src/public/node/toml/codec.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
packages/cli-kit/src/public/node/toml.ts → ...ges/cli-kit/src/public/node/toml/codec.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| export {decodeToml, encodeToml} from './codec.js' | ||
| export type {JsonMapType} from './codec.js' |
299 changes: 299 additions & 0 deletions
299
packages/cli-kit/src/public/node/toml/toml-file.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,299 @@ | ||
| import {TomlFile, TomlParseError} from './toml-file.js' | ||
| import {writeFile, readFile, inTemporaryDirectory} from '../fs.js' | ||
| import {joinPath} from '../path.js' | ||
| import {describe, expect, test} from 'vitest' | ||
|
|
||
| describe('TomlFile', () => { | ||
| describe('read', () => { | ||
| test('reads and parses a TOML file', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "my-app"\nclient_id = "123"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
|
|
||
| expect(file.path).toBe(path) | ||
| expect(file.content).toStrictEqual({name: 'my-app', client_id: '123'}) | ||
| }) | ||
| }) | ||
|
|
||
| test('reads nested tables', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '[build]\ndev_store_url = "my-store.myshopify.com"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
|
|
||
| expect(file.content).toStrictEqual({build: {dev_store_url: 'my-store.myshopify.com'}}) | ||
| }) | ||
| }) | ||
|
|
||
| test('throws TomlParseError with file path on invalid TOML', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'bad.toml') | ||
| await writeFile(path, 'name = [invalid') | ||
|
|
||
| await expect(TomlFile.read(path)).rejects.toThrow(TomlParseError) | ||
| await expect(TomlFile.read(path)).rejects.toThrow(/bad\.toml/) | ||
| }) | ||
| }) | ||
|
|
||
| test('throws if file does not exist', async () => { | ||
| await expect(TomlFile.read('/nonexistent/path/test.toml')).rejects.toThrow() | ||
| }) | ||
| }) | ||
|
|
||
| describe('patch', () => { | ||
| test('sets a top-level value', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "old"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({name: 'new'}) | ||
|
|
||
| expect(file.content.name).toBe('new') | ||
| const raw = await readFile(path) | ||
| expect(raw).toContain('name = "new"') | ||
| }) | ||
| }) | ||
|
|
||
| test('sets a nested value', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '[build]\ndev_store_url = "old.myshopify.com"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({build: {dev_store_url: 'new.myshopify.com'}}) | ||
|
|
||
| expect(file.content).toStrictEqual({build: {dev_store_url: 'new.myshopify.com'}}) | ||
| }) | ||
| }) | ||
|
|
||
| test('creates intermediate tables', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({build: {dev_store_url: 'store.myshopify.com'}}) | ||
|
|
||
| expect(file.content).toStrictEqual({ | ||
| name: 'app', | ||
| build: {dev_store_url: 'store.myshopify.com'}, | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| test('sets multiple values at once', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\nclient_id = "123"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({name: 'updated', client_id: '456'}) | ||
|
|
||
| expect(file.content.name).toBe('updated') | ||
| expect(file.content.client_id).toBe('456') | ||
| }) | ||
| }) | ||
|
|
||
| test('preserves comments', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '# This is a comment\nname = "app"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({name: 'updated'}) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('# This is a comment') | ||
| expect(raw).toContain('name = "updated"') | ||
| }) | ||
| }) | ||
|
|
||
| test('handles array values', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '[auth]\nredirect_urls = ["https://old.com"]\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.patch({auth: {redirect_urls: ['https://new.com', 'https://other.com']}}) | ||
|
|
||
| const content = file.content as {auth: {redirect_urls: string[]}} | ||
| expect(content.auth.redirect_urls).toStrictEqual(['https://new.com', 'https://other.com']) | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| describe('remove', () => { | ||
| test('removes a top-level key', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\nclient_id = "123"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.remove('name') | ||
|
|
||
| expect(file.content.name).toBeUndefined() | ||
| expect(file.content.client_id).toBe('123') | ||
| }) | ||
| }) | ||
|
|
||
| test('removes a nested key', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile( | ||
| path, | ||
| '[build]\ndev_store_url = "store.myshopify.com"\nautomatically_update_urls_on_dev = true\n', | ||
| ) | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.remove('build.dev_store_url') | ||
|
|
||
| const build = file.content.build as {[key: string]: unknown} | ||
| expect(build.dev_store_url).toBeUndefined() | ||
| expect(build.automatically_update_urls_on_dev).toBe(true) | ||
| }) | ||
| }) | ||
|
|
||
| test('preserves unrelated content', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\nclient_id = "123"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.remove('name') | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('client_id = "123"') | ||
| expect(raw).not.toContain('name') | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| describe('replace', () => { | ||
| test('replaces the entire file content', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "old"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.replace({name: 'new', client_id: '789'}) | ||
|
|
||
| expect(file.content).toStrictEqual({name: 'new', client_id: '789'}) | ||
| const raw = await readFile(path) | ||
| expect(raw).toContain('name = "new"') | ||
| expect(raw).toContain('client_id = "789"') | ||
| }) | ||
| }) | ||
|
|
||
| test('does not preserve comments', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '# Comment\nname = "old"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.replace({name: 'new'}) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).not.toContain('# Comment') | ||
| }) | ||
| }) | ||
|
|
||
| test('round-trips read → replace → read', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| const original = { | ||
| name: 'my-app', | ||
| client_id: 'abc123', | ||
| build: {dev_store_url: 'store.myshopify.com'}, | ||
| } | ||
| await writeFile(path, 'name = "placeholder"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.replace(original) | ||
|
|
||
| const reread = await TomlFile.read(path) | ||
| expect(reread.content).toStrictEqual(original) | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| describe('transformRaw', () => { | ||
| test('transforms the raw TOML string and updates content', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.transformRaw((raw) => `# Header comment\n${raw}`) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('# Header comment') | ||
| expect(raw).toContain('name = "app"') | ||
| expect(file.content.name).toBe('app') | ||
| }) | ||
| }) | ||
|
|
||
| test('injected comments survive subsequent patch calls', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, 'name = "app"\nclient_id = "123"\n') | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await file.transformRaw((raw) => `# Keep this comment\n${raw}`) | ||
| await file.patch({name: 'updated'}) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('# Keep this comment') | ||
| expect(raw).toContain('name = "updated"') | ||
| }) | ||
| }) | ||
|
|
||
| test('works after replace to add comments', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| await writeFile(path, '') | ||
|
|
||
| const file = new TomlFile(path, {}) | ||
| await file.replace({name: 'app', client_id: '123'}) | ||
| await file.transformRaw((raw) => `# Doc link\n${raw}`) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('# Doc link') | ||
| expect(raw).toContain('name = "app"') | ||
| expect(file.content).toStrictEqual({name: 'app', client_id: '123'}) | ||
| }) | ||
| }) | ||
|
|
||
| test('throws TomlParseError and does not write to disk when transform produces invalid TOML', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'test.toml') | ||
| const originalContent = 'name = "app"\n' | ||
| await writeFile(path, originalContent) | ||
|
|
||
| const file = await TomlFile.read(path) | ||
| await expect(file.transformRaw(() => 'name = [invalid')).rejects.toThrow(TomlParseError) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toBe(originalContent) | ||
| expect(file.content).toStrictEqual({name: 'app'}) | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| describe('constructor', () => { | ||
| test('creates a TomlFile instance for new files', async () => { | ||
| await inTemporaryDirectory(async (dir) => { | ||
| const path = joinPath(dir, 'new.toml') | ||
| const file = new TomlFile(path, {}) | ||
| await file.replace({type: 'ui_extension', name: 'My Extension'}) | ||
|
|
||
| const raw = await readFile(path) | ||
| expect(raw).toContain('type = "ui_extension"') | ||
| expect(raw).toContain('name = "My Extension"') | ||
| }) | ||
| }) | ||
| }) | ||
| }) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you shouldn't need this, is already covered by the wildcard below :thinking_face:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, IIRC imports from the new toml dir didn't work without this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea this is necessary to import from `@shopify/cli-kit/node/toml which the next pr does
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all other modules work with the wildcard export, for instance
/node/cryptoor/node/pathI guess the issue is having an index file, can we do this without an index file and expose a
toml.tsdirectly?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
node/crypto and node/path both point to files in the public/node root, not subfolders. i put the toml files in their own subdir since it's a domain with multiple file instances and i didn't want to keep expanding the root, which is already very large.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i guess we could do re-exports from the root, but i don't see much value in it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just worried of having to manually export things here, can grow a lot very quickly, let's leave it for now and we can revisit if this grows too much.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious -- why? The cost of change is basically zero to update an import alias/etc. I personally prefer more structured directories, as it's easier to visually navigate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a preference of having automatic exports (or maybe an index file) and not needing to update the package.json for every folder/file we add to the project