-
-
Notifications
You must be signed in to change notification settings - Fork 26
feat(scraper): ZaparooCompanion xml support #839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
47e343c
parent title work seemingly stable but likely to revert
BossRighteous 484d87e
companion scraper updates, removing parent stuff
BossRighteous 8ea1e55
stable companion scraper fmtand recallbox paths
BossRighteous 77ba466
lint fixes
BossRighteous 879dcf4
test update
BossRighteous c6acc70
Merge branch 'main' into zaparoocompanion-xml-support
BossRighteous aec0a8b
slug ext path matcher and test coverage
BossRighteous 92c44a0
zip as dirs on windows
BossRighteous 1869f1e
final review changes
BossRighteous 493845a
fix(api): rename media title parse method
wizzomafizzo 968f3f8
Merge branch 'main' into zaparoocompanion-xml-support
wizzomafizzo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| // Zaparoo Core | ||
| // Copyright (c) 2026 The Zaparoo Project Contributors. | ||
| // SPDX-License-Identifier: GPL-3.0-or-later | ||
| // | ||
| // This file is part of Zaparoo Core. | ||
| // | ||
| // Zaparoo Core is free software: you can redistribute it and/or modify | ||
| // it under the terms of the GNU General Public License as published by | ||
| // the Free Software Foundation, either version 3 of the License, or | ||
| // (at your option) any later version. | ||
| // | ||
| // Zaparoo Core is distributed in the hope that it will be useful, | ||
| // but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| // GNU General Public License for more details. | ||
| // | ||
| // You should have received a copy of the GNU General Public License | ||
| // along with Zaparoo Core. If not, see <http://www.gnu.org/licenses/>. | ||
|
|
||
| package methods | ||
|
|
||
| import ( | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/api/models" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/api/models/requests" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/api/validation" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/database/mediadb" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/database/mediascanner" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/database/slugs" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/database/systemdefs" | ||
| ) | ||
|
|
||
| // HandleMediaTitleParse computes a MediaTitle from a system ID and path | ||
| // without touching the filesystem or database. Used to preview the title | ||
| // parsing and slug generation that the media scanner would produce. | ||
| func HandleMediaTitleParse( | ||
| env requests.RequestEnv, //nolint:gocritic // single-use parameter in API handler | ||
| ) (any, error) { | ||
| var params models.MediaTitleParseParams | ||
| if err := validation.ValidateAndUnmarshal(env.Params, ¶ms); err != nil { | ||
| return nil, models.ClientErrf("invalid params: %w", err) | ||
| } | ||
|
|
||
| mediaType := slugs.MediaTypeGame | ||
| if params.SystemID != "" { | ||
| if system, err := systemdefs.GetSystem(params.SystemID); err == nil { | ||
| mediaType = system.GetMediaType() | ||
| } | ||
| } | ||
|
|
||
| pf := mediascanner.GetPathFragments(&mediascanner.PathFragmentParams{ | ||
| Config: env.Config, | ||
| Path: params.Path, | ||
| SystemID: params.SystemID, | ||
| MediaType: mediaType, | ||
| }) | ||
|
|
||
| metadata := mediadb.GenerateSlugMetadataFromTokens(mediaType, pf.Title, pf.Slug, pf.SlugTokens) | ||
|
|
||
| var secondarySlug *string | ||
| if metadata.SecondarySlug != "" { | ||
| s := metadata.SecondarySlug | ||
| secondarySlug = &s | ||
| } | ||
|
|
||
| return models.MediaTitleParseResponse{ | ||
| Slug: pf.Slug, | ||
| Name: pf.Title, | ||
| SecondarySlug: secondarySlug, | ||
| SlugLength: metadata.SlugLength, | ||
| SlugWordCount: metadata.SlugWordCount, | ||
| }, nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,253 @@ | ||
| // Zaparoo Core | ||
| // Copyright (c) 2026 The Zaparoo Project Contributors. | ||
| // SPDX-License-Identifier: GPL-3.0-or-later | ||
| // | ||
| // This file is part of Zaparoo Core. | ||
| // | ||
| // Zaparoo Core is free software: you can redistribute it and/or modify | ||
| // it under the terms of the GNU General Public License as published by | ||
| // the Free Software Foundation, either version 3 of the License, or | ||
| // (at your option) any later version. | ||
| // | ||
| // Zaparoo Core is distributed in the hope that it will be useful, | ||
| // but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| // GNU General Public License for more details. | ||
| // | ||
| // You should have received a copy of the GNU General Public License | ||
| // along with Zaparoo Core. If not, see <http://www.gnu.org/licenses/>. | ||
|
|
||
| package methods | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "path/filepath" | ||
| "testing" | ||
| "unicode/utf8" | ||
|
|
||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/api/models" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/api/models/requests" | ||
| "github.com/ZaparooProject/zaparoo-core/v2/pkg/config" | ||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func makeTitleParseEnv(t *testing.T, systemID, path string) requests.RequestEnv { | ||
| t.Helper() | ||
| params, err := json.Marshal(map[string]string{ | ||
| "systemId": systemID, | ||
| "path": path, | ||
| }) | ||
| require.NoError(t, err) | ||
| return requests.RequestEnv{ | ||
| Context: context.Background(), | ||
| Config: &config.Instance{}, | ||
| Params: params, | ||
| } | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_InvalidParams(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| nesPath := filepath.Join("roms", "nes", "game.nes") | ||
|
|
||
| missingSystemParams, err := json.Marshal(map[string]string{"path": nesPath}) | ||
| require.NoError(t, err) | ||
|
|
||
| emptySystemParams, err := json.Marshal(map[string]string{"systemId": "", "path": nesPath}) | ||
| require.NoError(t, err) | ||
|
|
||
| tests := []struct { | ||
| name string | ||
| params []byte | ||
| }{ | ||
| { | ||
| name: "missing systemId", | ||
| params: missingSystemParams, | ||
| }, | ||
| { | ||
| name: "empty systemId", | ||
| params: emptySystemParams, | ||
| }, | ||
| { | ||
| name: "missing path", | ||
| params: []byte(`{"systemId": "NES"}`), | ||
| }, | ||
| { | ||
| name: "empty path", | ||
| params: []byte(`{"systemId": "NES", "path": ""}`), | ||
| }, | ||
| { | ||
| name: "invalid JSON", | ||
| params: []byte(`not valid json`), | ||
| }, | ||
| { | ||
| name: "null params", | ||
| params: nil, | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range tests { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := requests.RequestEnv{ | ||
| Context: context.Background(), | ||
| Config: &config.Instance{}, | ||
| Params: tt.params, | ||
| } | ||
| _, err := HandleMediaTitleParse(env) | ||
| require.Error(t, err) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_SimpleGame(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "NES", | ||
| filepath.Join("roms", "nes", "Super Mario Bros.nes"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.Equal(t, "Super Mario Bros", resp.Name) | ||
| assert.NotEmpty(t, resp.Slug) | ||
| assert.Nil(t, resp.SecondarySlug) | ||
| assert.Equal(t, utf8.RuneCountInString(resp.Slug), resp.SlugLength) | ||
| assert.Positive(t, resp.SlugWordCount) | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_TitleWithColonProducesSecondarySlug(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "NES", | ||
| filepath.Join("roms", "nes", "The Legend of Zelda: Links Awakening.nes"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.NotEmpty(t, resp.Name) | ||
| assert.NotEmpty(t, resp.Slug) | ||
| require.NotNil(t, resp.SecondarySlug) | ||
| assert.NotEmpty(t, *resp.SecondarySlug) | ||
| assert.Equal(t, utf8.RuneCountInString(resp.Slug), resp.SlugLength) | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_TitleWithDashProducesSecondarySlug(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "SNES", | ||
| filepath.Join("roms", "snes", "Donkey Kong Country - Tropical Freeze.sfc"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.NotEmpty(t, resp.Name) | ||
| require.NotNil(t, resp.SecondarySlug) | ||
| assert.NotEmpty(t, *resp.SecondarySlug) | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_NoSubtitleHasNilSecondarySlug(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "NES", | ||
| filepath.Join("roms", "nes", "Tetris.nes"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.Equal(t, "Tetris", resp.Name) | ||
| assert.Nil(t, resp.SecondarySlug) | ||
| assert.Positive(t, resp.SlugLength) | ||
| assert.Positive(t, resp.SlugWordCount) | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_UnknownSystemFallsBack(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "UNKNOWN_SYSTEM_XYZ", | ||
| filepath.Join("roms", "misc", "Some Game.rom"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.Equal(t, "Some Game", resp.Name) | ||
| assert.NotEmpty(t, resp.Slug) | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_SlugLengthMatchesRuneCount(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| tests := []struct { | ||
| name string | ||
| systemID string | ||
| filename string | ||
| }{ | ||
| { | ||
| name: "NES multi-word", | ||
| systemID: "NES", | ||
| filename: "Mega Man 2.nes", | ||
| }, | ||
| { | ||
| name: "Genesis multi-word", | ||
| systemID: "Genesis", | ||
| filename: "Sonic the Hedgehog 2.md", | ||
| }, | ||
| { | ||
| name: "SNES single word", | ||
| systemID: "SNES", | ||
| filename: "Tetris.sfc", | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range tests { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, tt.systemID, | ||
| filepath.Join("roms", tt.filename), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok) | ||
| assert.Equal(t, utf8.RuneCountInString(resp.Slug), resp.SlugLength, | ||
| "SlugLength must equal rune count of Slug") | ||
| assert.Positive(t, resp.SlugWordCount) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func TestHandleMediaTitleParse_ResponseType(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| env := makeTitleParseEnv(t, "NES", | ||
| filepath.Join("roms", "nes", "Castlevania.nes"), | ||
| ) | ||
|
|
||
| result, err := HandleMediaTitleParse(env) | ||
| require.NoError(t, err) | ||
| require.NotNil(t, result) | ||
|
|
||
| _, ok := result.(models.MediaTitleParseResponse) | ||
| require.True(t, ok, "result must be MediaTitleParseResponse") | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Move
MediaTitleParseParamsabove method declarations.MediaTitleParseParamsis introduced afterReaderConnection.IsEnabled(Line 112). Please keep new type declarations before functions/methods in this file.As per coding guidelines, "Define Go types and consts near the top of the file, before functions and methods".
🤖 Prompt for AI Agents