listenbrainz: add option to import listens from ListenBrainz data export zip#6606
listenbrainz: add option to import listens from ListenBrainz data export zip#6606Maxr1998 wants to merge 12 commits into
Conversation
|
Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry. |
There was a problem hiding this comment.
Pull request overview
grug see PR try make lbimport faster and nicer to ListenBrainz server. it add way to read listens from ListenBrainz export zip, then update play counts in beets.
Changes:
- add
lbimport --export-file PATHto read listens from ListenBrainz data export zip instead of API - add zip/jsonl import helper and wire it into
_lbupdate - tweak MBID pick logic to handle
mbid_mapping: nulland prefer explicit MBID when present
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6606 +/- ##
==========================================
+ Coverage 72.46% 72.49% +0.02%
==========================================
Files 161 161
Lines 20719 20766 +47
Branches 3280 3288 +8
==========================================
+ Hits 15014 15054 +40
- Misses 4979 4986 +7
Partials 726 726
🚀 New features to boost your workflow:
|
|
Thanks for the PR! I'm of the opinion that more documentation is always better. If there's an option that isn't currently documented, then please do add documentation for it, either in this PR or another one. As for the grug is correct that this interaction should be noted in the documentation though. |
|
@Serene-Arc thanks, I made some further adjustments, added some documentation, tests, and a changelog entry. Should be ready to review now. |
|
Made the requested changes in 1e089ec. |
|
@Maxr1998 See test failures. |
| self._log.debug("Invalid Search Error: {}", e) | ||
| return None | ||
|
|
||
| def import_listenbrainz_data_export(self, export_file: str | Path): |
There was a problem hiding this comment.
Just add a return type here for clarity as well please!
There was a problem hiding this comment.
Done. Let me know if you want me to squash the commits before you merge.
There was a problem hiding this comment.
No worries about that. Would you mind defining a specific TypedDict type for those listens?
There was a problem hiding this comment.
Done. I hope all the checks pass cleanly. I can also extract the types to a separate file like the Deezer plugin does, but that's definitely a separate PR.
I also feel like the recording_mbid behavior change should be a separate PR. Please tell me if you agree, then I will extract that part first. (Alternatively, I could reorder and squash my commits so that they stack cleanly when you do a normal merge commit or rebase merge.)
Description
Since importing listens from the API is rather slow and puts unnecessary load on the servers, importing a dump exported from https://listenbrainz.org/settings/export/ should also be possible. This change introduces the
export-fileparameter to thelbimportcommand, which takes all listens from the zipfile to update the play counts.I also fixed an issue where
mbid_mappingcould beNoneif explicitly set tonullin the JSON (which happens for these exports sometimes), and changed the recording mbid logic to prefer one explicitly supplied by the player in theadditional_data, instead of relying on the (sometimes inaccurate) mapper.Since the
--maxparameter isn't documented either, I'm unsure whether I should add the parameter to the plugin docs. In theory, the--helpparameter already tells you enough.I'm also not sure if the
--maxparameter should apply to exported files, especially since the order of zip file entries isn't deterministic.To Do
docs/to describe it.)docs/changelog.rstto the bottom of one of the lists near the top of the document.)