Skip to content

listenbrainz: add option to import listens from ListenBrainz data export zip#6606

Open
Maxr1998 wants to merge 12 commits into
beetbox:masterfrom
Maxr1998:lbimport-file
Open

listenbrainz: add option to import listens from ListenBrainz data export zip#6606
Maxr1998 wants to merge 12 commits into
beetbox:masterfrom
Maxr1998:lbimport-file

Conversation

@Maxr1998
Copy link
Copy Markdown
Contributor

@Maxr1998 Maxr1998 commented May 3, 2026

Description

Since importing listens from the API is rather slow and puts unnecessary load on the servers, importing a dump exported from https://listenbrainz.org/settings/export/ should also be possible. This change introduces the export-file parameter to the lbimport command, which takes all listens from the zipfile to update the play counts.

I also fixed an issue where mbid_mapping could be None if explicitly set to null in the JSON (which happens for these exports sometimes), and changed the recording mbid logic to prefer one explicitly supplied by the player in the additional_data, instead of relying on the (sometimes inaccurate) mapper.

Since the --max parameter isn't documented either, I'm unsure whether I should add the parameter to the plugin docs. In theory, the --help parameter already tells you enough.

I'm also not sure if the --max parameter should apply to exported files, especially since the order of zip file entries isn't deterministic.

To Do

  • Documentation. (If you've added a new command-line flag, for example, find the appropriate page under docs/ to describe it.)
  • Changelog. (Add an entry to docs/changelog.rst to the bottom of one of the lists near the top of the document.)
  • Tests. (Very much encouraged but not strictly required.)

Copilot AI review requested due to automatic review settings May 3, 2026 11:09
@Maxr1998 Maxr1998 requested a review from a team as a code owner May 3, 2026 11:09
@github-actions github-actions Bot added the listenbrainz listenbrainz plugin label May 3, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 3, 2026

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

grug see PR try make lbimport faster and nicer to ListenBrainz server. it add way to read listens from ListenBrainz export zip, then update play counts in beets.

Changes:

  • add lbimport --export-file PATH to read listens from ListenBrainz data export zip instead of API
  • add zip/jsonl import helper and wire it into _lbupdate
  • tweak MBID pick logic to handle mbid_mapping: null and prefer explicit MBID when present

Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py
Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 3, 2026

Codecov Report

❌ Patch coverage is 70.14925% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.49%. Comparing base (6dd6251) to head (ef39cd2).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
beetsplug/listenbrainz.py 70.14% 19 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6606      +/-   ##
==========================================
+ Coverage   72.46%   72.49%   +0.02%     
==========================================
  Files         161      161              
  Lines       20719    20766      +47     
  Branches     3280     3288       +8     
==========================================
+ Hits        15014    15054      +40     
- Misses       4979     4986       +7     
  Partials      726      726              
Files with missing lines Coverage Δ
beetsplug/listenbrainz.py 60.26% <70.14%> (+6.41%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Serene-Arc
Copy link
Copy Markdown
Contributor

Thanks for the PR! I'm of the opinion that more documentation is always better. If there's an option that isn't currently documented, then please do add documentation for it, either in this PR or another one.

As for the --max option I agree that it shouldn't apply to imported files. From the code, it seems to be an API-limiting option which doesn't really apply to when you're importing offline data. Further, I would expect someone who has downloaded that data to either filter the new dataset themselves or else want to import the whole thing, especially if the process isn't deterministic.

grug is correct that this interaction should be noted in the documentation though.

@Maxr1998
Copy link
Copy Markdown
Contributor Author

@Serene-Arc thanks, I made some further adjustments, added some documentation, tests, and a changelog entry. Should be ready to review now.

Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py Outdated
Comment thread beetsplug/listenbrainz.py Outdated
@Maxr1998
Copy link
Copy Markdown
Contributor Author

Made the requested changes in 1e089ec.

@Maxr1998 Maxr1998 requested a review from snejus May 21, 2026 21:03
@snejus
Copy link
Copy Markdown
Member

snejus commented May 21, 2026

@Maxr1998 See test failures.

Comment thread beetsplug/listenbrainz.py Outdated
self._log.debug("Invalid Search Error: {}", e)
return None

def import_listenbrainz_data_export(self, export_file: str | Path):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add a return type here for clarity as well please!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Let me know if you want me to squash the commits before you merge.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries about that. Would you mind defining a specific TypedDict type for those listens?

Copy link
Copy Markdown
Contributor Author

@Maxr1998 Maxr1998 May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I hope all the checks pass cleanly. I can also extract the types to a separate file like the Deezer plugin does, but that's definitely a separate PR.

I also feel like the recording_mbid behavior change should be a separate PR. Please tell me if you agree, then I will extract that part first. (Alternatively, I could reorder and squash my commits so that they stack cleanly when you do a normal merge commit or rebase merge.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

listenbrainz listenbrainz plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants