Skip to content

Conversation

@philipmat
Copy link
Owner

@philipmat philipmat commented Sep 13, 2020

File Record Count Python C# v1 C# v2
discogs_20200806_artists.xml.gz 7,046,615 6:22 2:35 0:28 - 13x / 5x
discogs_20200806_labels.xml.gz 1,571,873 1:15 0:22 0:05 - 15x / 4x
discogs_20200806_masters.xml.gz 1,734,371 3:56 1:57 0:31 - 7x / 4x
discogs_20200806_releases.xml.gz 12,867,980 1:45:16 42:38 15:17 - 7x / 3x

That's just a part of the story.
Parallel, processing is the other.
Screen Shot 2020-09-13 at 11 36 31 AM

As the screenshot above shows, the .NET version allows processing multiple files in parallel, achieving a 20x speedup from the Python version in this case.

in part it's done to have a more accurate parser
# Conflicts:
#	alternatives/dotnet/discogs/Program.cs
@philipmat philipmat added this to the v3.0 - .NET Parser is main milestone Sep 13, 2020
@MuleaneEve
Copy link

Very cool to go from a process that used to take almost 2 hours to barely 15 minutes :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants