Custom clustering#66
Open
whalebot-helmsman wants to merge 449 commits intoscrapinghub:masterfrom
whalebot-helmsman:custom_clustering
Open
Custom clustering#66whalebot-helmsman wants to merge 449 commits intoscrapinghub:masterfrom whalebot-helmsman:custom_clustering
whalebot-helmsman wants to merge 449 commits intoscrapinghub:masterfrom
whalebot-helmsman:custom_clustering
Conversation
It is better to put models somewhere else, and notebooks were broken.
add base classifier and global ngrams feature functions
1. rename DEFAULT_TAGSET to EXAMPLE_TAGSET; 2. rename DEFAULT_FEATURES to EXAMPLE_TOKEN_FEATURES; 3. make token_features empty by default in create_wapiti_pipeline.
except model_filename must be kwargs now. Also, this fixes the example from the tutorial.
…v/webstruct into speed_up_text_tokenizer
Speed up text tokenizer
Wapiti for webstruct
Ignore all non-text tags
Non-recursive implementation of algorithm
fix boolean bug
update travis to run different python versions
kmike
reviewed
Aug 27, 2018
| score_func=None, | ||
| score_kwargs=None, | ||
| get_position_func=None, | ||
| get_distance_func=None): |
Member
There was a problem hiding this comment.
Could you please document these two new parameters?
Codecov Report
@@ Coverage Diff @@
## master #66 +/- ##
=========================================
- Coverage 81.02% 80.43% -0.6%
=========================================
Files 40 40
Lines 2092 2131 +39
=========================================
+ Hits 1695 1714 +19
- Misses 397 417 +20 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR:
default_clustering_scoreto reduce score for clustering withthreshold=0_get_tree_positionand_get_tree_distancefunctions provide much better grouping resultsI didn't set these new functions as default functions. Using them as default will improve grouping quality.