Utilize ready-to-use sentence segmentation tools in RFP #40
Description
Sentence segmentation proved to be one of the more difficult subtasks in the RepoFromPaper pipeline.
Using tools for sentence segmentation like the ones from Spacy could be improve our current custom sentence segmentation.
Check out:
https://spacy.io/api/sentencizer
https://ashutoshtripathi.com/2020/05/04/how-to-perform-sentence-segmentation-or-sentence-tokenization-using-spacy-nlp-series-part-5/