Add dataset #1
Add dataset #1
Conversation
ProFatXuanAll
left a comment
ProFatXuanAll
left a comment
There was a problem hiding this comment.
Need to add a dataset class, see s2s.dset for examples.
| df['tgt'].apply(str).apply(self.__class__.preprocess).to_list() | ||
| ) | ||
|
|
||
| @staticmethod |
There was a problem hiding this comment.
For translation, one should use evaluation metrics other than exact match (i.e., generate words must "be the same word" and "in the same order" of target sequence , which is used by accuracy_score).
Why is exact match bad?
Consider the example translation pair I like apple and 我喜歡蘋果, 蘋果我喜歡 should also be an acceptable answer.
Thus, one should use some evaluation metrics with fozzy match, i.e., it's okay to not be that accurate but at the same time have same meaning (swapping order, synonym, etc.).
Nowadays people mostly use BLEU score as evaluation metric on translation task.
Go find some python package which calculate BLEU score for you.
| from s2s.dset._base import BaseDset | ||
| from s2s.path import DATA_PATH | ||
|
|
||
| class Eng2ChiDset(BaseDset): |
There was a problem hiding this comment.
Add docstring for Eng2ChiDset class, including reference of the source of the dataset.
Pipfile
Outdated
| tqdm = "4.49.0" | ||
| sklearn = "0.0" | ||
| tensorboard = "2.3.0" | ||
| nltk = "*" |
There was a problem hiding this comment.
Specify dependency version
run_eval_model.py
Outdated
| batch_tgt=dset.all_tgt(), | ||
| batch_pred=all_pred, | ||
| )) | ||
| print(DSET_OPTS[args.dset_name].bleu_score( |
There was a problem hiding this comment.
This method only work on translation dataset but not arithmetic dataset.
Overload batch_eval can do the trick.
s2s/dset/_eng2chi.py
Outdated
| from s2s.dset._base import BaseDset | ||
| from s2s.path import DATA_PATH | ||
|
|
||
| try: |
There was a problem hiding this comment.
User need to install dependency before running the code.
So you don't need to check whether a dependecies is missing or not.
Overload batch_eval function to count BLEU score Add docstring to _eng2chi.py
Accidentally changed the file run_train_tknzr.py
2 participants
add english to chinese dataset