An Important question

[sz17320828912-tech]

Hello!

I recently noticed your work and am in the process of reproducing your paper. I have a question I'd like to ask you.

I noticed that your code output has multiple scores. I would like to know which score you ultimately used for the data presented in your final paper?

The following is the output from running the code:

The following is the data table from your paper:

[HUST-AI-HYZ]

Hi. Thanks for your interest and following on our research work.

You can refer the Appendix A "A Details of Dataset" in our paper to get the details of metrics used in each sub-task. For example, the tasks SH-QA and MH-QA in Accurate Retrieval use the SubEM (substring_exact_match) metric for evaluation since most answers in these tasks are mainly short entities.

[sz17320828912-tech]

Hi. Thanks for your interest and following on our research work.

You can refer the Appendix A "A Details of Dataset" in our paper to get the details of metrics used in each sub-task. For example, the tasks SH-QA and MH-QA in Accurate Retrieval use the SubEM (substring_exact_match) metric for evaluation since most answers in these tasks are mainly short entities.

Thank you for your response, I have a new question for you:

In your paper, specifically in Table 9 in Appendix D, what is the chunk-size used for SH-Doc QA? Is it 512 or 1024?

[HUST-AI-HYZ]

Chunk size is 512 in your mentioned case.