Possible bug: Using index.searchCache when highlight option set produces inconsistent results

[stanislaw]

Hello @ts-thomas,

Thanks a lot for developing your open source project! I am trying to integrate it to the StrictDoc requirements and documentation tool to enable static HTML search for the generated documentation content.

I could adapt the autocomplete.html example to StrictDoc's environment (it works!) but during the testing of the integration I encountered something what I believe could be a bug:

If a data item contains a number such as in REQ-1, and I type in REQ in the search bar, only the RE part is highlighted but not the REQ. Note that this only affects the strings such REQ-1 as demonstrated with the following screenshot:

I am attaching an only slightly modified version of the autocomplete example that is provided on the README page. It should be enough to reproduce the issue quickly. The example works against the version 0.8.2 which is the latest version as of today.

autocomplete.html

My JS tooling skills are not very great. I tried to clone your repository but could not run tests due to some NPM dependency issues. I think I leave it to you as the maintainer. If the issue gets fixed, I am happy to provide more testing.

To give you some context: one application of your search library could be indexing a large set of software development standards, such as the European space (ECSS) standards. Having a fast search over such large documentation trees would be a very useful feature, and this is what I am trying to achieve by using your library.

Thanks,
Stanislav

[stanislaw]

cc @mettta

[stanislaw]

Here is another more realistic example. With this example, I would like to be able to match the entire ECSS-E-AS sequence, not just ECSS or ECS.

[ts-thomas]

This issue happens, when a string gets a different string length after encoding. I didn't came up with a nice solution to this yet. You can overcome this issue when disable dedupe within Encoder Options:

const encoder = new Encoder({ dedupe: false });

Or when creating an Index:

const index = new Document({
    document: {
        index: [{
            field: "title",
            tokenize: "forward",
            encoder: { dedupe: false }
        }]
    }
});