Prefetch rest responses

[AAwouters]

References

• Fixes Reduce request chaining and associated overhead #3961
• Contains the changes from Cache-bust dynamic configuration files and theme CSS #3993
• 8_x backport [Port dspace-8_x] Prefetch rest responses #4082
• 7_x backport [Port dspace-7_x] Prefetch rest responses #4083

Description

Prefetch & store REST responses for configured endpoints/urls to reduce request chaining and associated overhead.

How it works

On (Angular) server start, and periodically, a request is sent to every configured endpoint/url.
The responses & headers are stored in the config for later use.

During initialization of either the server or the browser a findByHref call is done for all configured endpoints.
The DspaceRestService will retrieve the responses from the config instead of making actual requests to the REST server during these calls.
By using the findByHref method the responses are then added to the cache.
On the browser side the stored responses are then cleared from the config so future requests are actually sent to the REST server.

Gains

The biggest gains with these changes should be with CSR on slow connections.
The images below show that CSR initial load is almost halved.

(The 'pointless gap' mentioned on the image will be tackled with issue #4074)

Instructions for Reviewers

In config.prod.yml (or a different configuration file used for your local prod environment) configure the following values:

prefetch:
  urls:
    - /api
    - /api/statistics
    - /api/discover
    - /api/discover/search
    - /api/config/properties/google.analytics.key
    - /api/config/properties/registration.verification.enabled
    - /api/config/properties/websvc.opensearch.enable
    - /api/config/properties/websvc.opensearch.svccontext
    - /api/discover/browses?size=9999
    - /api/core/sites
    # - {other frequently accessed REST urls you might want to add} 
  # refreshInterval: 60000 # Uncomment & change to refresh more/less often.

Then start the Angular server in prod mode and use DSpace as you normally would.

For the best effect, the configured urls should be endpoints that Angular will retrieve on every load or very often.

Points of discussion

Prefetch interval

At the moment, the default interval to refresh the stored responses is 60 seconds. Is this appropriate?

Storage of bootstrapped responses

The bootstrapped responses at this moment are stored in the config.json file for simplicity. Should this be moved to a separate JSON file?

Checklist

• My PR is created against the main branch of code (unless it is a backport or is fixing an issue specific to an older branch).
• My PR is small in size (e.g. less than 1,000 lines of code, not including comments & specs/tests), or I have provided reasons as to why that's not possible.
• My PR passes ESLint validation using npm run lint
• My PR doesn't introduce circular dependencies (verified via npm run check-circ-deps)
• My PR includes TypeDoc comments for all new (or modified) public methods and classes. It also includes TypeDoc for large or complex private methods.
• My PR passes all specs/tests and includes new/updated specs or tests based on the Code Testing Guide.
• My PR aligns with Accessibility guidelines if it makes changes to the user interface.
• My PR uses i18n (internationalization) keys instead of hardcoded English text, to allow for translations.
• My PR includes details on how to test it. I've provided clear instructions to reviewers on how to successfully test this fix or feature.
• If my PR includes new libraries/dependencies (in package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
• If my PR includes new features or configurations, I've provided basic technical documentation in the PR itself.
• If my PR fixes an issue ticket, I've linked them together.

[github-actions]

Hi @AAwouters,
Conflicts have been detected against the base branch.
Please resolve these conflicts as soon as you can. Thanks!