UoE/Version 8.2#2
Open
milanmajchrak wants to merge 46 commits into
Open
Conversation
The DatashareDataset JPA entity maps to a 'dataset' table but no Flyway
migration existed to create it. This caused Hibernate 6 schema validation
to fail ('Schema-validation: missing table [dataset]'), which prevented
the SessionFactory from being created, resulting in 59 unit test failures.
Creates the dataset table in both H2 (for tests) and PostgreSQL (for
production) with appropriate foreign keys to dspaceobject and item tables.
The tests hardcoded 'local*' and '*host' patterns assuming MockWebServer hostname is always 'localhost'. On some JDK/OS combinations the hostname may differ (e.g. '127.0.0.1'). Use mockServer.getHostName() to derive wildcard patterns dynamically.
- Add DSpace license headers to 13 DataShare custom files - Add checkstyle suppressions for DataShare custom code and modified upstream files (1893 pre-existing violations) - Remove unused imports in StatelessAuthenticationFilter
DataShare intentionally commented out searchFilterIssued from all discovery configurations but did not update the upstream MetadataExportSearchIT test that depends on the dateIssued filter. Re-enable it in the searchFilters property (not sidebarFacets) to allow search/CLI queries by dateIssued while keeping the sidebar unchanged.
…wing DatashareDatasetServiceImpl.find(Context, UUID) was throwing UnsupportedOperationException, which broke DSpaceObjectUtilsImpl.findDSpaceObject() when iterating all DSpaceObjectService implementations to resolve a UUID. This caused MetadataExportIT.metadataExportToCsvTest_NonValidIdentifier to fail because the UnsupportedOperationException was caught by run()'s handler instead of the expected IllegalArgumentException from handleExport().
getSupportsTypeConstant() is called during ContentServiceFactory iteration to find the service matching a DSO type. Throwing UnsupportedOperationException breaks any code that calls getDSpaceObjectService(type). Also implement getName(), findByIdOrLegacyId(), findByLegacyId() to return safe defaults instead of throwing.
DataShare had commented out searchFilterIssued in ALL discovery configurations (sidebarFacets and searchFilters). This caused DiscoveryRestControllerIT, DiscoveryScopeBasedRestControllerIT, BrowsesResourceControllerIT, and OpenSearchControllerIT failures because tests expect dateIssued in both searchFilters and sidebarFacets. Uncomments all 21 remaining searchFilterIssued references across defaultConfiguration, homepageConfiguration, workspaceConfiguration, workflowConfiguration, supervisedConfiguration, dateIssued sort, and all 5 entity-specific configurations.
DataShare customises the default discovery configuration with extra search filters (dateAccessioned, dateEmbargo), sidebar facets, and sort fields. The upstream integration tests (DiscoveryRestControllerIT, DiscoveryScopeBasedRestControllerIT) expected only the standard set. Changes: - SearchFilterMatcher: add dateAccessionedFilter() and dateEmbargoFilter() - DiscoveryRestControllerIT: populate customSearchFilters and customSidebarFacets with DataShare filter/facet matchers; fix sort expectation to include dateEmbargo sort - DiscoveryScopeBasedRestControllerIT: add dateAccessioned and dateEmbargo facets to default-config fallback expectations - discovery.xml: uncomment sortDateIssued in all configurations (12 references) so tests expecting dateIssued sort still pass
… NPE - Comment out duplicate first-set browse indexes and sort options in dspace.cfg (DSpace reads first value for same-keyed properties, making DataShare overrides ineffective) - Add dateissued browse index and sort option to DataShare config section - Add dateAccessionedBrowseIndex and subjectClassificationBrowseIndex matchers - Update sort option expectations to include dateembargo - Fix subjectBrowseIndex metadata from dc.subject.* to dc.subject - Update findAll to expect 6 browse indexes - Change findBrowseByVocabulary to expect 404 (srsc disabled) - Fix NPE in ChoiceAuthorityServiceImpl.getVocabularyIndex when vocabulary has no submission form definitions (formsToFields null check)
…Share config - Add dateAccessioned and dateEmbargo facets to workspace, workflow, workflowAdmin, and supervision test assertions - Change discoverSearchByFieldNotConfiguredTest to use dc.date.available instead of dc.date.accessioned (now configured as a valid sort field) - Update supervision test facets[4] index to facets[6] for supervisedBy - Add jacs vocabulary to VocabularyRestRepositoryIT.findAllTest (DataShare added jacs to submission-forms.xml, totalElements 6 -> 7)
v1.3.0 internally uses actions/upload-artifact@v3 which is now deprecated and automatically rejected by GitHub Actions runners.
Remove DataShare-specific checkstyle suppressions and fix all 1899 violations properly: - Replace tabs with 4-space indentation - Fix import ordering (jakarta/java before org/com) - Wrap lines exceeding 120-char limit - Remove trailing whitespace - Fix brace placement and whitespace around operators - Add comments to empty catch blocks - Remove unused imports All 14 modules now pass checkstyle with 0 violations.
…/google.com The testURLRedirectCurateTest was using https://demo7.dspace.org/handle/123456789/1 as a redirect URL, but the DSpace 7 demo server is unreliable/offline. Replace with http://google.com which reliably redirects to https://google.com, matching the fix already applied in upstream DSpace.
The codecov job always fails in forks that don't have the CODECOV_TOKEN secret set. Skip the job entirely when the token is not available to avoid blocking the workflow.
The secrets context is not available in job-level if conditions. Use continue-on-error: true to prevent codecov upload failures from blocking the overall workflow status.
…e config - Revert dspace.cfg to target branch (no customer config changes) - Revert discovery.xml to target branch, then add minimal sortDateIssued + searchFilterIssued in defaultConfiguration only (upstream test compat) - Remove CI-FIX-SUMMARY.md from repository - Adapt BrowsesResourceControllerIT for DataShare browse indices: dateissued browse returns 404, use title browse instead - Adapt BrowseIndexMatcher sort options to match effective config (first-wins: title, dateissued, dateaccessioned from upstream) - Fix DiscoveryRestControllerIT pool task sort (dc.date.accessioned) and workspace/workflow/supervision facet assertions
…migration to V8.3 - Revert production discovery.xml: comment out searchFilterIssued (2 places) and sortDateIssued (1 place) in defaultConfiguration bean - Add defaultConfiguration bean override in test-discovery.xml with searchFilterIssued and sortDateIssued active for upstream test compatibility - Rename dataset migration scripts from V8.0_2025 to V8.3_2026 to match DSpace 8.x versioning convention
UoE/Fix github action issues
UoE/feat: add bitstream accessStatus endpoint with embargoDate
…alk bugs UoE/Removed ds schema; fix temporal coverage step and DataCite crosswalk bugs
…v/uoe-dspace-datashare-backend into uoe-fix-github-actions-it-2
…s status endpoint
UoE/fix: Github actions - it2
There was a problem hiding this comment.
Pull request overview
This PR updates the DSpace/DataShare configuration and REST/API behavior to align DataShare submission metadata with standard dc.* fields, expand discovery/browse capabilities, and introduce dataset/access-status related features.
Changes:
- Update DataShare submission forms and processing steps to write/read standard
dc.*metadata (including temporal coverage hydration/encoding and license/rights handling). - Extend REST API access-status support (now including embargo date) and add a Bitstream
accessStatuslink endpoint; update associated REST integration tests/matchers. - Add DB migrations and service/DAO updates for a DataShare
datasetentity/table, and adjust discovery/browse configuration + tests for DataShare defaults.
Reviewed changes
Copilot reviewed 50 out of 56 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| dspace/config/submission-forms.xml | Re-maps DataShare submission fields to dc.* metadata; updates license, funder, and temporal coverage fields/value-pairs. |
| dspace/config/spring/api/discovery.xml | Adjusts discovery sort options (comments out sortDateIssued in the shown section). |
| dspace/config/crosswalks/DIM2DataCite.xsl | Updates DOI handling and enhances Rights mapping (adds SPDX/rightsURI for CC-BY-4.0). |
| dspace-server-webapp/src/test/java/org/dspace/curate/CurationScriptIT.java | Modifies link-checking curation test inputs/expectations (redirect behavior). |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/submit/step/datashare/DatashareSpatialAndTemporalStepTest.java | Adds unit tests for encoding/decoding temporal coverage logic. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/SearchFilterMatcher.java | Adds matchers for new date-based discovery search filters. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BrowseIndexMatcher.java | Updates browse index expectations and adds matchers for DataShare-specific browse indexes. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BitstreamMatcher.java | Updates bitstream matcher expectations to include the new accessStatus link/embed. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/VocabularyRestRepositoryIT.java | Updates vocabulary list tests to include jacs and adjusts expected totals. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java | Updates discovery facet expectations to include new date facets. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/DiscoveryRestControllerIT.java | Updates discovery tests for DataShare facets/filters/sort fields and configuration behavior. |
| dspace-server-webapp/src/test/java/org/dspace/app/rest/BrowsesResourceControllerIT.java | Updates browse endpoint tests to match DataShare browse configuration (404s, title browse, etc.). |
| dspace-server-webapp/src/test/data/dspaceFolder/config/spring/api/test-discovery.xml | Overrides discovery config for tests to re-enable upstream-expected issued filter/sort. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/submit/step/datashare/DatashareSpatialAndTemporalStep.java | Adds temporal hydration/encoding logic and exposes encode/decode helpers for tests. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/submit/step/datashare/DatashareLicenseStep.java | Simplifies license handling to rely on dc.rights and manage CC license bundle accordingly. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/submit/step/datashare/DatashareFunderStep.java | Removes prior ds-to-dc metadata translation logic in PATCH processing. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/security/StatelessAuthenticationFilter.java | Removes unused authentication imports. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/repository/ItemAccessStatusLinkRepository.java | Adds embargo date to Item access-status REST payload. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/repository/BitstreamAccessStatusLinkRepository.java | Introduces Bitstream access-status REST link repository endpoint. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/model/BitstreamRest.java | Adds accessStatus link definition and constant to Bitstream REST model. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/model/AccessStatusRest.java | Adds embargoDate field to AccessStatus REST model. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/datashare/DatashareDatasetRestController.java | Adds/normalizes license header and servlet import ordering. |
| dspace-api/src/test/java/org/dspace/statistics/export/ITIrusExportUsageEventListener.java | Fixes JUnit import ordering. |
| dspace-api/src/test/java/org/dspace/content/datashare/DatashareItemDatasetTest.java | Adds standard license header. |
| dspace-api/src/test/java/org/dspace/app/client/DSpaceHttpClientFactoryTest.java | Makes proxy-ignore tests use the runtime mock server hostname patterns. |
| dspace-api/src/main/resources/org/dspace/storage/rdbms/sqlmigration/postgres/V8.3_2026.04.12__create_dataset_table.sql | Adds Postgres migration creating dataset table for DataShare dataset entity. |
| dspace-api/src/main/resources/org/dspace/storage/rdbms/sqlmigration/postgres/V8.0_2025.07.20__drop-datashare-ds-schema.sql | Adds migration to remove the intermediate ds metadata schema and its values/registry entries. |
| dspace-api/src/main/resources/org/dspace/storage/rdbms/sqlmigration/postgres/V8.0_2025.07.11__metadata-inserts_for_datashare-coverage-dates.sql | Registers dc.coverage.startDate and dc.coverage.endDate metadata fields. |
| dspace-api/src/main/resources/org/dspace/storage/rdbms/sqlmigration/h2/V8.3_2026.04.12__create_dataset_table.sql | Adds H2 migration creating dataset table for tests/dev environments. |
| dspace-api/src/main/java/uk/ac/ed/datashare/event/DatashareEvent.java | Adds standard license header. |
| dspace-api/src/main/java/uk/ac/ed/datashare/event/DatashareConsumer.java | Re-formats consumer and adds standard license header. |
| dspace-api/src/main/java/uk/ac/ed/datashare/commands/DatashareDoiCitationUpdaterCLI.java | Re-formats CLI and adds standard license header. |
| dspace-api/src/main/java/org/dspace/identifier/doi/DataCiteConnector.java | Formatting fixes in exception handling blocks. |
| dspace-api/src/main/java/org/dspace/identifier/doi/DOIOrganiser.java | Minor comment whitespace/formatting adjustments. |
| dspace-api/src/main/java/org/dspace/embargo/service/EmbargoService.java | Re-indents DataShare-specific checkForExpiry method declaration/comment. |
| dspace-api/src/main/java/org/dspace/embargo/EmbargoServiceImpl.java | Fixes import ordering for MessagingException. |
| dspace-api/src/main/java/org/dspace/embargo/DefaultEmbargoSetter.java | Cleans up commented code and adjusts log output formatting for null-safe policy reporting. |
| dspace-api/src/main/java/org/dspace/core/Constants.java | Whitespace cleanup around DataShare constants. |
| dspace-api/src/main/java/org/dspace/content/datashare/service/impl/DatashareDatasetServiceImpl.java | Adds license header, formats logic, and updates some interface methods’ return behaviors/constants. |
| dspace-api/src/main/java/org/dspace/content/datashare/service/UUN2EmailService.java | Adds standard license header. |
| dspace-api/src/main/java/org/dspace/content/datashare/service/DatashareDatasetService.java | Adds license header and reformats interface declaration. |
| dspace-api/src/main/java/org/dspace/content/datashare/service/BatchImportService.java | Adds standard license header. |
| dspace-api/src/main/java/org/dspace/content/datashare/dao/impl/DatashareDatasetDAOImpl.java | Import/format cleanup; contains “latest dataset” query logic. |
| dspace-api/src/main/java/org/dspace/content/datashare/dao/DatashareDatasetDAO.java | Adds license header and reformats interface methods. |
| dspace-api/src/main/java/org/dspace/content/datashare/DatashareItemDataset.java | Large reformat; maintains dataset ZIP creation/availability logic. |
| dspace-api/src/main/java/org/dspace/content/datashare/DatashareDataset.java | Adds license header and fixes import ordering. |
| dspace-api/src/main/java/org/dspace/content/authority/ChoiceAuthorityServiceImpl.java | Adds null-guard when retrieving form definitions for vocab index creation. |
| dspace-api/src/main/java/org/dspace/app/requestitem/RequestItemEmailNotifier.java | Minor whitespace cleanup in Javadoc. |
| dspace-api/src/main/java/org/dspace/app/itemupdate/ItemUpdate.java | Import ordering and formatting; retains DataShare dataset deletion hook on bitstream update actions. |
| dspace-api/src/main/java/org/dspace/access/status/service/AccessStatusService.java | Extends API to support Bitstream access status and embargo retrieval. |
| dspace-api/src/main/java/org/dspace/access/status/DefaultAccessStatusHelper.java | Implements Bitstream access-status/embargo methods. |
| dspace-api/src/main/java/org/dspace/access/status/AccessStatusServiceImpl.java | Delegates Bitstream methods to helper plugin. |
| dspace-api/src/main/java/org/dspace/access/status/AccessStatusHelper.java | Extends helper plugin interface with Bitstream methods. |
| .gitignore | Ignores scripts/ directory. |
| .github/workflows/codescan.yml | Updates CodeQL GitHub Action from v2 to v3. |
| .github/workflows/build.yml | Removes coverage profile and Codecov upload workflow logic; simplifies Maven invocation. |
Comments suppressed due to low confidence (2)
dspace/config/submission-forms.xml:533
- The Funder dropdown and free-text onebox both write directly to dc.contributor.other. Since the funding_bodies value-pairs includes an option stored as "Other", selecting it will persist the literal value "Other" into dc.contributor.other alongside any real funder text. Consider changing the value-pairs so "Other" isn’t stored (e.g., empty stored-value) and/or using a separate metadata field for the free-text value to avoid polluting funder metadata.
dspace-api/src/main/java/org/dspace/content/datashare/dao/impl/DatashareDatasetDAOImpl.java:72 - This JPQL uses ddset.id / d.id, which refers to the entity identifier (DSpaceObject.uuid) rather than the integer legacyId column. Because UUIDs are not ordered by creation, this query won’t reliably return the “latest” dataset for an item. If the intent is to use the numeric legacy id, query on ddset.legacyId (and ensure it’s generated), or order by a creation timestamp/sequence and limit to 1.
public DatashareDataset findLatestDatashareDatasetByItem(Context context, Item item) throws SQLException {
String queryString = "SELECT ddset FROM DatashareDataset ddset WHERE ddset.item = :item"
+ " AND ddset.id = (SELECT MAX(d.id) FROM DatashareDataset d WHERE d.item = :item)";
Query query = createQuery(context, queryString);
query.setParameter("item", item);
log.info("queryString: " + queryString);
return (DatashareDataset) uniqueResult(query);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR with all fixed must have issues.