Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
74d088a
database changes
leoraba Jan 9, 2026
28f683a
update dbml
leoraba Jan 9, 2026
e1ada63
migration services
leoraba Jan 9, 2026
c1f87dd
updates migration repository
leoraba Jan 13, 2026
118fc42
Merge branch 'feat/dictionary_migration_db' into feat/migration_imple…
leoraba Jan 13, 2026
0a26540
update migration service
leoraba Jan 15, 2026
1ec4dc4
data validation
leoraba Jan 30, 2026
afc4f87
refactor migration process
leoraba Feb 3, 2026
58d4ad8
block commit submission
leoraba Feb 3, 2026
b081dd9
fix sort imports
leoraba Feb 3, 2026
e6dc14d
Merge branch 'feat/dictionary_migration' into feat/dictionary_migrati…
leoraba Feb 3, 2026
608c096
Merge branch 'feat/dictionary_migration_db' into feat/migration_imple…
leoraba Feb 3, 2026
b23d648
Merge branch 'feat/dictionary_migration' into feat/dictionary_migrati…
leoraba Apr 1, 2026
8688240
update docs
leoraba Apr 2, 2026
2487bbe
migration table index
leoraba Apr 7, 2026
96b6c63
Merge branch 'feat/dictionary_migration_db' into feat/migration_imple…
leoraba Apr 10, 2026
d351b2e
fix typos and logs
leoraba Apr 13, 2026
fb05263
migration audit and logs
leoraba Apr 13, 2026
b5d6756
migration on worker thread
leoraba Apr 14, 2026
f5af236
GET migration endpoints
leoraba Apr 15, 2026
d7d17d0
fix unit tests
leoraba Apr 15, 2026
2d05348
Merge branch 'feat/migration_implementation' into feat/get_migration_…
leoraba Apr 15, 2026
2758202
GET migration records
leoraba Apr 16, 2026
b8cd961
adding ts config noUncheckedIndexedAccess
leoraba Apr 17, 2026
cdf32cc
rename migration enum status IN_PROGRESS
leoraba Apr 17, 2026
2c889d1
Merge branch 'feat/dictionary_migration_db' into feat/migration_imple…
leoraba Apr 17, 2026
430ea27
Merge branch 'feat/migration_implementation' into feat/get_migration_…
leoraba Apr 17, 2026
c512349
Merge branch 'feat/dictionary_migration' into feat/migration_implemen…
leoraba Apr 27, 2026
f5994eb
refactoring migration function with Result
leoraba Apr 28, 2026
280395f
Merge branch 'feat/migration_implementation' into feat/get_migration_…
leoraba Apr 29, 2026
c682a47
default constants pagination
leoraba Apr 29, 2026
b41d1ad
change logs and response NotFound
leoraba Apr 29, 2026
6f231ce
fix swagger docs
leoraba Apr 29, 2026
4f7a01b
include errors or migration changes
leoraba Apr 30, 2026
57cf05b
type paginated result
leoraba Apr 30, 2026
8f69160
Merge branch 'feat/dictionary_migration' into feat/get_migration_endp…
leoraba Apr 30, 2026
a1c4d8f
using type paginated result
leoraba Apr 30, 2026
e588d0b
unit test migration formatter functions
leoraba Apr 30, 2026
d8b930d
remove unhandled error thrown
leoraba May 4, 2026
a7cfcf1
retry dictionary registration
leoraba May 4, 2026
a2b05e8
updating docs
leoraba May 4, 2026
f28c9aa
Merge branch 'feat/dictionary_migration' into feat/retry_failed_migra…
leoraba May 12, 2026
feb65f5
update documentation
leoraba May 29, 2026
ec4a580
DRY refactor
leoraba May 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions apps/server/swagger/dictionary-api.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,17 @@

/dictionary/register:
post:
summary: Register new dictionary
summary: Register a dictionary to a category. Creates the category if it does not exist, or updates its active dictionary and triggers a data migration to validate and update existing data against the new dictionary.
tags:
- Dictionary
parameters:
- name: force
description: Runs dictionary registration and migration again for this category, even if the same dictionary is already registered. Use this only when a previous migration ended unexpectedly and you must rerun both steps.
in: query
required: false
schema:
type: boolean
default: false
Comment on lines +9 to +15
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add more detail about this behaviour? I don't know what re-registering a dictionary will do. Is this the same as initiating a migration? If the dictionary version is the same as the current dictionary will it run the migration with the intention of revalidating the data (should have no impact on the current data state)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

description updated to Runs dictionary registration and migration again for this category, even if the same dictionary is already registered. Use this only when a previous migration ended unexpectedly and you must rerun both steps.

requestBody:
content:
application/json:
Expand Down Expand Up @@ -38,8 +46,8 @@
$ref: '#/components/responses/BadRequest'
401:
$ref: '#/components/responses/UnauthorizedError'
404:
$ref: '#/components/responses/NotFound'
409:
$ref: '#/components/responses/StatusConflict'
Comment on lines +49 to +50
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if registering the same Dictionary for the category then it throws a 409 error, unless the "force" flag is true.

500:
$ref: '#/components/responses/ServerError'
503:
Expand Down
2 changes: 2 additions & 0 deletions packages/data-provider/docs/dictionary-registration.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ A Category is uniquely identified by its case-sensitive `categoryName`.

Category groups data that is related and shares the same data structure, for that reason, a category must be associated to a registered dictionary. Over time, if the dictionary requires an update, the category needs to be updates accordingly, See [Dictionary Migration](#dictionary-migration) for more details.

If a category is already using the same dictionary name and version, registration returns `409 Conflict` by default. Set the `force` query parameter to `true` only when you need to override this and rerun registration for the same dictionary. This also triggers migration validation for existing category data, which is mainly useful when a previous registration or migration ended unexpectedly.

## Centric entity

Some dictionaries define a centric entity, representing the root of the data model hierarchy (used on compound views).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const controller = (dependencies: BaseDependencies) => {
const dictionaryName = req.body.dictionaryName;
const dictionaryVersion = req.body.dictionaryVersion;
const defaultCentricEntity = req.body.defaultCentricEntity;
const forceRegistration = req.query.force?.toLowerCase() === 'true';
const user = req.user;

logger.info(
Expand All @@ -35,6 +36,7 @@ const controller = (dependencies: BaseDependencies) => {
dictionaryVersion,
defaultCentricEntity,
username: user?.username,
forceRegistration,
});

logger.info(LOG_MODULE, `Register Dictionary completed!`);
Expand Down
60 changes: 49 additions & 11 deletions packages/data-provider/src/services/dictionaryService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { BaseDependencies } from '../config/config.js';
import lecternClient from '../external/lecternClient.js';
import categoryRepository from '../repository/categoryRepository.js';
import dictionaryRepository from '../repository/dictionaryRepository.js';
import { BadRequest } from '../utils/errors.js';
import { BadRequest, StatusConflict } from '../utils/errors.js';
import migrationService from './migrationService.js';

const dictionaryService = (dependencies: BaseDependencies) => {
Expand Down Expand Up @@ -99,12 +99,14 @@ const dictionaryService = (dependencies: BaseDependencies) => {
dictionaryVersion,
defaultCentricEntity,
username,
forceRegistration = false,
}: {
categoryName: string;
dictionaryName: string;
dictionaryVersion: string;
defaultCentricEntity?: string;
username?: string;
forceRegistration?: boolean;
}): Promise<{ dictionary: Dictionary; category: Category; migrationId?: number }> => {
logger.debug(
LOG_MODULE,
Expand All @@ -131,11 +133,54 @@ const dictionaryService = (dependencies: BaseDependencies) => {
// Check if Category exist
const foundCategory = await categoryRepo.getCategoryByName(categoryName);

const initiateMigrationOrThrow = async ({
categoryId,
fromDictionaryId,
toDictionaryId,
}: {
categoryId: number;
fromDictionaryId: number;
toDictionaryId: number;
}): Promise<number> => {
const resultMigration = await initiateMigration({
categoryId,
fromDictionaryId,
toDictionaryId,
userName: username || '',
});

if (!resultMigration.success) {
const errorMessage = `Failed to initiate migration for category '${categoryName}' with error: ${resultMigration.data}`;
logger.error(LOG_MODULE, errorMessage);
throw new Error(errorMessage);
}

return resultMigration.data;
};

if (foundCategory && foundCategory.activeDictionaryId === savedDictionary.id) {
// Dictionary and Category already exists
logger.info(LOG_MODULE, `Dictionary and Category already exists`);

return { dictionary: savedDictionary, category: foundCategory };
if (forceRegistration) {
logger.info(
LOG_MODULE,
`Force flag is true, initiating migration for Category '${foundCategory.name}'
with Dictionary '${savedDictionary.name}' version '${savedDictionary.version}'`,
);

const migrationId = await initiateMigrationOrThrow({
categoryId: foundCategory.id,
fromDictionaryId: foundCategory.activeDictionaryId,
toDictionaryId: savedDictionary.id,
});

return { dictionary: savedDictionary, category: foundCategory, migrationId };
}

throw new StatusConflict(
`Category '${categoryName}' with Dictionary '${savedDictionary.name}' version '${savedDictionary.version}' already exists`,
);
Comment on lines +165 to +183
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not very DRY, a lot of repeated code from the else block. The only difference is omitting the fromDictionaryId and changed log statements. Is there a way to write this that won't open us up to a future code change that is only made in one branch of this if statement?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored this code to make it DRY and reusing the initiateMigration logic.

} else if (foundCategory && foundCategory.activeDictionaryId !== savedDictionary.id) {
// Update the dictionary on existing Category
const updatedCategory = await categoryRepo.update(foundCategory.id, {
Expand All @@ -144,25 +189,18 @@ const dictionaryService = (dependencies: BaseDependencies) => {
updatedBy: username,
});

const resultMigration = await initiateMigration({
const migrationId = await initiateMigrationOrThrow({
categoryId: updatedCategory.id,
fromDictionaryId: foundCategory.activeDictionaryId,
toDictionaryId: savedDictionary.id,
userName: username || '',
});

if (!resultMigration.success) {
const errorMessage = `Failed to initiate migration for category '${categoryName}' with error: ${resultMigration.data}`;
logger.error(LOG_MODULE, errorMessage);
throw new Error(errorMessage);
}

logger.info(
LOG_MODULE,
`Category '${updatedCategory.name}' updated successfully with Dictionary '${savedDictionary.name}' version '${savedDictionary.version}'`,
);

return { dictionary: savedDictionary, category: updatedCategory, migrationId: resultMigration.data };
return { dictionary: savedDictionary, category: updatedCategory, migrationId };
} else {
// Create a new Category
const newCategory: NewCategory = {
Expand Down
9 changes: 8 additions & 1 deletion packages/data-provider/src/utils/schemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,9 +234,13 @@ export interface DictionaryRegisterBodyParams {
defaultCentricEntity?: string;
}

export interface DictionaryRegisterQueryParams extends ParsedQs {
force?: string;
}

export const dictionaryRegisterRequestSchema: RequestValidation<
DictionaryRegisterBodyParams,
ParsedQs,
DictionaryRegisterQueryParams,
ParamsDictionary
> = {
body: z.object({
Expand All @@ -245,6 +249,9 @@ export const dictionaryRegisterRequestSchema: RequestValidation<
dictionaryVersion: stringNotEmpty,
defaultCentricEntity: entityNameSchema.or(z.literal('')).optional(),
}),
query: z.object({
force: booleanSchema.default('false'),
}),
};

// Migration Requests
Expand Down