Application configuration is handled via a YAML file validated with Pydantic. You can find the default settings in application.yaml at the project root. Customize it and provide its path through the CONFIG_PATH environment variable (defaults to /app/application.yaml inside the container).
The Data Catalog Plugin needs to be configured in order to authenticate against a Collibra environment and to manage the correct domains and assets. It also allows to configure the base Community where the plugin expects everything to be stored under (see High Level Design).
| Configuration | Description | Default |
|---|---|---|
api.username |
Username to be used to access Collibra and perform all of the Data Catalog plugin operations | ${COLLIBRA_USERNAME} |
api.password |
Password to be used to access Collibra and perform all of the Data Catalog plugin operations | ${COLLIBRA_PASSWORD} |
api.endpoint |
Base endpoint of the target Collibra environment | ${COLLIBRA_HOST} |
Data Mesh entities in Witboost are mapped to Collibra concepts using a set of type IDs for each of the assets and their attributes. Furhtermore, the Data Catalog plugin expects all the assets and domains to be stored under a common Collibra community (see High Level Design), whose ID is also configurable.
| Configuration | Description | Default |
|---|---|---|
collibra.baseCommunityId |
ID of the base Collibra community that will store all domains and assets | |
collibra.initialStatusId |
ID of the initial status to assign to a Collibra asset. Default: Candidate | 00000000-0000-0000-0000-000000005008 |
Witboost domains map to a Collibra community containing two domains: A domain for all the assets, and a domain for the business terms. These are all configured on the collibra.domains configuration:
| Configuration | Description | Default |
|---|---|---|
collibra.domains.assetDomain.typeId |
ID of the domain type that represent the domain for storing Data Product assets. Default: Data Asset Domain | 00000000-0000-0000-0000-000000030001 |
collibra.domains.assetDomain.name |
Name of the domain for storing Data Product assets. | Data Products |
collibra.domains.glossary.typeId |
ID of the domain type that represent the domain for storing business terms. Default: Glossary | 00000000-0000-0000-0000-000000010001 |
collibra.domains.glossary.name |
Name of the domain for storing business terms. | Glossary |
Data Products and all their entities are created as different types of assets stored under the Collibra community on the appropriate domain: Data Products are stored on the asset domain configured on collibra.domains.assetDomain, while business terms are stored on the glossary domain configured on collibra.domains.glossary, all linked to the community linked to the Data Product domain.
Each asset type contains a attributes configuration to dynamically pick values from the input descriptor and add them as attributes of the corresponding type. These are expressed as key-value maps, where the key expresses a JSONPath to access the field from the respective root, and the value stores a Collibra Attribute Type ID used to create the desired attribute. Currently only single-value attributes are supported (string, number, boolean).
| Configuration | Description | Default |
|---|---|---|
collibra.assets.dataProduct.typeId |
ID of the asset that represents Data Products. Default: Data Asset | 00000000-0000-0000-0000-000000031002 |
collibra.assets.dataProduct.attributes |
Key-value object mapping a field of the data product to a Collibra attribute type ID. The field is expressed as a JSONPath with the Data Product as root. The JSONPath must be surrounded by square brackets []. Default: Mapping between Data Product description<->Collibra Description AttributeType |
attributes: { "[$.description]": "00000000-0000-0000-0000-000000003114" } |
collibra.assets.dataProduct.containsOutputPortRelationId |
ID of the relation type that links Data Products and Output Ports. Default: Data Asset groups Data Asset | 00000000-0000-0000-0000-000000007017 |
collibra.assets.outputPort.typeId |
ID of the asset that represents Output Ports. Default: Data Set | 00000000-0000-0000-0001-000400000001 |
collibra.assets.outputPort.attributes |
Key-value object mapping a field of the output port to a Collibra attribute type ID. The field is expressed as a JSONPath with the respective Output Port as root. The JSONPath must be surrounded by square brackets []. Default: Mapping between OutputPort description<->Collibra Description Attribute Type, OutputPort outputPortType<->Collibra tableType Attribute Type |
attributes: { "[$.description]": "00000000-0000-0000-0000-000000003114", "[$.outputPortType]": "00000000-0000-0000-0001-000500000008" } |
collibra.assets.outputPort.containsColumnRelationId |
ID of the relation type that links Output Ports and their columns. Default: Data Set contains Data Element | 00000000-0000-0000-0000-000000007062 |
collibra.assets.column.typeId |
ID of the asset that represents Columns. Default: Column | 00000000-0000-0000-0000-000000031008 |
collibra.assets.column.attributes |
Key-value object mapping a field of the column to a Collibra attribute type ID. The field is expressed as a JSONPath with the respective Column as root. The JSONPath must be surrounded by square brackets []. Default: Mapping between Column description<->Collibra Description Attribute Type, Column dataType<->Collibra Technical Data Type Attribute Type |
attributes: { "[$.description]": "00000000-0000-0000-0000-000000003114", "[$.dataType]": "00000000-0000-0000-0000-000000000219" } |
collibra.assets.businessTerm.typeId |
ID of the asset that represents business terms. Default: Business Term | 00000000-0000-0000-0000-000000011001 |
The Custom URL Picker endpoints expose CORS settings through the customUrlPicker.cors block in application.yaml.
| Configuration | Description | Default |
|---|---|---|
customUrlPicker.cors.endpointMask |
Endpoint mask to allow CORS | /v1/resources/** |
customUrlPicker.cors.origin |
Origin URL allowed on CORS requests | * |