Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions API_SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,21 @@ Get specific agency details.
#### POST /ckb/agency/add
Create new agency.

> **Constraint**: CKB supports only **one agency per deployment**. This endpoint returns `409 Conflict` if an agency already exists. The Add Agency button in the GUI is hidden once an agency has been created.

**Request Body:**
```json
{
"name": "string",
"sector": "string",
"external_id": "string"
"sector": "string",
"externalId": "string"
}
```

**Responses:**
- `200 OK` — Agency created successfully.
- `409 Conflict` — An agency already exists. Body: `"An agency already exists. CKB supports only one agency per deployment."`

#### POST /ckb/agency/edit
Update agency information.

Expand All @@ -114,14 +120,11 @@ Update agency information.
```

#### POST /ckb/agency/remove
Delete agency.

**Request Body:**
```json
{
"base_id": "uuid"
}
```
> **Blocked**: Agency deletion is not permitted. This endpoint always returns `405 Method Not Allowed`. An agency can only be updated via `POST /ckb/agency/edit`.

**Response:**
- `405 Method Not Allowed` — Body: `"Agency deletion is not permitted. An agency can only be updated."`

### Source Management

Expand Down
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ graph TB
- **Purpose**: User interface for CKB management
- **Location**: `/GUI/`
- **Technology**: React, TypeScript, Vite
- **Features**: Agency management, source configuration, file uploads, monitoring
- **Features**: Agency management (single-agency enforcement), source configuration, file uploads, monitoring

### 2. Ruuter External API
- **Purpose**: External-facing REST API with authentication
Expand Down Expand Up @@ -196,7 +196,7 @@ sequenceDiagram

### Core Tables

1. **agency**: Organization/agency information
1. **agency**: Organization/agency information — **only one non-deleted agency is permitted per CKB deployment**
2. **source**: Data source configurations
3. **source_file**: Individual file metadata
4. **source_run_page**: Scraping execution logs
Expand Down
3 changes: 1 addition & 2 deletions DSL/DMapper/ckb/hbs/enrich_signed_urls.handlebars
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
[
{{#each agencies}}
{
"clientId": "{{this.clientId}}",
"clientDataHash": "{{this.clientDataHash}}",
"agencyDataHash": "{{this.agencyDataHash}}",
{{#if (eq (getObjectKeyFromObjectArray ../downloadUrls "path" this.path "download_url") null) }}
"signedS3Url": null
{{else}}
Expand Down
24 changes: 24 additions & 0 deletions DSL/Resql/ckb/GET/agency/check_agency_exists.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
declaration:
version: 0.1
description: "Check if any agency exists in this deployment"
method: get
namespace: agency
returns: json
allowlist:
query: []
response:
fields:
- field: exists
type: boolean
description: "True if at least one non-deleted agency exists"
*/
SELECT EXISTS (
SELECT 1
FROM (
SELECT DISTINCT ON (base_id) is_deleted
FROM agency_management.agency
ORDER BY base_id, updated_at DESC
) latest
WHERE is_deleted = FALSE
) AS exists;
30 changes: 10 additions & 20 deletions DSL/Resql/ckb/GET/agency/list_agency_data_hash_and_url.sql
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,25 @@ declaration:
description: "list agency with data hash and url"
method: get
namespace: agency
allowlist: {}
returns: json
allowlist:
query:
- field: agencyIds
type: string
description: "agency base ids, comma separated"
response:
fields:
- field: data_hash
type: integer
description: "data hash"
- field: client_id
- field: agency_data_hash
type: string
description: "base id of agency"
description: "data hash of the agency"
- field: path
type: string
description: "storage path for zipped data"

*/
SELECT
external_id AS client_id,
data_hash AS client_data_hash,
data_hash AS agency_data_hash,
zipped_data_url AS path
FROM agency_management.agency a1
WHERE external_id = ANY(STRING_TO_ARRAY(:agencyIds, ','))
FROM agency_management.agency
WHERE is_deleted = FALSE
AND updated_at = (
SELECT MAX(updated_at)
FROM agency_management.agency a2
WHERE a2.base_id = a1.base_id
AND a2.is_deleted = FALSE
)
AND is_deleted = FALSE;
SELECT MAX(updated_at)
FROM agency_management.agency
WHERE is_deleted = FALSE
);
22 changes: 21 additions & 1 deletion DSL/Resql/ckb/POST/agency/create_agency.sql
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,26 @@ declaration:
type: string
description: "Record last update timestamp"
*/
WITH
_lock AS (
-- Serialize concurrent create requests; second request blocks until first commits/rolls back
SELECT pg_advisory_xact_lock(hashtext('single_agency_create'))
),
_guard AS (
-- Re-check inside the lock so a concurrent request that passed the Ruuter-level
-- check but hasn't committed yet is still blocked
SELECT EXISTS (
SELECT 1
FROM (
SELECT DISTINCT ON (base_id) is_deleted
FROM agency_management.agency
ORDER BY base_id, updated_at DESC
) latest
WHERE is_deleted = FALSE
) AS already_exists
)
INSERT INTO agency_management.agency (name, sector, external_id)
VALUES (:name, :sector, :external_id)
SELECT :name, :sector, :external_id
FROM _lock, _guard
WHERE NOT already_exists
RETURNING id, base_id, name, sector, external_id, created_at, updated_at;
36 changes: 36 additions & 0 deletions DSL/Resql/ckb/POST/agency/reset_agency_is_zipping.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
declaration:
version: 0.1
description: "Reset is_zipping flag to FALSE and set zip_dirty to TRUE for the latest agency record by base_id, allowing the cron to retry zipping"
method: post
accepts: json
returns: json
namespace: agency
allowlist:
body:
- field: base_id
type: string
description: "Agency base ID"
response:
fields:
- field: id
type: string
description: "Record ID"
*/
SELECT copy_row_with_modifications(
'agency_management.agency',
'id', '::UUID', id::VARCHAR,
ARRAY[
'is_zipping', '::BOOLEAN', 'FALSE',
'zip_dirty', '::BOOLEAN', 'TRUE',
'updated_at', '::TIMESTAMP WITH TIME ZONE', NOW()::VARCHAR
]::VARCHAR[]
) as id
FROM agency_management.agency
WHERE base_id = :base_id::UUID
AND updated_at = (
SELECT MAX(updated_at)
FROM agency_management.agency
WHERE base_id = :base_id::UUID
)
AND is_deleted = FALSE;
13 changes: 2 additions & 11 deletions DSL/Ruuter.internal/ckb/GET/client/data/import.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,13 @@ declaration:
accepts: json
returns: json
namespace: client
allowlist:
params:
- field: clientIds
type: string
description: "Base IDs of the clients"

extractRequestData:
assign:
clientIds: "${incoming.params.clientIds}"
allowlist:
paramas: []

getAgenciesData:
call: http.get
args:
url: "[#CKB_RESQL]/agency/list_agency_data_hash_and_url"
query:
agencyIds: ${clientIds}
result: agencyRes

mapPaths:
Expand Down
27 changes: 27 additions & 0 deletions DSL/Ruuter.internal/ckb/POST/agency/reset-is-zipping.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
declaration:
call: declare
version: 0.1
description: "Reset is_zipping to FALSE for an agency and set zip_dirty to TRUE, allowing the cron to retry zipping"
method: post
accepts: json
returns: json
namespace: agency
allowlist:
body:
- field: agency_id
type: string
description: "Agency base ID"

resetIsZipping:
call: http.post
args:
url: "[#CKB_RESQL]/agency/reset_agency_is_zipping"
headers:
type: json
body:
base_id: ${incoming.body.agency_id}
result: res

returnSuccess:
return: "OK"
next: end
20 changes: 20 additions & 0 deletions DSL/Ruuter.internal/ckb/POST/agency/update-data-hash.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,21 @@ declaration:
items:
type: object
properties:
status:
type: string
data_hash:
type: string
description: "result of zip and upload"
- field: zip_path
type: string
description: "ZIP path"

checkZipSucceeded:
switch:
- condition: ${incoming.body.results != null && incoming.body.results.length > 0 && incoming.body.results[0].status == 'success'}
next: updateAgencyDataHash
next: resetIsZipping

updateAgencyDataHash:
call: http.post
args:
Expand All @@ -34,6 +42,18 @@ updateAgencyDataHash:
base_id: ${incoming.body.agency_id}
data_hash: ${incoming.body.results[0].data_hash}
result: res
next: scheduleSource

resetIsZipping:
call: http.post
args:
url: "[#CKB_RUUTER_INTERNAL]/agency/reset-is-zipping"
headers:
type: json
body:
agency_id: ${incoming.body.agency_id}
result: res
next: returnSuccess

scheduleSource:
call: http.get
Expand Down
23 changes: 10 additions & 13 deletions DSL/Ruuter/ckb/POST/agency/add.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,22 @@ extractRequestData:
sector: "${incoming.body.sector}"
external_id: "${incoming.body.externalId}"

checkNameExists:
checkAgencyExists:
call: http.get
args:
url: "[#CKB_RESQL]/agency/check_name_exists"
query:
name: ${name}
exclude_base_id: ""
result: nameCheckResult
next: validateNameUnique
url: "[#CKB_RESQL]/agency/check_agency_exists"
result: agencyExistsResult
next: validateSingleAgency

validateNameUnique:
validateSingleAgency:
switch:
- condition: ${nameCheckResult.response.body[0].exists == true}
next: returnError
- condition: ${agencyExistsResult.response.body[0].exists == true}
next: returnAgencyExistsError
next: createAgency

returnError:
return: "Agency with this name already exists"
status: 400
returnAgencyExistsError:
return: "An agency already exists. CKB supports only one agency per deployment."
status: 409
next: end

createAgency:
Expand Down
7 changes: 4 additions & 3 deletions DSL/Ruuter/ckb/POST/agency/remove.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ declaration:
type: string
description: "Base ID of the agency to delete"

extractRequestData:
assign:
base_id: "${incoming.body.baseId}"
returnNotAllowed:
return: "Agency deletion is not permitted. CKB supports only one agency per deployment, and deletion is not allowed to prevent accidental data loss."
status: 405
next: end

deleteAgencyFromDatabase:
call: http.post
Expand Down
4 changes: 3 additions & 1 deletion GUI/src/pages/Agency/SaveAgency.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,9 @@ const SaveAgency: FC = () => {
toast.open({
type: 'error',
title: t('global.notificationError'),
message: error.message || t('knowledgeBase.agencyCreateError'),
message: error.response?.status === 409
? t('knowledgeBase.agencyAlreadyExists')
: error.message || t('knowledgeBase.agencyCreateError'),
});
},
});
Expand Down
10 changes: 6 additions & 4 deletions GUI/src/pages/Agency/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ const AgencyComponent: FC = () => {
appearance="text"
onClick={() => setDeleteModal(row.original)}
className="agencies__action-btn"
disabled={deleteAgencyMutation.isLoading}
disabled={deleteAgencyMutation.isLoading || knowledgeBaseData.total === 1}
>
<Icon
icon={<MdOutlineDeleteOutline fontSize={20} />}
Expand Down Expand Up @@ -230,9 +230,11 @@ const AgencyComponent: FC = () => {
>
<h1 className="h1">{t('knowledgeBase.agencies')}</h1>
<Track gap={12}>
<Link to="/agency/add">
<Button appearance="primary">{t('knowledgeBase.addAgency')}</Button>
</Link>
{!knowledgeBaseData.total && (
<Link to="/agency/add">
<Button appearance="primary">{t('knowledgeBase.addAgency')}</Button>
</Link>
)}
</Track>
</Track>

Expand Down
1 change: 1 addition & 0 deletions GUI/translations/en/common.json
Original file line number Diff line number Diff line change
Expand Up @@ -599,6 +599,7 @@
"discardChanges": "Discard changes",
"confirmDiscardChanges": "Are you sure you want to discard changes?",
"agencyCreated": "Agency created successfully",
"agencyAlreadyExists": "An agency already exists. CKB supports only one agency per deployment.",
"agencyLoadError": "Failed to load agency",
"agencyUpdated": "Agency updated successfully",
"agencyUpdateError": "Failed to update agency",
Expand Down
Loading
Loading