Uk national #157

peterdudfield · 2025-12-17T10:25:15Z

Pull Request

Description

Move uk-national over to this quartz-api

National routes
GSP routes
status routes - this loads from the old database for the moment. This will change in the future
system routes

Currently

forecast/all takes about ~13 seconds, which is to slow (benchmark from old app was). We currently looping over ForecastAtTimestamp
pvlive/all takes about ~10 seconds

#147

How Has This Been Tested?

CI tests
ran locally, and UAT from airflow run against the API
deployed and ran on aws,
and UAT from airflow run against the API
deploy UI to use this API - link

dont do in this PR: add end to end test for uk-national with the app #179
dont do in this PR: add seperate UAT in airflow that tests it

Checklist:

My code follows OCF's coding style guidelines
I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
I have checked my code and corrected any misspellings

…nal-and-gsp-locations # Conflicts: # src/quartz_api/cmd/main.py # src/quartz_api/internal/backends/dataplatform/client.py # src/quartz_api/internal/models.py

…nal-and-gsp-framework # Conflicts: # src/quartz_api/cmd/main.py

…-locations # Conflicts: # src/quartz_api/cmd/main.py # src/quartz_api/internal/service/uk_national/gsp.py # src/quartz_api/internal/service/uk_national/national.py # src/quartz_api/internal/service/uk_national/system.py

# Conflicts: # src/quartz_api/internal/service/uk_national/national.py

…/forecast

# Conflicts: # src/quartz_api/internal/service/uk_national/national.py

# Conflicts: # src/quartz_api/internal/service/uk_national/gsp.py # src/quartz_api/internal/service/uk_national/national.py # src/quartz_api/internal/service/uk_national/system.py

# Conflicts: # src/quartz_api/internal/backends/dataplatform/client.py

peterdudfield · 2026-01-20T18:48:19Z

src/quartz_api/internal/backends/dataplatform/client.py

        return forecasts_one_timestamp

+    @override
+    async def get_forecast_for_multiple_locations(


I could also move this into the gsp.py route, and just use the route for get forecast at one timestamp. But we do need to make sure we only call ListForecastersRequest request once

devsjc

Lots of notes to do with changing where default values are managed and removing a bunch of optional ones. I think this makes the logic path much easier to follow.

src/quartz_api/cmd/main.py

src/quartz_api/internal/backends/dataplatform/client.py

src/quartz_api/cmd/main.py

src/quartz_api/internal/backends/dataplatform/client.py

devsjc · 2026-01-21T10:09:19Z

src/quartz_api/internal/backends/dataplatform/client.py

+        forecaster_name: str | None = None,
+        start_datetime: dt.datetime | None = None,
+        end_datetime: dt.datetime | None = None,
+        created_before_datetime: dt.datetime | None = None,


I don't think these should be optional at this level; it should be the job of the HTTP route calling these interface functions to put suitable defaults onto them.

Suggested change

forecaster_name: str | None = None,

start_datetime: dt.datetime | None = None,

end_datetime: dt.datetime | None = None,

created_before_datetime: dt.datetime | None = None,

forecaster_name: str,

start_datetime: dt.datetime,

end_datetime: dt.datetime,

created_before_datetime: dt.datetime,

I agree with datetimes, but the forecaster_name sometimes we just want to pull a forecast, and API doesnt need to know which model to pull this for.

Fair enough.

src/quartz_api/internal/service/uk_national/status.py

src/quartz_api/internal/service/uk_national/system.py

devsjc · 2026-01-21T11:33:48Z

src/quartz_api/internal/service/uk_national/test_time_utils.py

Most of this can go

I think this needs to stay. We need to have a function somewhere that floor the start time to the nearest 6 hours, and then minuses it. I think this is the right place for the tests

Not if they live in the defaults though? It would be odd to test the pandas 'floor' or subtract timedelta functions, as you trust pandas to have tested that.

Its not pandas

the logic of floor 6 and minus 2 days I think is worth its own function

linting didnt let me put datetime.now() in the defaults
B008 Do not perform function call `dt.datetime.now` in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable

devsjc · 2026-01-21T11:34:14Z

src/quartz_api/internal/service/uk_national/time_utils.py

Similarly, most of this can go

I think this needs to stay. We need to have a function somewhere that floor the start time to the nearest 6 hours, and then minuses it. I think this is the right place for it.

See my above comment.

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

peterdudfield · 2026-01-21T14:43:33Z

BTW, i got a linting error

B008 Do not perform function call `dt.datetime.now` in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable

you ok with this @devsjc ?

src/quartz_api/internal/service/uk_national/system.py

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

peterdudfield added 30 commits December 2, 2025 16:17

add framework for UK national and GSP

0aab57c

add gsp locations route

ac833c2

lint

f35f5b9

Merge branch 'main' into uk-national-and-gsp-framework

7a60342

Merge branch 'main' into uk-national-and-gsp-locations

5128fd3

PR comments

c382360

Update TODO elexon

caba26b

remove legacy routes

8d09c92

add start to getting national forecast

d16be98

Merge commit 'b1c66bee95de4ee6e27f7a4aa27865bd039d5523' into uk-natio…

c18b8be

…nal-and-gsp-locations # Conflicts: # src/quartz_api/cmd/main.py # src/quartz_api/internal/backends/dataplatform/client.py # src/quartz_api/internal/models.py

Merge commit 'b1c66bee95de4ee6e27f7a4aa27865bd039d5523' into uk-natio…

6203f51

…nal-and-gsp-framework # Conflicts: # src/quartz_api/cmd/main.py

rename + lint

3c7e911

Merge branch 'uk-national-and-gsp-framework' into national/forecast

3110128

# Conflicts: # src/quartz_api/internal/service/uk_national/national.py

get working for model_name

f01f99c

Merge commit 'ba856c2ed063512c437dbc9cd10472bf3d1ec146' into national…

618d0c7

…/forecast

Merge branch 'main' into national/forecast

b9b3fb0

# Conflicts: # src/quartz_api/internal/service/uk_national/national.py

Merge branch 'main' into uk-national-and-gsp-locations

702b73b

# Conflicts: # src/quartz_api/internal/service/uk_national/gsp.py # src/quartz_api/internal/service/uk_national/national.py # src/quartz_api/internal/service/uk_national/system.py

lint

e3e068f

lint

0996fbf

national/pvlive

7ac4a80

lint

b038e79

Merge branch 'national/pvlive' into uk-national-and-gsp-locations

cce60d6

# Conflicts: # src/quartz_api/internal/backends/dataplatform/client.py

tidy up

332f540

add options for get_window

2a8cb85

Merge branch 'start-end' into national/forecast

74ee1d6

add start and end datetime

91445bf

add start and end filters

539c4c8

lint

9297666

Merge branch 'start-end' into national/pvlive

0469b5b

peterdudfield commented Jan 20, 2026

View reviewed changes

move Region class

aa9924f

devsjc requested changes Jan 21, 2026

View reviewed changes

peterdudfield and others added 9 commits January 21, 2026 11:59

Update src/quartz_api/cmd/main.py

79cefd9

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

Update src/quartz_api/cmd/main.py

fca0507

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

Apply suggestions from code review

6307a31

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

move CamelCase to UK-National only

505dc4b

lint

171b68e

UUID remove get_window function

710590f

role back get_window

9807b4c

remove TODO

6b0f938

TODO

fb4526a

peterdudfield added 12 commits January 21, 2026 14:44

permissions in cache

2d34034

remove get_forecast_for_multiple_locations put into uk-national

ca81593

lint

5b8c247

limit end_datetime by permissions

06d7686

import datetime as dt

6bee067

update to use datetime

fe2013a

change format_datetime to add_timezone

699906d

add comments and format

f027fa6

use forecast init

d2e66f3

fix tests

8d0700c

lint

3b0e767

tidy up

5be37d5

devsjc reviewed Jan 22, 2026

View reviewed changes

src/quartz_api/internal/service/uk_national/system.py Outdated Show resolved Hide resolved

peterdudfield and others added 4 commits January 22, 2026 07:50

Update src/quartz_api/internal/service/uk_national/system.py

1f0ba2d

Co-authored-by: devsjc <47188100+devsjc@users.noreply.github.com>

remove get_window from backend dataplatform client

6006e7a

fix pvlive/all start time

08f10b3

Merge branch 'main' into uk-national

6cba52d

Uh oh!

Uk national #157

Are you sure you want to change the base?

Uk national #157

Uh oh!

Conversation

peterdudfield commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request

Description

How Has This Been Tested?

Checklist:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devsjc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peterdudfield commented Jan 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

peterdudfield commented Dec 17, 2025 •

edited

Loading