fix(ci): improvements and CI time reduction by kyteinsky · Pull Request #231 · nextcloud/context_chat

kyteinsky · 2026-04-10T10:19:15Z

also update the actions

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

kyteinsky

there seems to be a weird issue with integration-test-file-listener.

OUT_TEST_PROMPT_JOBS test in Run the prompts again step fails.
It checks for the presence of "background jobs" in the prompt returned by the testing app, which contains the chunks of documents and their paths.
The output shows "files/developer_manual/basics/backgroundjobs.md" (files__default: 1540) but the body is empty, same for a few others.

The most part of Nextcloud is written in PHP, Typescript / JavaScript, so we have some more fine grained coding standards for those languages:

.. toctree::
	:maxdepth: 1

	php
	javascript
	html_css

License headers

files/developer_manual/basics/backgroundjobs.md

files/developer_manual/basics/dependency_injection.md

files/developer_manual/design/atomiccomponents.md

How do I set my TimedJob to be time insensitive?

Indexing logs show that it was not empty when parsed. They were most probably added since there is no log saying "Some sources were not loaded".

2026-04-10T12:23:01.9868809Z {"timestamp": "2026-04-10T11:51:27.073947+00:00", "level": "DEBUG", "logger": "ccb.injest", "message": "processing source", "filename": "injest.py", "function": "_sources_to_indocuments", "line": 57, "thread_name": "AnyIO worker thread", "pid": 18270, "version": "5.3.0", "source_id": "files__default: 1540"}
2026-04-10T12:23:01.9870170Z {"timestamp": "2026-04-10T11:51:27.074654+00:00", "level": "DEBUG", "logger": "ccb.injest", "message": "decoded non empty source", "filename": "injest.py", "function": "_sources_to_indocuments", "line": 77, "thread_name": "AnyIO worker thread", "pid": 18270, "version": "5.3.0", "source_id": "files__default: 1540"}
2026-04-10T12:23:01.9871720Z {"timestamp": "2026-04-10T11:51:27.074846+00:00", "level": "DEBUG", "logger": "ccb.injest", "message": "split document into chunks", "filename": "injest.py", "function": "_sources_to_indocuments", "line": 88, "thread_name": "AnyIO worker thread", "pid": 18270, "version": "5.3.0", "source_id": "files__default: 1540", "len(split_docs)": 5}

Then the share to the test user also goes through:

2026-04-10T12:23:02.3501015Z {"timestamp": "2026-04-10T12:08:36.882961+00:00", "level": "DEBUG", "logger": "ccb.controller", "message": "Update access declarative request:", "filename": "controller.py", "function": "_", "line": 222, "thread_name": "AnyIO worker thread", "pid": 6021, "version": "5.3.0", "user_ids": ["admin", "test"], "source_id": "files__default: 1540"}
2026-04-10T12:23:02.3502264Z {"timestamp": "2026-04-10T12:08:37.388690+00:00", "level": "DEBUG", "logger": "ccb.vectordb", "message": "decl update access", "filename": "service.py", "function": "decl_update_access", "line": 63, "thread_name": "AnyIO worker thread", "pid": 28273, "version": "5.3.0", "user_ids": ["admin", "test"], "source_id": "files__default: 1540"}
2026-04-10T12:23:02.3503330Z {"timestamp": "2026-04-10T12:08:37.411353+00:00", "level": "INFO", "logger": "uvicorn.access", "message": "127.0.0.1:54262 - \"POST /updateAccessDeclarative HTTP/1.1\" 200", "filename": "h11_impl.py", "function": "send", "line": 482, "thread_name": "MainThread", "pid": 6021, "version": "5.3.0"}

marcelklehr · 2026-04-13T08:49:31Z

+        run: |
+          echo "Downloading release tarball from $DOWNLOAD_URL"
+          wget $DOWNLOAD_URL -O nextcloud.zip
+          unzip nextcloud.zip


Why do we need this?

came with update of github actions 🤷
https://github.com/nextcloud/.github/blob/506aa0161a94d04ba8bed29c9271b373bc1c3829/workflow-templates/appstore-build-publish.yml#L149-L156

marcelklehr · 2026-04-13T08:51:07Z

+          ./occ config:app:set --value 30 --type integer context_chat indexing_job_interval # 30 seconds
+          ./occ config:app:set --value 10 --type integer context_chat crawl_job_interval # 10 seconds
+          ./occ config:app:set --value '10' --type string context_chat action_job_interval # 10 seconds
+          ./occ config:app:set --value '10' --type string context_chat fs_listener_job_interval # 10 seconds


We can probably remove action_job_interval, indexing_job_interval

still on the old base, will do if we decide to switch to the new one, or do separate changes for the new PR.

marcelklehr · 2026-04-13T08:52:01Z

+          ./occ config:app:set --value 30 --type integer context_chat indexing_job_interval # 30 seconds
+          ./occ config:app:set --value 10 --type integer context_chat crawl_job_interval # 10 seconds
+          ./occ config:app:set --value '10' --type string context_chat action_job_interval # 10 seconds
+          ./occ config:app:set --value '10' --type string context_chat fs_listener_job_interval # 10 seconds


marcelklehr · 2026-04-13T08:52:32Z

+
+          echo "Check for user admin: Should be there"  # ---------------
+
+          echo "::group::Check english prompt with admin user"


neat trick!

marcelklehr · 2026-04-13T08:53:30Z

+          ./occ files:scan admin
+          ./occ context_chat:scan admin -m text/markdown &
+          ./occ context_chat:scan admin -m text/x-rst &
+          ./occ context_chat:scan admin -m application/pdf &


This command was removed no?

we're basing this still on the previous version
although, it might make sense to just do the CI improvements for the new reverse flow PR.

marcelklehr · 2026-04-13T08:53:48Z

 	"require-dev": {
 		"nextcloud/coding-standard": "^1.3.2",
-		"nextcloud/ocp": "dev-master",
+		"nextcloud/ocp": "dev-stable32",


I had accidentally used some function that wasn't available in NC32 once but it worked on my machine and IDE completion since it referred to the latest definitions.
It was caught in CI but this might still make sense.

kyteinsky · 2026-04-13T10:30:32Z

force-merging since major changes land in reverse content flow PR which require changing the CI, the issue here will be fixed there.

kyteinsky added 12 commits April 10, 2026 13:39

chore(ci): set NC loglevel to 1

9211f5f

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

chore(ci): run only on protected branches PRs

1340676

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): checkout the auto-enabled viewer app

c355ddb

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

chore(ci): split integration-test workflows into separate files

712d92f

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

chore(ci): update gh workflows

7a36d1b

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): add a folder/files filter for integration tests

821b0cb

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): reduce CI time while testing most of the functionality

5b33d78

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): requires changes step in integration tests

51f74c8

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(composer): use dev-stable32 and update composer deps

60e52c8

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): don't re-require roave's package in psalm

e13aef9

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): integration tests should have different concurrency groups

2a0948c

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): allow the embedding server to start up in integration tests

52ad657

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

kyteinsky force-pushed the fix/ci-improvements branch from 217252c to 52ad657 Compare April 10, 2026 11:27

kyteinsky requested a review from marcelklehr April 10, 2026 12:08

kyteinsky added 2 commits April 10, 2026 20:36

fix(ci): add file-deletion-method matrix option back to file listener

ddd94e4

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

fix(ci): format prompts in integration test

9dfa991

Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>

kyteinsky commented Apr 10, 2026

View reviewed changes

marcelklehr reviewed Apr 13, 2026

View reviewed changes

kyteinsky merged commit 4876020 into main Apr 13, 2026
26 of 29 checks passed

kyteinsky deleted the fix/ci-improvements branch April 13, 2026 10:31


		echo "Check for user admin: Should be there" # ---------------

		echo "::group::Check english prompt with admin user"

Conversation

kyteinsky commented Apr 10, 2026

Uh oh!

kyteinsky left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kyteinsky commented Apr 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants