Adding OKP for RAG support#2926
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
blublinsky
force-pushed
the
okp-rag
branch
2 times, most recently
from
4c89880 to
99d82f9
Compare
blublinsky
force-pushed
the
okp-rag
branch
3 times, most recently
from
914faae to
37a7b67
Compare
|
|
||
| SOLR_DOCS_TOOL_AGENT_SUPPLEMENT = """ | ||
| Solr docs tool: | ||
| * ``search_openshift_documentation`` searches published Red Hat product docs (not the cluster). Prefer one short query before answering from memory on product behavior, APIs, operators, upgrades, or whether docs exist. |
There was a problem hiding this comment.
"(not the cluster)" seems not clear: not the documents about this specific cluster? tool does not search inside the cluster for related objects?
|
|
||
| Omit the key to leave Solr hybrid RAG off (no client, no tool). When present, values map to `SolrHybridSettings`; `enabled: false` disables the feature without removing the block. `solr_http_base` must be a valid `http` or `https` URL with a host when `enabled` is true (checked in `validate_yaml()`). | ||
|
|
||
| When ``solr_hybrid.enabled`` is true, every entry under ``reference_content.indexes`` |
There was a problem hiding this comment.
solr_hybrid in config does not have the property enabled.
we can change the phrase to "when solr_hybrid" is defined in the config" corresponding to current code in PR
| "No docs above hybrid_score_threshold=%s for: %s", | ||
| cfg.hybrid_score_threshold, | ||
| query, | ||
| ) |
There was a problem hiding this comment.
should it be added here the fallback to "/select" endpoint?
There was a problem hiding this comment.
hybrid_score_threshold defaults to 0.0 (no filtering) and the operator does not expose this parameter — it always uses the default. When someone explicitly sets a threshold, they're choosing "no context over bad context." Falling back to /select would return the same or weaker documents that the hybrid path already rejected, undermining that intent.
| if retrieved_nodes: | ||
| logger.info( | ||
| "Retrieved %d document nodes for RAG context", len(retrieved_nodes) | ||
| ) |
There was a problem hiding this comment.
would it be useful to sort the documents from multiple sources using the same standard to avoid those at the beggining of the list are most weighted to the LLM's reasoning?
There was a problem hiding this comment.
Valid concern, but in practice:
FAISS and Solr are never used together for OpenShift docs — once we switch to Solr, FAISS-based product documentation goes away.
They can coexist for OpenShift docs (Solr) + BYOK. In that case there are two options: BYOK as direct RAG + Solr as a tool, or both BYOK and Solr as tools (PR for this is coming).
So the mixed-source scoring problem doesn't arise in the direct RAG path — it's either one source or the other, never both unsorted.
|
/retest e2e-ols-cluster |
|
/retest |
|
@blublinsky: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2 participants
Description
Summary
Adds optional Solr hybrid retrieval against the OKP portal-rag core: hybrid-search with a parallel lexical /select fallback, embedding reranking for lexical rows, and configurable behavior for either merging passages into the ask prompt (direct RAG) or exposing a search_openshift_documentation LangChain tool (tool-only path). Enables tool calling when only the Solr docs tool is configured (no MCP servers required), extends ASK-mode system prompts when that tool is active, and hardens retrieval so Solr failures degrade to empty passages instead of failing the request.
User-visible behavior
ols_config.solr_hybrid: enabled, solr_http_base, solr_direct_rag, lexical_rerank_min_cosine, and existing hybrid tuning fields (max_results, hybrid_pool_docs, hybrid_vector_boost, hybrid_score_threshold, chunk_filter_query, hybrid_solr_timeout_s, etc.).
solr_direct_rag: true: Solr chunks are merged with index RAG in DocsSummarizer._prepare_prompt_context (same flow as other RAG nodes).
solr_direct_rag: false: Solr is not merged into the initial prompt; the model may call search_openshift_documentation, which returns JSON passages (text, score, title, docs_url).
Tool calling: If hybrid is enabled for the docs tool path and the Solr client is available, _tool_calling_enabled is true even with no mcp_servers, so agent instructions and the tool loop run.
Prompts (ASK + tool guidance): Short Solr docs tool and grounded-answer style supplements appended via GeneratePrompt(..., solr_docs_tool_guidance=...).
How to try locally
Install Solr / rhokp on solr_http_base (e.g. http://localhost:8080).
export OLS_CONFIG_FILE=... with solr_hybrid.enabled: true and either solr_direct_rag: true or false.
First run may download the embedding model from Hugging Face.
Config
lm_providers:
type: openai
url: "https://api.openai.com/v1"
credentials_path: /tmp/test-openai-key
LLM:
models:gpt-5.4-mini(listed for this API key underGET /v1/models).ols_config:
solr_hybrid:
enabled: true
solr_http_base: "http://localhost:8080"
solr_direct_rag: true
conversation_cache:
type: memory
memory:
max_entries: 1000
logging_config:
app_log_level: info
lib_log_level: warning
default_provider: openai
default_model: gpt-5.4-mini
user_data_collection:
transcripts_disabled: true
authentication_config:
module: "noop"
dev_config:
enable_dev_ui: true
disable_auth: true
disable_tls: true
uvicorn_port_number: 8081
This is direct RAG. For tool-based solr_direct_rag: false
Type of change
Related Tickets & Documents
Checklist before requesting a review
Testing