UN-2946 [FEAT] Prompt Studio lookups bridge, executor hook, and IDE wiring (OSS side)#1929
Open
chandrasekharan-zipstack wants to merge 56 commits intomainfrom
Open
UN-2946 [FEAT] Prompt Studio lookups bridge, executor hook, and IDE wiring (OSS side)#1929chandrasekharan-zipstack wants to merge 56 commits intomainfrom
chandrasekharan-zipstack wants to merge 56 commits intomainfrom
Conversation
…prompt list endpoint
- Add CustomToolListSerializer for the list action to avoid N+1 queries
(profile lookups, prompt fetching, coverage calculation per tool)
- Add ToolStudioPromptListSerializer with only prompt_id, prompt_key,
enforce_type, sequence_number
- Add GET /prompt-studio/prompt/?tool_id={uuid} list endpoint
- List action uses select_related and Subquery annotation for prompt_count
- Detail endpoint unchanged (still uses full CustomToolSerializer)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add lookup-studio plugin detection with dynamic import - Add PromptStudioPopoverContent for hover submenu (Projects / Look-Ups) following the same Popover pattern as HITL and Platform Settings - Register lookups/* route in useMainAppRoutes.js Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ofile_manager Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on enrichment - Add lookup_config export in prompt_studio_helper and registry_helper via cloud plugin guard (try/except ImportError) - Store raw output in PromptStudioOutputManager, enriched in cloud LookupOutputResult — preserving both for UI tab display - Add LookupEnrichmentProtocol and plugin call in post-extraction pipeline using ExecutorPluginLoader (no-op in OSS) - Track lookup LLM usage via standard metrics pipeline (usage_kwargs with run_id/execution_id, capture_metrics) - Move webhook postprocessing from answer_prompt to pipeline - Frontend: dynamic plugin imports for LookupMenuItem, LookupIndicator, LookupOutputTabs in prompt cards; fetch lookup outputs on page load - Add scroll-to-prompt support via query param in DocumentParser Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…age reason - Extract get_lookup_config() to prompt_studio/lookup_utils.py, replacing 4 identical try/except ImportError blocks across prompt_studio_helper and prompt_studio_registry_helper - Add LOOKUP to LLMUsageReason choices (was missing, causing invalid choice on usage records from lookup enrichment LLM calls) - Migration: usage_v2/0004_add_lookup_usage_reason Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Store prompt/completion/total token counts from the most recent complete() call on the LLM object itself, making usage data queryable without relying on the Audit pipeline roundtrip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…hook methods Move lookup result-application logic to the cloud plugin, matching the challenge plugin pattern where the plugin owns metadata mutation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…port fallback Add reusable extraction_complete/extraction_error callback tasks to the ide_callback worker, replacing the need for Django-based celery workers for text extraction. Add ExtractionAPIClient for internal API calls. Add polling fallback to WebSocket transport for local dev reliability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ntrusive UX" This reverts commit d6e136d.
Replace inline DRAFT lookup check with pluggable cloud-only hook. Uses try/except ImportError pattern — zero lookup code in OSS. Collects all DRAFT lookups in one pass with markdown-linked error messages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Consolidate cloud imports into lookup_utils.py with persist_lookup_output() and validate_lookups_for_export() wrappers - Fix LookupEnrichmentProtocol.run() return type to None matching challenge/evaluation pattern - Revert logger.info to logger.debug in websocket_views.py - Eliminate duplicated LookupOutputTabs ternary with renderWithLookupWrapper helper - Move lookups menu constants from SideNavBar.jsx to cloud plugin - Harden DocumentParser.jsx scrollTo with UUID validation and fix useEffect dependency - Revert SocketContext transport to ["websocket"] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move Prompt Studio / Look-Ups navigation from a hover popover on the sidebar into a Segmented control within the ToolNavBar. CustomTools dynamically imports LookupList from the plugin and renders tabs when available, falling back to projects-only view in OSS mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch from eager per-call Audit HTTP push to a deferred batch write pattern for adapter usage. LLM/embedding calls stash records in-memory; the executor flushes them into ExecutionResult metadata; the Celery task batch-writes via a new internal endpoint. Adds 5 nullable columns to Usage (reference_id, reference_type, execution_time_ms, status, error_message) and a composite index for lookup dashboard queries. Extensible choice lists allow cloud plugins to register additional usage reasons and reference types. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…izer Bridge function in lookup_utils.py lets cloud plugins enrich PromptStudioOutputSerializer with lookup data (enriched output, lookup name). Enables real-time lookup results via WebSocket without page refresh. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… metadata passthrough - Wire usage_kwargs_extra from lookup config into LLM usage_kwargs for execution observability - Add error handling around enricher.run() with explicit ERROR usage records - Generic passthrough of _usage_kwargs into usage records for arbitrary metadata (e.g. reference_id) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…hooks Add dynamic import of getEnrichedCopyText so the copy button copies enriched lookup output when the Enriched tab is active. Applied to both single-pass and multi-profile output paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…modified_at fix - Add /prompt-studio/<pk>/lookup-validation/ endpoint backing the FE Export/Deploy gate; multi-var block check accepts prompt_ids so a single prompt run isn't blocked by an unrelated multi-var lookup. - Add /prompt-output/latest-by-keys/ endpoint that returns the most recent raw output per prompt_key for the test panel's "Use Latest Outputs" helper. - Fix prompt output modified_at not refreshing on re-runs (QuerySet.update bypasses auto_now); set timezone.now() explicitly in the update args. - lookup_utils: bridge get_lookup_validation_for_tool and get_multi_var_lookups_for_tool with prompt_ids scoping. - Header wires useLookupExportGate via try-import (no-op stub in OSS). - TokenUsage treats all-null Usage rows as empty. - CombinedOutput / JsonView build enriched dict from metadata.lookup_outputs to back the Raw|Enriched output toggle. - .gitignore: widen docker/compose.*.yaml. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ab on back - ProfileInfoBar: swap Row/Col for plain flex-wrap div — kills Ant Row negative-margin quirk that overlapped wrapped pills in combined output. - CustomTools: honor location.state.activeTab so back navigation from lookup detail lands on the Look-Ups tab instead of defaulting to Projects. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces nullable last_exported_at on CustomTool (populated on first successful export) so staleness checks can compare against downstream mutations without a data backfill. NULL is treated as "unknown" and suppresses the lookup-dirty flag to avoid false alarms on pre-feature projects. Adds the get_latest_lookup_mutation_for_tool bridge in lookup_utils so OSS stays decoupled from the cloud plugin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When enricher.run() raises, surface a user-visible ERROR log line in the workflow execution log alongside the existing usage record. Keeps lookup failures observable next to the other pre/post lookup lines we already emit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Loads useLookupDirtySeed (server-side is_lookup_dirty) and useLookupExportGate from the cloud plugin via dynamic imports so the reminder banner reflects lookup changes across page reloads and the banner's Export flow goes through the same validation modal as the main buttons. Also adds a titleAdornment slot on ToolNavBar for rendering the onboarding tooltip and relaxes EmptyState.text to accept nodes for the tagline + link composition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… state - Delegate production lookup enrichment to LookupEnrichment.run_with_metrics so the executor and the IDE test path share LLM construction, error handling, and usage-record emission. - Let ExecutionLogs callers pass an arbitrary backRouteState via location state so nested UI restore (e.g. a sub-tab) no longer needs special casing in this component. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bucket of hardening fixes driven by a staff-level PR re-review: - Org-scope latest_outputs_by_keys (was cross-tenant readable via raw .objects.filter() that bypassed OrganizationFilterBackend). - Hide lookup payload shape from OSS: three new opaque bridge helpers (get_original_value_if_enriched, attach_combined_output_enrichment, extract_prompt_output_enrichment) replace direct reads of metadata["lookup_outputs"] / _lookup_outputs / lookup_outputs in output_manager_helper, CombinedOutput.jsx, and usePromptOutput.js. - Split usage_v2 index into a new 0005 migration that uses AddIndexConcurrently + atomic=False so prod doesn't lock the billing table during build. - Delete stale workers/tests/test_usage.py that imported the removed UsageHelper module. - SDK1 LLM gains public get_last_usage_record() so downstream code stops reaching into _pending_usage across plugin boundaries. - legacy_executor stamps metadata["lookup_errors"][prompt] on a failed lookup outcome for dashboards that surface partial-failure runs. - extraction_client docstring notes the cloud-only endpoint contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Module-level probe in prompt_studio/lookup_utils.py — swap per-function try/except ImportError for a single LOOKUPS_AVAILABLE flag. Add attach_lookup_config / attach_lookup_configs_to_tool_settings helpers so the direct metadata["lookup_errors"] write and the lookup_config key stamping both route through the bridge. - Reject org=None in UsageBatchCreateView (usage_v2/internal_views.py). - Lift useLookupExportGate to a single mount in ToolIde.jsx; thread checkLookups down into custom-tools/header/Header.jsx (eliminates the double modal-portal risk). - Delete the direct metadata["lookup_errors"] write from workers/executor/executors/legacy_executor.py — flat summary is now stamped by LookupEnrichment.write_lookup_error in the cloud plugin. Replace hardcoded "lookup_llm" metrics key with lookup_cls.METRICS_KEY. - Trim boilerplate comments across CombinedOutput, PromptCardItems, PromptOutput, usePromptOutput, prompt-card/Header, CustomToolsHelper, SideNavBar — keep the why-comments, drop the what-comments.
OSS counterpart to the cloud-side data-model change. Wires the prompt studio runtime to the new wire shape, surfaces lookup runnability state in the prompt card, and adds the usage_v2 enum entries the cloud side records against (lookup as an LLM usage reason, lookup_version as a reference type). Partially working — known follow-up: TODO: rework lookup input UX. The current variable-mapping flow is awkward (separate rows, manual prompt selection per variable); needs a redesign that mirrors how users actually compose a lookup template. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bring in: - modified_at auto-bump fix (BaseModelQuerySet/BaseModelManager + save() override) - legacy_executor: tool-context warning, backtick log formatting, organization IDs threaded into ExecutorToolShim - prompt-service log streaming + markdown rendering - minio bucket-listing scope fix and CustomMarkdown URL-safety helper Conflict resolutions: - workers/executor/executors/legacy_executor.py: combine HEAD's _usage_records field and typed annotations with main's _execution_id/_organization_id context fields and tool-context warning. Keep both the usage flush after challenge and main's backticked stream_log message. Drop redundant explicit modified_at workarounds now that BaseModelQuerySet auto-bumps modified_at on QuerySet.update(): - prompt_studio_output_manager_v2/output_manager_helper.py: remove the "modified_at": timezone.now() entry passed to PromptStudioOutputManager .objects.filter(...).update(**args), and the now-unused timezone import.
When a configured lookup runs but extraction returned None for the source prompt, _run_lookup_enrichment used to fall through silently — leaving users wondering why enrichment didn't appear. Stream a one-line workflow log via shim.stream_log so the skip is visible alongside other tool-run events.
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds lookup/enrichment plumbing with OSS-safe fallbacks, lookup validation and export staleness tracking, buffered/batched usage reporting with schema migrations and internal bulk API, extraction completion/error callbacks and client, worker-side usage buffering/flush, and multiple frontend integrations for lookup UI and enriched output. Changes
Sequence Diagram(s)sequenceDiagram
participant Executor as Worker Executor
participant Orchestrator as Orchestrator/Task
participant WorkerTask as Celery Task
participant UsageClient as UsageAPIClient
participant Backend as Backend API
participant DB as Database
Executor->>Orchestrator: run execution, collect _usage_records
Orchestrator->>WorkerTask: return ExecutionResult (metadata includes usage_records)
WorkerTask->>UsageClient: bulk_create_usage(usage_records, organization_id)
UsageClient->>Backend: POST /v1/usage/batch/ (records)
Backend->>DB: INSERT usage rows
DB-->>Backend: OK
Backend-->>UsageClient: 200 { created: N } / success
UsageClient-->>WorkerTask: success/failure
WorkerTask-->>Orchestrator: log result / continue
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (10)
workers/executor/tasks.py (1)
113-130:⚠️ Potential issue | 🟠 MajorUsage flush failure is still non-recoverable (records can be dropped).
When
bulk_create_usagefails, this path only logs and continues. That still allows silent billing-row loss on transient outages.Proposed fix
if not ok: logger.error( "bulk_create_usage returned failure for %d records " "(run_id=%s organization_id=%s)", len(usage_records), context.run_id, context.organization_id, ) + raise ConnectionError("bulk_create_usage returned failure") except Exception: logger.error( "Failed to flush %d usage records for run_id=%s organization_id=%s", len(usage_records), context.run_id, context.organization_id, exc_info=True, ) + raise🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@workers/executor/tasks.py` around lines 113 - 130, The current flush path in tasks.py logs failures from bulk_create_usage but swallows them, risking dropped billing rows; modify the failure handling inside the block that calls bulk_create_usage so that on a non-ok result or on exception you propagate an error to the caller (either by re-raising the caught exception or raising a RecoverableError) instead of only logging, so the orchestration can retry; keep the existing logger.error messages (including run_id and organization_id) but follow them with a raise to fail the task and preserve usage_records/context for retry handling.backend/usage_v2/internal_views.py (1)
164-193:⚠️ Potential issue | 🟠 MajorValidate record shape before building
Usageobjects.Current permissive defaults (
usage_type,adapter_instance_id,model_name, etc.) can mask producer bugs, and one malformed row can fail the entire batch write.Proposed fix sketch
usage_objects = [] - for r in records: + for i, r in enumerate(records): + if not isinstance(r, dict): + return JsonResponse( + {"success": False, "error": f"Invalid record at index {i}"}, + status=status.HTTP_400_BAD_REQUEST, + ) + missing = [k for k in ("usage_type", "adapter_instance_id", "model_name") if k not in r] + if missing: + return JsonResponse( + { + "success": False, + "error": f"Missing required keys at index {i}: {', '.join(missing)}", + }, + status=status.HTTP_400_BAD_REQUEST, + ) usage_objects.append( Usage(🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/usage_v2/internal_views.py` around lines 164 - 193, The loop that constructs Usage objects from records (the for r in records -> Usage(...) block) is too permissive and can mask producer bugs; add explicit validation before building each Usage: verify required fields (e.g., workflow_id, execution_id, usage_type, adapter_instance_id or run_id as applicable) are present and of the expected type, coerce/validate numeric fields (embedding_tokens, prompt_tokens, completion_tokens, total_tokens as ints; cost_in_dollars as float), ensure model_name is non-empty when usage_type == "llm", and enforce the llm_usage_reason None/coercion rule; for records that fail validation, log a clear error including the offending record and either skip the record or collect/report them so the batch write isn't aborted by a single malformed row, then only append validated Usage instances to usage_objects.unstract/sdk1/src/unstract/sdk1/usage_handler.py (1)
108-112:⚠️ Potential issue | 🟠 MajorGuard
token_counterbefore dereference in embedding callback.
token_counteris nullable, but Line 111 and Line 154 assume it is always present. This can raiseAttributeErrorin production callback flows and skip usage persistence.Proposed fix
): if self.embed_model is None: return + if self.token_counter is None: + logger.warning( + "Embedding usage callback invoked without token_counter; skipping usage record" + ) + return model_name = self.embed_model.model_name embedding_tokens = self.token_counter.total_embedding_token_count @@ - self.token_counter.reset_counts() + self.token_counter.reset_counts()Also applies to: 154-154
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/src/unstract/sdk1/usage_handler.py` around lines 108 - 112, The code dereferences self.token_counter (accessing total_embedding_token_count) without checking for None, which can raise AttributeError and skip usage persistence; fix by guarding token_counter in the embedding callback(s): check if self.token_counter is not None before reading self.token_counter.total_embedding_token_count (use 0 or an appropriate default when None) and apply the same guard wherever total_embedding_token_count is accessed (e.g., the lines referencing self.token_counter at the embedding callback and at the later usage around line 154); ensure stream_log and any usage persistence code use the guarded/ default value instead of assuming token_counter exists.workers/ide_callback/tasks.py (2)
589-590:⚠️ Potential issue | 🟡 MinorNormalize
extracted_textwhen present-but-None.At Line 590,
len(extracted_text)can still fail ifdata["extracted_text"]is explicitlyNone.Proposed fix
- extracted_text = (result_dict.get("data") or {}).get("extracted_text", "") + data = result_dict.get("data") or {} + extracted_text = data.get("extracted_text") or "" token_count = len(extracted_text) // 4🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@workers/ide_callback/tasks.py` around lines 589 - 590, The code assigns extracted_text = (result_dict.get("data") or {}).get("extracted_text", "") but if data["extracted_text"] exists and is None len(extracted_text) will raise; normalize extracted_text to a string before computing token_count by coercing None to "" (e.g., replace None with "" for the extracted_text variable used in token_count calculation in tasks.py), then compute token_count = len(extracted_text) // 4 so len() is safe.
567-572:⚠️ Potential issue | 🟠 MajorTreat HTTP 404 from extraction persistence endpoints as terminal OSS no-op.
If cloud-only extraction endpoints are absent, persistence calls can 404 and currently flow into the outer exception path, then re-raise on Line 644. That breaks OSS-safe degradation for this callback path.
Proposed fix
@@ def extraction_complete( @@ - try: + def _is_http_404(exc: Exception) -> bool: + status_code = getattr(exc, "status_code", None) + if status_code is None: + status_code = getattr(getattr(exc, "response", None), "status_code", None) + return status_code == 404 + + try: @@ - api.mark_extraction_error( - source=source, - file_id=file_id, - error=error_msg, - organization_id=org_id, - ) + try: + api.mark_extraction_error( + source=source, + file_id=file_id, + error=error_msg, + organization_id=org_id, + ) + except Exception as e: + if _is_http_404(e): + logger.info( + "Skipping extraction_error persistence (404 endpoint missing): source=%s file=%s", + source, + file_id, + ) + return {"status": "ok", "note": "extraction callback endpoint not present (OSS)"} + raise @@ - api.mark_extraction_complete( - source=source, - file_id=file_id, - token_count=token_count, - extracted_text_path=extracted_text_path, - organization_id=org_id, - ) + try: + api.mark_extraction_complete( + source=source, + file_id=file_id, + token_count=token_count, + extracted_text_path=extracted_text_path, + organization_id=org_id, + ) + except Exception as e: + if _is_http_404(e): + logger.info( + "Skipping extraction_complete persistence (404 endpoint missing): source=%s file=%s", + source, + file_id, + ) + return {"status": "ok", "note": "extraction callback endpoint not present (OSS)"} + raiseAlso applies to: 592-598, 620-644
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@workers/ide_callback/tasks.py` around lines 567 - 572, The persistence calls to the API (e.g., api.mark_extraction_error called with source, file_id, organization_id/org_id and similar calls at the other locations) must treat an HTTP 404 from the extraction persistence endpoints as a terminal OSS no-op: wrap each api.* persistence call (api.mark_extraction_error, the matching api.mark_extraction_success/persist call(s) around lines ~592-598 and ~620-644) in a try/except that catches the HTTP error type your client raises (e.g., HTTPError/ApiError) and if the caught exception indicates response.status_code == 404, swallow it and return/exit this callback path instead of propagating; for any other exception re-raise so existing error handling still applies. Ensure you reference the same parameters (source, file_id, org_id/organization_id, error_msg, extracted_files, checksum) when locating and wrapping the calls.backend/prompt_studio/prompt_studio_core_v2/views.py (2)
923-926:⚠️ Potential issue | 🔴 Critical
prompt_studio_toolis still effectively caller-controlled on this detail route.Lines 923-926 only change which tool is used for the profile-count check. On Line 934,
self.perform_create(serializer)still persists anyprompt_studio_toolthe request supplied, so a caller with access to one tool can create a profile manager under another tool ID. Force the save againstself.get_object()here, or make the serializer field permission-scoped.Suggested fix
- prompt_studio_tool = ( - serializer.validated_data.get(ProfileManagerKeys.PROMPT_STUDIO_TOOL) - or self.get_object() - ) + prompt_studio_tool = self.get_object() @@ - self.perform_create(serializer) + serializer.save(prompt_studio_tool=prompt_studio_tool)Also applies to: 934-934
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_core_v2/views.py` around lines 923 - 926, The code currently allows a client-supplied prompt_studio_tool to be persisted because perform_create(serializer) uses serializer data; before calling self.perform_create(serializer) (in the view method surrounding get_object and serializer.validated_data), force the tool to the detail object by setting serializer.validated_data[ProfileManagerKeys.PROMPT_STUDIO_TOOL] = self.get_object() or call serializer.save(prompt_studio_tool=self.get_object()) so the persisted ProfileManager always uses self.get_object() instead of any caller-supplied value; alternatively, enforce the field is read-only/permission-scoped on the serializer to prevent client override.
99-110:⚠️ Potential issue | 🟠 MajorUse the requested execution path to decide whether to skip this gate.
Line 109 still keys the bypass off
custom_tool.single_pass_extraction_mode, sofetch_response()/bulk_fetch_response()can skip the non-single-pass lookup block just by calling those endpoints on a tool configured for single-pass mode. Pass an explicitis_single_passflag from each caller instead of inferring it from tool settings.Suggested direction
-def _multi_var_lookup_block_response(custom_tool, prompt_ids=None): +def _multi_var_lookup_block_response( + custom_tool, *, is_single_pass: bool, prompt_ids=None +): @@ - if getattr(custom_tool, "single_pass_extraction_mode", False): + if is_single_pass: return None- if err := _multi_var_lookup_block_response(custom_tool, prompt_ids=[prompt_id]): + if err := _multi_var_lookup_block_response( + custom_tool, is_single_pass=False, prompt_ids=[prompt_id] + ): return err- if err := _multi_var_lookup_block_response(custom_tool, prompt_ids=prompt_ids): + if err := _multi_var_lookup_block_response( + custom_tool, is_single_pass=False, prompt_ids=prompt_ids + ): return err🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_core_v2/views.py` around lines 99 - 110, The gate function _multi_var_lookup_block_response currently checks custom_tool.single_pass_extraction_mode directly; change its signature to accept an explicit is_single_pass boolean (e.g., def _multi_var_lookup_block_response(custom_tool, prompt_ids=None, is_single_pass=False)) and use that flag to decide skipping instead of getattr(custom_tool, "single_pass_extraction_mode", False). Update each caller (e.g., fetch_response and bulk_fetch_response) to pass the correct execution path by computing and passing is_single_pass from the request context/parameters rather than relying on tool settings. Ensure all call sites of _multi_var_lookup_block_response are updated to include the new parameter to avoid defaulting to the wrong behavior.backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py (1)
301-304:⚠️ Potential issue | 🟠 MajorValidate lookup assignments only for prompts that are actually exportable.
Validation is still executed on the full prompt list, but NOTES/inactive prompts are skipped during export. This can fail export for prompts that will never be emitted.
🔧 Suggested fix
- lookup_configs, lookup_error = validate_lookups_for_export(prompts) + lookup_validation_prompts = [ + prompt + for prompt in prompts + if prompt.prompt_type != JsonSchemaKey.NOTES and prompt.active + ] + lookup_configs, lookup_error = validate_lookups_for_export( + lookup_validation_prompts + ) if lookup_error: raise InValidCustomToolError(lookup_error)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py` around lines 301 - 304, The code calls validate_lookups_for_export(prompts) over the full prompts list which includes NOTES/inactive prompts and can block export for items that are never emitted; change the call to run only on the subset of exportable prompts (filter prompts to exclude NOTES/inactive entries before calling validate_lookups_for_export) so that validate_lookups_for_export only inspects prompts that will be exported and only raise InValidCustomToolError when those exportable prompts fail validation.backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py (1)
1180-1182:⚠️ Potential issue | 🟠 MajorSingle-pass lookup wiring is still missing in the legacy execution path.
This only updates
build_single_pass_payload().prompt_responder()still routes no-idflow through_execute_prompts_in_single_pass()→_fetch_single_pass_response(), wheretool_settings["lookup_configs"]is still not attached.🔧 Suggested follow-up
def _fetch_single_pass_response(...): @@ tool_settings[TSPKeys.SIMILARITY_TOP_K] = default_profile.similarity_top_k + + lookup_configs = get_lookup_configs_for_tool(tool, prompts=prompts) + if lookup_configs: + tool_settings["lookup_configs"] = lookup_configs🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py` around lines 1180 - 1182, The no-id legacy execution path in prompt_responder still fails to pass lookup configs into the single-pass flow, so ensure tool_settings["lookup_configs"] is populated for that path by calling get_lookup_configs_for_tool(tool, prompts=prompts) and assigning the result to tool_settings["lookup_configs"] before invoking _execute_prompts_in_single_pass() (and consequently _fetch_single_pass_response()); update prompt_responder to mirror the build_single_pass_payload() change by computing lookup_configs from prompts and attaching them to the tool_settings used in the no-id flow.backend/prompt_studio/prompt_studio_output_manager_v2/views.py (1)
74-96:⚠️ Potential issue | 🟠 MajorValidate
tool_idformat before ORM filters.A malformed
tool_idcan still bubble out as a server error instead of a clean 400. Add a UUID shape guard before both query paths usetool_id.🔧 Suggested fix
+from uuid import UUID @@ tool_id = request.GET.get("tool_id") @@ if not tool_id: raise ValidationError(detail=PromptOutputManagerErrorMessage.TOOL_VALIDATION) + try: + UUID(str(tool_id)) + except (TypeError, ValueError): + raise ValidationError( + detail=PromptOutputManagerErrorMessage.TOOL_VALIDATION + ) @@ tool_id = request.GET.get("tool_id") @@ if not tool_id: raise ValidationError(detail=tool_validation_message) + try: + UUID(str(tool_id)) + except (TypeError, ValueError): + raise ValidationError(detail=tool_validation_message)Please verify with an API test that
tool_id=not-a-uuidreturns HTTP 400 on both endpoints.Also applies to: 127-138
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_output_manager_v2/views.py` around lines 74 - 96, Add a UUID-shape guard that validates tool_id before any ORM usage: attempt to parse request.GET.get("tool_id") with uuid.UUID(...) in a try/except and if parsing fails raise ValidationError(detail=PromptOutputManagerErrorMessage.TOOL_VALIDATION). Apply this check in the current handler before building prompt_keys and before the other query path referenced in the comment (the block using ToolStudioPrompt.objects.filter and the similar block at lines 127-138), so no malformed tool_id reaches the ORM; keep using the same ValidationError and message to ensure the endpoint returns HTTP 400.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/prompt_studio/lookup_utils.py`:
- Around line 18-38: The ImportError guard misses the parent package name, so
when pluggable_apps is absent the exception is re-raised; update
_CLOUD_LOOKUP_MODULES to include "pluggable_apps" (in addition to the existing
entries) so the except block will treat missing the parent package as expected
and set LOOKUPS_AVAILABLE = False; ensure this change covers the import block
that imports execution/output_enrichment/staleness/validation and
LookupOutputResult (symbols: _CLOUD_LOOKUP_MODULES, _execution,
_output_enrichment, _staleness, _validation, _LookupOutputResult,
LOOKUPS_AVAILABLE, and the ImportError handling using e.name).
---
Duplicate comments:
In `@backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py`:
- Around line 1180-1182: The no-id legacy execution path in prompt_responder
still fails to pass lookup configs into the single-pass flow, so ensure
tool_settings["lookup_configs"] is populated for that path by calling
get_lookup_configs_for_tool(tool, prompts=prompts) and assigning the result to
tool_settings["lookup_configs"] before invoking
_execute_prompts_in_single_pass() (and consequently
_fetch_single_pass_response()); update prompt_responder to mirror the
build_single_pass_payload() change by computing lookup_configs from prompts and
attaching them to the tool_settings used in the no-id flow.
In `@backend/prompt_studio/prompt_studio_core_v2/views.py`:
- Around line 923-926: The code currently allows a client-supplied
prompt_studio_tool to be persisted because perform_create(serializer) uses
serializer data; before calling self.perform_create(serializer) (in the view
method surrounding get_object and serializer.validated_data), force the tool to
the detail object by setting
serializer.validated_data[ProfileManagerKeys.PROMPT_STUDIO_TOOL] =
self.get_object() or call serializer.save(prompt_studio_tool=self.get_object())
so the persisted ProfileManager always uses self.get_object() instead of any
caller-supplied value; alternatively, enforce the field is
read-only/permission-scoped on the serializer to prevent client override.
- Around line 99-110: The gate function _multi_var_lookup_block_response
currently checks custom_tool.single_pass_extraction_mode directly; change its
signature to accept an explicit is_single_pass boolean (e.g., def
_multi_var_lookup_block_response(custom_tool, prompt_ids=None,
is_single_pass=False)) and use that flag to decide skipping instead of
getattr(custom_tool, "single_pass_extraction_mode", False). Update each caller
(e.g., fetch_response and bulk_fetch_response) to pass the correct execution
path by computing and passing is_single_pass from the request context/parameters
rather than relying on tool settings. Ensure all call sites of
_multi_var_lookup_block_response are updated to include the new parameter to
avoid defaulting to the wrong behavior.
In `@backend/prompt_studio/prompt_studio_output_manager_v2/views.py`:
- Around line 74-96: Add a UUID-shape guard that validates tool_id before any
ORM usage: attempt to parse request.GET.get("tool_id") with uuid.UUID(...) in a
try/except and if parsing fails raise
ValidationError(detail=PromptOutputManagerErrorMessage.TOOL_VALIDATION). Apply
this check in the current handler before building prompt_keys and before the
other query path referenced in the comment (the block using
ToolStudioPrompt.objects.filter and the similar block at lines 127-138), so no
malformed tool_id reaches the ORM; keep using the same ValidationError and
message to ensure the endpoint returns HTTP 400.
In
`@backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py`:
- Around line 301-304: The code calls validate_lookups_for_export(prompts) over
the full prompts list which includes NOTES/inactive prompts and can block export
for items that are never emitted; change the call to run only on the subset of
exportable prompts (filter prompts to exclude NOTES/inactive entries before
calling validate_lookups_for_export) so that validate_lookups_for_export only
inspects prompts that will be exported and only raise InValidCustomToolError
when those exportable prompts fail validation.
In `@backend/usage_v2/internal_views.py`:
- Around line 164-193: The loop that constructs Usage objects from records (the
for r in records -> Usage(...) block) is too permissive and can mask producer
bugs; add explicit validation before building each Usage: verify required fields
(e.g., workflow_id, execution_id, usage_type, adapter_instance_id or run_id as
applicable) are present and of the expected type, coerce/validate numeric fields
(embedding_tokens, prompt_tokens, completion_tokens, total_tokens as ints;
cost_in_dollars as float), ensure model_name is non-empty when usage_type ==
"llm", and enforce the llm_usage_reason None/coercion rule; for records that
fail validation, log a clear error including the offending record and either
skip the record or collect/report them so the batch write isn't aborted by a
single malformed row, then only append validated Usage instances to
usage_objects.
In `@unstract/sdk1/src/unstract/sdk1/usage_handler.py`:
- Around line 108-112: The code dereferences self.token_counter (accessing
total_embedding_token_count) without checking for None, which can raise
AttributeError and skip usage persistence; fix by guarding token_counter in the
embedding callback(s): check if self.token_counter is not None before reading
self.token_counter.total_embedding_token_count (use 0 or an appropriate default
when None) and apply the same guard wherever total_embedding_token_count is
accessed (e.g., the lines referencing self.token_counter at the embedding
callback and at the later usage around line 154); ensure stream_log and any
usage persistence code use the guarded/ default value instead of assuming
token_counter exists.
In `@workers/executor/tasks.py`:
- Around line 113-130: The current flush path in tasks.py logs failures from
bulk_create_usage but swallows them, risking dropped billing rows; modify the
failure handling inside the block that calls bulk_create_usage so that on a
non-ok result or on exception you propagate an error to the caller (either by
re-raising the caught exception or raising a RecoverableError) instead of only
logging, so the orchestration can retry; keep the existing logger.error messages
(including run_id and organization_id) but follow them with a raise to fail the
task and preserve usage_records/context for retry handling.
In `@workers/ide_callback/tasks.py`:
- Around line 589-590: The code assigns extracted_text =
(result_dict.get("data") or {}).get("extracted_text", "") but if
data["extracted_text"] exists and is None len(extracted_text) will raise;
normalize extracted_text to a string before computing token_count by coercing
None to "" (e.g., replace None with "" for the extracted_text variable used in
token_count calculation in tasks.py), then compute token_count =
len(extracted_text) // 4 so len() is safe.
- Around line 567-572: The persistence calls to the API (e.g.,
api.mark_extraction_error called with source, file_id, organization_id/org_id
and similar calls at the other locations) must treat an HTTP 404 from the
extraction persistence endpoints as a terminal OSS no-op: wrap each api.*
persistence call (api.mark_extraction_error, the matching
api.mark_extraction_success/persist call(s) around lines ~592-598 and ~620-644)
in a try/except that catches the HTTP error type your client raises (e.g.,
HTTPError/ApiError) and if the caught exception indicates response.status_code
== 404, swallow it and return/exit this callback path instead of propagating;
for any other exception re-raise so existing error handling still applies.
Ensure you reference the same parameters (source, file_id,
org_id/organization_id, error_msg, extracted_files, checksum) when locating and
wrapping the calls.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ce8433f8-3196-4387-824b-9de175c39622
📒 Files selected for processing (20)
.gitignorebackend/prompt_studio/lookup_utils.pybackend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.pybackend/prompt_studio/prompt_studio_core_v2/views.pybackend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.pybackend/prompt_studio/prompt_studio_output_manager_v2/serializers.pybackend/prompt_studio/prompt_studio_output_manager_v2/views.pybackend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.pybackend/usage_v2/internal_views.pybackend/usage_v2/migrations/0006_alter_usage_status_and_more.pybackend/usage_v2/models.pyfrontend/src/components/custom-tools/prompt-card/PromptOutput.jsxfrontend/src/hooks/usePromptOutput.jsunstract/sdk1/src/unstract/sdk1/llm.pyunstract/sdk1/src/unstract/sdk1/usage_handler.pyworkers/executor/executors/legacy_executor.pyworkers/executor/executors/plugins/protocols.pyworkers/executor/tasks.pyworkers/ide_callback/tasks.pyworkers/shared/clients/usage_client.py
✅ Files skipped from review due to trivial changes (2)
- backend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.py
- workers/executor/executors/legacy_executor.py
🚧 Files skipped from review as they are similar to previous changes (1)
- .gitignore
Five threads on PR #1929 raised against the latest disposition push: * prompt_studio_core_v2/views.py: drop the ``single_pass_extraction_mode`` bypass from ``_multi_var_lookup_block_response`` — fetch_response / bulk_fetch_response are always non-SP, so the tool-setting check just let multi-var lookups slip past the gate when the tool happened to be configured for SP. * prompt_studio_registry_helper.py: filter out NOTES + inactive prompts *before* calling ``validate_lookups_for_export`` so an incomplete lookup on a non-exportable prompt no longer fails the whole export. * unstract/sdk1/usage_handler.py: guard ``self.token_counter is None`` in the embedding-end branch — degrade with a warning instead of an AttributeError on early callbacks. * workers/executor/legacy_executor.py: move the ``outcome.usage_records`` / ``outcome.llm_metrics`` access inside the ``try`` so plugin contract drift hits the same graceful-degrade branch as a thrown ``run_with_metrics``. * backend/prompt_studio/lookup_utils.py: include ``pluggable_apps`` itself in ``_CLOUD_LOOKUP_MODULES``. Pure OSS images don't have the parent package, so ``ImportError.name`` surfaces as ``"pluggable_apps"`` and the previous filter re-raised instead of setting LOOKUPS_AVAILABLE=False.
Contributor
Author
Round-2 dispositionOSS commit
🤖 Generated with Claude Code |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
The I'll resolve all my open threads now. [resolve] |
Contributor
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
athul-rs
reviewed
Apr 30, 2026
Contributor
athul-rs
left a comment
athul-rs
left a comment
There was a problem hiding this comment.
A few nits from a follow-up pass — non-blocking.
Contributor
|
nit (couldn't anchor inline — line is unchanged context): the docstring at |
athul-rs
approved these changes
Apr 30, 2026
Contributor
athul-rs
left a comment
athul-rs
left a comment
There was a problem hiding this comment.
@chandrasekharan-zipstack please chekc the NIt comments added
Drop archaeology / "previously / before-X-now-Y" framing; collapse multi-line WHAT walkthroughs to single-line WHY. No logic changes.
…ient Pairs with the cloud-side removal of the lookup token_count / estimated_tokens fields — the worker no longer computes a value to send.
…ration - UsageBatchCreateView raises DRF exceptions (drf-standardized-errors envelope) instead of hand-rolled JsonResponse — serializer validation via raise_exception=True, dedicated UsagePersistError(APIException) for the bulk_create failure path. - Records validated through UsageBatchCreateSerializer / nested UsageRecordCreateSerializer so adapter_instance_id, model_name, usage_type are required and the rest get explicit defaults. - Fold 0006_alter_usage_status_and_more (UsageStatus choices + reference_pair CheckConstraint) into 0004 — branch hasn't merged to main, so squashing avoids an extra ALTER on deploy.
…helpers
- llm.py: token_counter fallback when prompt_tokens=0; rsplit('/',1) for
multi-segment provider IDs; spread _usage_kwargs first so explicit
billing fields win.
- utils/common.py: stamp every record appended during the call window
(was clobbering only the last entry).
- legacy_executor.py: extract run_lookup_enrichment / run_webhook_postprocessing
/ is_blank into workers/executor/executors/lookup_enrichment.py — caller
passes shim/state in, plugin returns usage_records for the caller to
extend its billing batch. Orchestrator stays pure dispatch.
- ide_callback/tasks.py: drop the char-÷4 token estimate heuristic;
context-manage the API clients so HTTP sessions don't leak.
- prompt_studio/views.py: _multi_var_lookup_block_response uses
``detail`` instead of ``error`` to match drf-standardized-errors.
- CombinedOutput.jsx: hoist build helpers to module scope (selectedProfile
passed as arg) — no per-render allocation, fewer useEffect closures.
- DocumentParser.jsx: hoist UUID_RE to module scope (no per-render compile).
- PromptOutput.jsx: silent ``catch {}`` on plugin dynamic-import failures
so OSS doesn't surface noisy warnings for cloud-only modules.
- SideNavBar.jsx: hide the lookups submenu item when the lookup-studio
plugin isn't loaded (keeps OSS nav clean).
…izer Both columns are UUIDField on the Usage model — leaving them as CharField in the serializer let invalid UUIDs slip through to bulk_create and surface as a 500. UUIDField catches them at validation with a standard DRF 400.
Drops the stale "Computes token count" line — the callback no longer derives a token count. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If _validate_file_execution_id raised, the except handler hit UnboundLocalError and masked the original ValueError. Pre-bind a str(file_execution_id) fallback so the error response carries the real cause. Also gitignore Codex's AGENTS.md scratchpad. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jaseemjaskp
approved these changes
May 7, 2026
Contributor
jaseemjaskp
left a comment
jaseemjaskp
left a comment
There was a problem hiding this comment.
Re-reviewed with the multi-agent toolkit (Code Reviewer, Silent Failure Hunter, Type Design Analyzer, Comment Analyzer, PR Test Analyzer) against the latest HEAD. All previously-raised concerns from my prior pass have been addressed in subsequent commits — the salvage block, billing-flush ERROR promotion, JSON-parse webhook gate, broad-except narrowing, run_id/UUID coercion, and lookup helper extraction all look good. New agent findings either duplicate already-posted reviewer threads or fall below the bar I'm holding for this round (test coverage gaps and stricter serializer choice validation are worth follow-ups but don't block the merge). Approving.
…olumns + post-write hooks Drop reference_id / reference_type from Usage in favour of typed project_id / prompt_id columns indexed CONCURRENTLY for dashboard rollups. Cloud-only attribution (e.g. lookup_version_id) now flows through an opaque cloud_extras carrier on the batch endpoint, which forwards it verbatim to plugin-registered post-write hooks invoked inside the same atomic transaction as bulk_create — a hook failure rolls back the Usage rows so attribution stays consistent or nothing is written. Removes the need for cloud to subclass UsageBatchCreateView or prepend a URL override; the hook seam is generic for any future cloud feature.
Contributor
Frontend Lint Report (Biome)✅ All checks passed! No linting or formatting issues found. |
Contributor
Test ResultsSummary
Runner Tests - Full Report
SDK1 Tests - Full Report
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
OSS-side glue for Lookups V2 — a cloud-only Prompt Studio feature that runs an LLM-powered post-extraction enrichment over a prompt's structured output. This PR is the OSS half: a try-import bridge module, executor hook for non-SP runs, IDE wiring for the Raw/Enriched tabs and gating modals, and a few small schema extensions.
The cloud-side PR (where the feature actually lives) is on the
unstract-cloudrepo, same branch name, same ticket: https://github.com/Zipstack/unstract-cloud/pull/1463.Why
The cloud plugin needs to integrate with extraction (executor), with the per-prompt output flow (
prompt_studio_helper,prompt_studio_output_manager_v2), with export (prompt_studio_registry_helper), and with the IDE UI (Raw/Enriched tabs, gate modal, dirty-seed banner). All of that has to live in OSS — but OSS must never import cloud schema directly. This PR is that boundary.How
backend/prompt_studio/lookup_utils.pytry-imports the cloud plugin and exposes ~10 stable, opaque functions. Every function is a no-op in OSS. Callers don't know cloud key names or model shapes.legacy_executor._run_lookup_enrichmentresolves the worker plugin viaExecutorPluginLoader.get("lookup-enrichment")and runs after each prompt's extraction. Skips with a workflow-log entry when the source prompt extracted nothing.fetch_response/bulk_fetch_response, scoped to the prompt set being run.lookup-validationaction onCustomToolViewreturns the validation buckets that the FE modal renders.CustomTool.last_exported_at(new field) plusis_lookup_dirtyincheck_deployment_usagepowers the "needs re-export" banner.Usagemodel adds a few cloud-extensible enum choices (reference_type,llm_usage_reason="lookup",status,error_message,execution_time_ms).Reviewer aid
A condensed one-page primer for human reviewers — system shape, data model, execution diagrams, gates, smoke checklist — is at
lookups-v2-reviewer-refresher.html(attached, same file on both PRs). Read it before opening the diff.The full KB lives in the reviewer's vault at
~/Documents/Obsidian Vault/zipstuff/lookups/.Main files to review (sorted by significance)
The bridge — central to the OSS↔cloud contract
backend/prompt_studio/lookup_utils.py— every cloud call funnels through here. Tiny, opaque, no-op in OSS.Executor — non-SP entry
2.
workers/executor/executors/legacy_executor.py—_run_lookup_enrichmentdispatches the worker plugin; new "skip-with-log" branch when extraction yieldsNone.3.
workers/executor/executor_tool_shim.py—stream_logrouting (confirms the user-facing path for the new skip log).Prompt Studio integration
4.
backend/prompt_studio/prompt_studio_core_v2/views.py— multi-var pre-flight,lookup-validationaction,is_lookup_dirtyincheck_deployment_usage.5.
backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py— three call sites that attachlookup_configto per-prompt output / tool settings.6.
backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py— export-time validation + per-prompt config attach.7.
backend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.py— persists lookup output via the bridge after enrichment.8.
backend/prompt_studio/prompt_studio_core_v2/models.py— addsCustomTool.last_exported_at.Usage / observability
9.
backend/usage_v2/models.py— extensibleLLM_USAGE_REASON_CHOICES/REFERENCE_TYPE_CHOICESvia try-import; new fields for lookup attribution.Frontend wiring
10.
frontend/src/components/custom-tools/prompt-card/PromptOutput.jsx— Raw vs Enriched tabs.11.
frontend/src/components/custom-tools/header/Header.jsx— wiresuseLookupExportGate(try-imported, no-op in OSS).12.
frontend/src/components/custom-tools/tool-ide/ToolIde.jsx— wiresuseLookupDirtySeed.13.
frontend/src/components/custom-tools/combined-output/{CombinedOutput,JsonView}.jsx— combined-output enrichment passthrough.Can this PR break any existing features?
No production breakage expected. All cloud calls flow through
lookup_utilsand degrade toNone/[]/{ok: True}in OSS, so existing flows behave identically when no lookup is assigned or when the cloud plugin isn't installed.Watch areas:
_run_lookup_enrichmentruns after every prompt in the non-SP path. Whenlookup_configisNone(no assignment) it returns immediately — should be invisible to non-lookup users.metadata["lookup_outputs"]is a new key on prompt output. FE consumers reflecting the fullmetadatashape need to ignore unknown keys.Usagetable now sees rows withreference_type="lookup_version"; dashboards should treat these as additional rows.legacy_executor.pyskip log usesshim.stream_logwhich routes throughLogPublisherto the workflow log UI — verified path inexecutor_tool_shim.py.Database Migrations
usage_v2/0005_usage_reason_ref_created_idxusage_v2/0006_alter_usage_llm_usage_reason_and_moreprompt_studio_core_v2/0007_customtool_last_exported_atEnv Config
None new.
Relevant Docs
~/Documents/Obsidian Vault/zipstuff/lookups/.lookups-v2-reviewer-refresher.html.Related Issues or PRs
Dependencies Versions
No new dependencies.
Notes on Testing
Smoke checklist in the reviewer refresher. The OSS-side hooks and bridge can be exercised against a cloud build; in pure OSS, every lookup-related code path is exercised but no-ops out via
LOOKUPS_AVAILABLE = False.Screenshots
UI screenshots live on the cloud PR (the OSS-side FE changes are wiring only).
Checklist
I have read and understood the Contribution Guidelines.