Skip to content

feat: SmartHopper 2.0.0 - Google Gemini Provider, Batch API, Vision Input, and Breaking Changes#420

Open
marc-romu wants to merge 181 commits into
devfrom
feature/2.0.0-text2json
Open

feat: SmartHopper 2.0.0 - Google Gemini Provider, Batch API, Vision Input, and Breaking Changes#420
marc-romu wants to merge 181 commits into
devfrom
feature/2.0.0-text2json

Conversation

@marc-romu
Copy link
Copy Markdown
Member

@marc-romu marc-romu commented Apr 2, 2026 •
edited
Loading

feat: SmartHopper 2.0.0 - Google Gemini Provider, Batch API, Vision Input, and Breaking Changes

Description

This PR introduces SmartHopper 2.0.0, a major release with significant new features, architectural improvements, and breaking changes. This is a large-scale refactor spanning 490 files with >45,000 insertions and 7,500 deletions.

Key Features Added

🚀 Google Gemini Provider (Full Integration)

  • Complete provider implementation with support for Gemini 3.1, 2.5, 2.0, and 1.5 models
  • Text generation with streaming support
  • Image generation capabilities
  • Structured outputs with JSON Schema support (Gemini subset)
  • Tool calling with function declarations
  • Extended thinking/reasoning with configurable thinking levels
  • Batch processing with priority support
  • Context caching infrastructure
  • x-goog-api-key authentication in centralized CallApi method

⚡ Mixed-Type Data Tree Support

  • Infrastructure for handling heterogeneous Grasshopper data types (GH_Boolean, GH_String, etc.)
  • New GHStructureConverter utility for converting typed structures to IGH_Goo
  • Enhanced DataTreeProcessor with IGH_Goo type gate for branch grouping
  • ProcessingResult<T> classes for returning output trees and messages
  • Migrated AIText2BooleanComponent and AIList2BooleanComponent to mixed-type pipelines

📊 Batch API Support (OpenAI, Anthropic, MistralAI, Gemini)

  • IAIBatchProvider interface implementation across three providers
  • Multi-item batch submission with custom ID tracking
  • Live progress counter during batch processing
  • Batch state persistence across file save/reopen
  • Per-item error surfacing to Grasshopper runtime messages
  • Configurable HTTP timeouts for large file operations
  • BatchTier parameter in AIRequestParameters (replaces service_tier extra)

🖼️ Vision Input Support

  • Image-to-text capabilities via img2text AI tool
  • AIInteractionImage with vision input methods and MimeType property
  • GH_ExtractedImage Goo type for Grasshopper integration
  • AIImgToTextComponent for standalone image description
  • AIFile2MdComponent with image modes: embed, describe, caption
  • Provider-specific vision encoding (OpenAI, Anthropic, MistralAI)

📄 File-to-Markdown Conversion

  • file2md AI tool supporting 12 formats: PDF, DOCX, XLSX, PPTX, HTML, CSV, JSON, XML, TXT, EML, EPUB, RTF
  • PDF layout intelligence with column detection, table detection, caption recognition
  • Image extraction from PDF, DOCX, and PPTX
  • File2MdComponent (non-AI) and AIFile2MdComponent (AI-powered)

🌐 Web-to-Markdown Conversion

  • web2md AI tool (replaces web_generic_page_read)
  • Specialized handlers for Wikipedia, GitHub, GitLab, Discourse, Stack Exchange
  • Readability scoring with boilerplate removal

⚙️ AI Settings Components

  • AISettingsComponent: Universal settings component for cross-provider parameters
  • AIExtraSettingsComponent: Dynamic inputs based on provider-specific extras
  • AIRequestParameters immutable record with fluent builder
  • GH_AIRequestParameters Grasshopper wrapper with backward-compatible string casting
  • Per-property resolution in all providers (input settings override global settings)

🛠️ JSON Tools & Components

  • text2json AI tool for structured JSON generation from prompts
  • AIText2JsonComponent for AI-powered JSON creation
  • JSON Helper Components: JsonSchemaComponent, JsonObjectComponent, JsonArrayComponent, JsonArray2TextListComponent, JsonObject2TextComponent, JsonGetValueComponent, JsonMergeComponent
  • Visual JSON Schema Builder: JsonSchemaPropComponent, JsonSchemaPropObjectComponent, JsonSchemaPropArrayComponent

Breaking Changes

There is a new "Timeout" input in the "AI Settings" component. This means that files saved in previous versions may need to reconnect inputs.

⚠️ AI Tool Renames (Internal change - code migration required)

Old Name New Name
text_generate text2text
text_evaluate text2boolean
list_generate text2textlist
list_evaluate textlist2boolean
img_generate text2img
img_to_text img2text
web_to_md web2md
web_generic_page_read REMOVED (use web2md)

⚠️ Component Renames (Migration Required)

The components listed in this table that were saved in previous files will not work with SmartHopper 2.0 or above. You will need to manually replace them with their equivalent new component.

Old Class Name New Class Name
AITextGenerate AIText2TextComponent
AITextEvaluate AIText2BooleanComponent
AITextListGenerate AIText2TextListComponent
AIListEvaluate AIList2BooleanComponent
AIImgGenerateComponent AIText2ImgComponent
AIImgToTextComponent AIImg2TextComponent
WebPageReadComponent REMOVED (use Web2MdComponent)

Testing Done

Comprehensive testing plan created in docs/Reviews/260402-PR-Testing-Plan.md covering 202 test cases across 11 feature areas.

🔴 P0 - Breaking Changes (15 tests)

  • TC-BREAK-01: Open existing .gh files with old components -> Renamed components need to replace
  • TC-BREAK-08: Verify WebPageReadComponent removal doesn't crash file load
  • TC-BREAK-09: Test AI tools are accessible by new names in chat/tool calls
  • TC-BREAK-10: Verify old tool names return appropriate errors (not silent failures)
  • TC-BREAK-11: Check component outputs maintain same structure after rename
  • TC-BREAK-12: Verify Grasshopper wire connections preserved after component rename -> NO
  • TC-BREAK-13: Verify service_tier=batch in existing files is silently ignored
  • TC-BREAK-14: Test that Batch input on AISettingsComponent works correctly

🔴 P0 - Google Gemini Provider (19 tests)

  • TC-GEMINI-01: Install and enable Google Gemini provider on clean install
  • TC-GEMINI-02: Configure API key in settings - verify secure storage
  • TC-GEMINI-03: Test provider appears in provider list with correct icon
  • TC-GEMINI-06: Test Gemini 2.0 Flash - text generation
  • TC-GEMINI-07: Test Gemini 2.5 Pro - text generation with reasoning
  • TC-GEMINI-08: Test Gemini 3.1 - text generation
  • TC-GEMINI-09: Test image generation models (if available)
  • TC-GEMINI-10: Verify all models appear in model selection dropdown with correct capabilities
  • TC-GEMINI-11: Text generation with streaming
  • TC-GEMINI-12: Structured output (JSON Schema)
  • TC-GEMINI-13: Tool calling with function declarations
  • TC-GEMINI-14: Extended thinking/reasoning levels
  • TC-GEMINI-15: Context caching (verify infrastructure in place)
  • TC-GEMINI-16: Test with AIText2TextComponent
  • TC-GEMINI-17: Test with AIText2BooleanComponent
  • TC-GEMINI-18: Test with AIText2JsonComponent
  • TC-GEMINI-19: Verify batch processing works with Gemini

🔴 P0 - Mixed-Type Data Trees (17 tests)

  • TC-DATATREE-01: Convert GH_Structure<GH_String> to GH_Structure<IGH_Goo>
  • TC-DATATREE-02: Convert GH_Structure<GH_Boolean> to GH_Structure<IGH_Goo>
  • TC-DATATREE-03: Convert GH_Structure<GH_Integer> to GH_Structure<IGH_Goo>
  • TC-DATATREE-04: Convert GH_Structure<GH_Number> to GH_Structure<IGH_Goo>
  • TC-DATATREE-05: Handle mixed-type trees with multiple IGH_Goo types
  • TC-DATATREE-06: Test groupIdenticalBranches with IGH_Goo type gate
  • TC-DATATREE-07: Verify branch grouping works with heterogeneous data
  • TC-DATATREE-08: Test RunAsync<T> overload for heterogeneous output
  • TC-DATATREE-09: Test ExtractTypedTree<U> helper method
  • TC-DATATREE-10: AIText2BooleanComponent - mixed-type input tree with GH_Boolean fallback
  • TC-DATATREE-11: AIList2BooleanComponent - mixed-type input tree with GH_Boolean fallback
  • TC-DATATREE-12: Verify fallback values stored natively without string conversion
  • TC-DATATREE-13: Test ProcessingResult<IGH_Goo>.Outputs access
  • TC-DATATREE-14: Test ExtractTypedTree<GH_String>() from heterogeneous results
  • TC-DATATREE-15: Test RunProcessingAsync<GH_String> with tree broadcasting
  • TC-DATATREE-16: Verify ItemGraft path management consistency
  • TC-DATATREE-17: Test ComponentProcessingOptions property behavior

🔴 P0 - Batch API (26 tests)

  • TC-BATCH-01: Submit multi-request JSONL file to /v1/files
  • TC-BATCH-02: Create batch via /v1/batches
  • TC-BATCH-03: Poll batch status with live progress counter
  • TC-BATCH-04: Download output from /v1/files/{output_file_id}/content
  • TC-BATCH-05: Cancel batch via /v1/batches/{id}/cancel
  • TC-BATCH-06: Verify request_counts.completed updates progress
  • TC-BATCH-07: Submit multiple items via POST /v1/messages/batches
  • TC-BATCH-08: Poll processing_status on GET /v1/messages/batches/{id}
  • TC-BATCH-09: Download JSONL results from results_url
  • TC-BATCH-10: Cancel via POST /v1/messages/batches/{id}/cancel
  • TC-BATCH-11: Verify request_counts.succeeded updates progress
  • TC-BATCH-12: Submit inline batching via POST /v1/batch/jobs
  • TC-BATCH-13: Poll job status on GET /v1/batch/jobs/{id}
  • TC-BATCH-14: Download output from /v1/files/{output_file}/content
  • TC-BATCH-15: Cancel via POST /v1/batch/jobs/{id}/cancel
  • TC-BATCH-16: Verify succeeded_requests updates progress
  • TC-BATCH-17: Test AITextGenerate batch completion with OnBatchCompleted override
  • TC-BATCH-18: Verify ReconstructOutputTree<T> replaces sentinels correctly
  • TC-BATCH-19: Test batch state persistence across file save/reopen
  • TC-BATCH-20: Verify CustomIds serialization in Write() / Read()
  • TC-BATCH-21: Test immediate first poll when restoring batch state
  • TC-BATCH-22: Verify batch capability validation prevents unsupported providers
  • TC-BATCH-23: Test batch item errors surface as Grasshopper runtime messages
  • TC-BATCH-24: Verify AIInteractionError detection in provider Decode() methods
  • TC-BATCH-25: Test large file upload/download timeout (300s default)
  • TC-BATCH-26: Verify progress messages show Preparing X/X... during data collection

🟡 P1 - Vision Input (25 tests)

  • TC-VISION-01: Create vision input from base64 data
  • TC-VISION-02: Create vision input from file path
  • TC-VISION-03: Verify MimeType property is correctly set
  • TC-VISION-04: Test CreateVisionInput() method
  • TC-VISION-05: Test CreateVisionInputFromBase64() method
  • TC-VISION-06: Test AddImageInput() fluent method
  • TC-VISION-07: Test AddImageInputFromBase64() fluent method
  • TC-VISION-08: OpenAI - verify base64 data URI encoding (data:{mime};base64,{data})
  • TC-VISION-09: Anthropic - native image content blocks with base64
  • TC-VISION-10: MistralAI - OpenAI-compatible image_url content blocks
  • TC-VISION-11: Test image description with imageUrl parameter
  • TC-VISION-12: Test image description with imageBase64 + mimeType
  • TC-VISION-13: Test with optional prompt parameter
  • TC-VISION-14: Verify AICapability.Image2Text requirement enforcement
  • TC-VISION-15: AIImgToTextComponent - file path input
  • TC-VISION-16: AIImgToTextComponent - URL input
  • TC-VISION-17: AIImgToTextComponent - base64 input
  • TC-VISION-18: AIImgToTextComponent - GH_ExtractedImage input (regression test for MistralAI fix)
  • TC-VISION-19: AIFile2MdComponent - image mode embed
  • TC-VISION-20: AIFile2MdComponent - image mode describe
  • TC-VISION-21: AIFile2MdComponent - image mode caption
  • TC-VISION-22: Verify ScriptVariable() returns Bitmap
  • TC-VISION-23: Test cast to/from GH_String
  • TC-VISION-24: Test cast to/from Bitmap
  • TC-VISION-25: Verify full serialization in GH files

🟡 P1 - File-to-Markdown (32 tests)

  • TC-F2MD-01: PDF conversion with layout intelligence
  • TC-F2MD-02: DOCX conversion (headings, tables, lists)
  • TC-F2MD-03: XLSX conversion
  • TC-F2MD-04: PPTX conversion
  • TC-F2MD-05: HTML conversion
  • TC-F2MD-06: CSV conversion
  • TC-F2MD-07: JSON conversion
  • TC-F2MD-08: XML conversion
  • TC-F2MD-09: TXT conversion
  • TC-F2MD-10: EML conversion
  • TC-F2MD-11: EPUB conversion
  • TC-F2MD-12: RTF conversion
  • TC-F2MD-13: Column detection with RecursiveXYCut
  • TC-F2MD-14: Reading order preservation
  • TC-F2MD-15: Header/footer removal (12%/88% zones)
  • TC-F2MD-16: Heading detection
  • TC-F2MD-17: Table detection via whitespace-gap analysis
  • TC-F2MD-18: Inline text styling preservation (bold, italic)
  • TC-F2MD-19: Caption detection for tables and figures
  • TC-F2MD-20: PDF image extraction via PdfPig
  • TC-F2MD-21: DOCX image extraction from ImageParts
  • TC-F2MD-22: PPTX image extraction from SlidePart.ImageParts
  • TC-F2MD-23: PNG conversion with TryGetPng
  • TC-F2MD-24: JPEG magic-byte fallback
  • TC-F2MD-25: File2MdComponent - basic conversion
  • TC-F2MD-26: File2MdComponent - Images output with GH_ExtractedImage
  • TC-F2MD-27: AIFile2MdComponent - AI-powered conversion
  • TC-F2MD-28: AIFile2MdComponent - image modes (embed, describe, caption)
  • TC-F2MD-29: AIFile2MdComponent - batch context persistence across save/reload
  • TC-F2MD-30: Test describeImages parameter
  • TC-F2MD-31: Verify images array always returned in result
  • TC-F2MD-32: Test imageMode parameter (embed, describe, caption)

🟡 P1 - Web-to-Markdown (12 tests)

  • TC-W2MD-01: Wikipedia article conversion
  • TC-W2MD-02: GitHub file URL conversion (raw/plain/markdown)
  • TC-W2MD-03: GitLab file URL conversion
  • TC-W2MD-04: Discourse forum conversion
  • TC-W2MD-05: Stack Exchange question conversion
  • TC-W2MD-06: Generic page fallback with readability scoring
  • TC-W2MD-07: Content scoring by text density
  • TC-W2MD-08: Link density filtering
  • TC-W2MD-09: Boilerplate removal (nav, header, footer, ads)
  • TC-W2MD-10: Semantic container prioritization (article, main)
  • TC-W2MD-11: Web2MdComponent - basic URL conversion
  • TC-W2MD-12: Verify web_generic_page_read removal - use web2md instead

🟡 P1 - AI Settings Components (23 tests)

  • TC-SETTINGS-01: Assemble AIRequestParameters from inputs
  • TC-SETTINGS-02: Test Model input (string)
  • TC-SETTINGS-03: Test Temperature input (number)
  • TC-SETTINGS-04: Test Max Tokens input (integer)
  • TC-SETTINGS-05: Test Top P input (number)
  • TC-SETTINGS-06: Test Seed input (integer)
  • TC-SETTINGS-07: Test Extras JSON input
  • TC-SETTINGS-08: Verify output Settings (S) wire connects to AI components
  • TC-SETTINGS-09: Test Batch (B) boolean input
  • TC-SETTINGS-10: Dynamic inputs rebuild on provider change
  • TC-SETTINGS-11: Test with OpenAI provider extras
  • TC-SETTINGS-12: Test with Anthropic provider extras
  • TC-SETTINGS-13: Test with MistralAI provider extras
  • TC-SETTINGS-14: Test with DeepSeek provider extras
  • TC-SETTINGS-15: Test with OpenRouter provider extras
  • TC-SETTINGS-16: Verify wire connections preserved on provider change
  • TC-SETTINGS-17: Test immutable record behavior
  • TC-SETTINGS-18: Test AIRequestParametersBuilder fluent methods
  • TC-SETTINGS-19: Test WithBatchTier() / ClearBatchTier()
  • TC-SETTINGS-20: Test serialization in GH_AIRequestParameters
  • TC-SETTINGS-21: Backward compatibility - plain string model name
  • TC-SETTINGS-22: Settings flow through to provider Encode()
  • TC-SETTINGS-23: Per-property resolution priority (input > global settings)

🟡 P1 - JSON Tools (24 tests)

  • TC-JSON-01: Generate JSON from prompt with JSON Schema
  • TC-JSON-02: Test prompt parameter (required)
  • TC-JSON-03: Test instructions parameter (optional)
  • TC-JSON-04: Test jsonSchema parameter (required)
  • TC-JSON-05: Verify AICapability.TextInput | JsonOutput enforcement
  • TC-JSON-06: Validate output conforms to provided schema
  • TC-JSON-07: Grasshopper component in SmartHopper > JSON category
  • TC-JSON-08: Test Prompt input (tree)
  • TC-JSON-09: Test Instructions input (tree, optional)
  • TC-JSON-10: Test Schema input (tree)
  • TC-JSON-11: Verify JSON output (text tree)
  • TC-JSON-12: Test ItemGraft + GroupIdenticalBranches processing topology
  • TC-JSON-13: JsonSchemaComponent - build schema from properties
  • TC-JSON-14: JsonSchemaComponent - nested properties via dot-notation
  • TC-JSON-15: JsonObjectComponent - create JSON from Key+Value lists
  • TC-JSON-16: JsonArrayComponent - create JSON array from items
  • TC-JSON-17: JsonArray2TextListComponent - parse JSON array to GH text list
  • TC-JSON-18: JsonObject2TextComponent - serialize JSON to string
  • TC-JSON-19: JsonGetValueComponent - extract nested value by dot-notation
  • TC-JSON-20: JsonMergeComponent - merge multiple JSON objects
  • TC-JSON-21: JsonSchemaPropComponent - scalar property definition
  • TC-JSON-22: JsonSchemaPropObjectComponent - object property with sub-properties
  • TC-JSON-23: JsonSchemaPropArrayComponent - array property with items type
  • TC-JSON-24: Verify components wire together correctly

🟢 P2 - Provider Model Updates (11 tests)

  • TC-MODEL-01: GPT-5.4 series models available
  • TC-MODEL-02: GPT-5.4-mini model works
  • TC-MODEL-03: GPT-5.4-nano model works
  • TC-MODEL-04: gpt-image-1.5 for image generation
  • TC-MODEL-05: Versioned model aliases available
  • TC-MODEL-06: mistral-small-4-0-26-03 works
  • TC-MODEL-07: mistral-large-3-25-12 works
  • TC-MODEL-08: ministral models work
  • TC-MODEL-09: codestral-25-08 works
  • TC-MODEL-10: GPT-5.4 series available
  • TC-MODEL-11: Claude 4.6 series available

🟢 P2 - UI/UX (8 tests)

  • TC-UI-01: Live progress counter shows Processing batch (0/XX)...
  • TC-UI-02: Counter updates during polling (YY/XX)...
  • TC-UI-03: Preparing X/X... shown during data collection
  • TC-UI-04: Stale progress reset on new run
  • TC-UI-05: Terminal states show base message (not stale progress)
  • TC-UI-06: User messages appear correctly (no dedup issues)
  • TC-UI-07: Tool results inherit TurnId from ToolCall
  • TC-UI-08: Metrics aggregation per turn works

Migration Guide

SmartHopper 2.0.0-alpha will automatically install from Rhino Package Manager once released.

For Breaking Changes

  1. Open existing .gh files - Components might have some issues loading. Remove and replace some components the changed components with the new ones.
  2. Replace removed components - WebPageReadComponent must be replaced with Web2MdComponent

For New Features

  1. Enable Google Gemini - Add API key in SmartHopper settings
  2. Try Batch Processing - Use AISettingsComponent with Batch=true on compatible components and providers
  3. Vision Workflows - Connect images to AIImgToTextComponent or use in AIFile2MdComponent
  4. JSON Generation - Use AIText2JsonComponent with schema inputs

Checklist

  • This PR is focused on a single major release (all 2.0.0 features)
  • Version in Solution.props updated
  • CHANGELOG.md has been updated with all changes under [Unreleased]
  • PR title follows Conventional Commits format
  • PR description follows Pull Request Description Template
  • Comprehensive testing plan created in docs/Reviews/260402-PR-Testing-Plan.md

Related Issues

  • Addresses multiple feature requests for batch processing, vision input, and JSON support
  • Implements Google Gemini provider (tracked internally)
  • Closes breaking change debt from inconsistent AI tool naming
Open in Devin Review

marc-romu added 30 commits March 26, 2026 23:39
@marc-romu
update working branch (#419)
3faf562
@marc-romu
feat(json): add comprehensive JSON component suite with AI-powered te…
a785221 
…xt2json tool

- Add `text2json` AI tool for generating structured JSON from prompts conforming to JSON Schema
- Add `AIText2JsonComponent` in `SmartHopper > JSON` category with Prompt, Instructions, and Schema inputs
- Add JSON utility components requiring no AI:
  - `JsonSchemaComponent`: builds JSON Schema from property definitions with dot-notation nesting support
  - `JsonObjectComponent`: creates JSON objects from key-value pairs
@marc-romu
Merge branch 'feature/2.0.0-text2json' of https://github.com/architec…
66232f6 
…ts-toolkit/SmartHopper into feature/2.0.0-text2json
@marc-romu
feat(json): add visual JSON Schema builder components for property co…
d842f99 
…mposition

- Add `JsonSchemaPropComponent` (`JsonSchemaProp`): builds scalar property definitions from Name, Type, and Description inputs
- Add `JsonSchemaPropObjectComponent` (`JsonSchemaPropObj`): builds object properties by prefixing sub-properties with dot-notation
- Add `JsonSchemaPropArrayComponent` (`JsonSchemaPropArr`): builds array properties with configurable Items Type using `array[itemsType]` encoding
- Update `JsonSchemaComponent`
@marc-romu
docs: fix file2md tool naming inconsistencies in documentation and co…
62fa947 
…mments
@marc-romu
feat(batch): add batch processing support to AI components with resul…
0a72e67 
…t reconstruction

- Add `OnBatchCompleted` override to decode batch results and reconstruct output trees
- Add batch submission calls in worker `DoWorkAsync` after processing all items
- Add reconstructed tree handling in worker `SetOutput` to use batch results when available
- Implement provider-based result decoding using `AIInteractionText` extraction
- Add batch support to `AIImg2TextComponent`, `AIText2JsonComponent`, `AIFile
@marc-romu
feat(components): add Keywords property to AI components for improved…
39f038d 
… searchability
@marc-romu
chore: add Keywords property to AIText2JsonComponent and fix code for…
b4edf5f 
…matting
@marc-romu
chore: add 'filetomd' keyword to AIFile2MdComponent for improved sear…
6a8e64b 
…chability
@marc-romu
feat(ai): expire solution when provider changes in AIExtraSettings to…
fb0482f 
… prevent stale parameter usage
@marc-romu
fix(batch): surface individual batch item errors to Grasshopper compo…
95889ed 
…nent

When a batch job completes but individual items fail (e.g. OpenAI `BadRequest` due to `max_tokens` too large), the error was silently discarded and the component showed "Done".
@marc-romu
feat(batch): rework AIFile2MdComponent to batch only image descriptio…
b2c3168 
…ns via img2text

- **AIFile2MdComponent**: File conversion and image extraction now run locally via `file2md` with `describeImages=false`. Each extracted image is then described via `CallAiToolAsync("img2text", ...)` which is batch-interceptable. `OnBatchCompleted` reassembles final markdown from locally-stored per-file context and batch image description results using `ImageSentinelContext` dictionary.
@marc-romu
feat(batch): validate batch requests and surface warnings/errors befo…
cda9b50 
…re queuing

- Add `IsValid()` validation call on batch requests before adding to queue
- Surface validation warnings to component using `SurfaceMessagesFromReturn`
- Surface validation errors but allow queuing to proceed (non-blocking)
- Add debug logging for validation message counts
@marc-romu
feat(file2md): use placeholder-based image insertion instead of appen…
32e8863 
…ding images section

- Add `InsertImagePlaceholders` to inject `[image N]` placeholders into markdown during file2md conversion
- Add `ImageIndex` property to `ImageSentinelContext` for 1-based placeholder tracking
- Replace image section assembly with placeholder substitution in both batch and non-batch modes
- Add `_sentinelContextsInitialized` flag to prevent clearing sentinel contexts during batch polling re-runs
@marc-romu
chore(release): reduce version promotion waiting period from 30 to 15…
a8114c6 
… days

- Update milestone management guide to reflect 15-day waiting period between beta and rc releases
- Update GitHub Actions default `days-lookback` parameter from 30 to 15 days
- Update promotion PR body template to reference 15-day period
- Update release workflow documentation to reflect 15-day requirements for release age and last closed issue
- Update release-promotion workflow comments and validation table to show 15-day minimum
@marc-romu
fix(batch): correct validation message handling and add missing closi…
176281c 
…ng brace in AIFile2MdComponent

- Add missing closing brace in `AIFile2MdComponent.OnBatchCompleted` after markdown assembly
- Consolidate warning and error validation message handling into single loop in `AIStatefulAsyncComponentBase`
- Use `AddRuntimeMessage(severity, origin, message)` instead of passing message object directly
- Update debug logging to reflect combined warning/error count
@marc-romu
refactor(batch): move metrics clearing to OnEnteringProcessingState a…
7b55dba 
…nd skip output during active batch

- Move AIReturnSnapshot and batch state clearing from BeforeSolveInstance to new OnEnteringProcessingState hook
- Remove metricsInitializedForRun flag in favor of state transition-based clearing
- Add OnEnteringProcessingState virtual method to StatefulComponentBase called during Processing state entry
- Skip SetMetricsOutput in OnSolveInstancePostSolve when batch submission is still active
@marc-romu
chore: rename WebToMd to Web2Md for consistency with component naming…
999f8b9 
… convention

- Rename `WebToMdComponent` to `Web2MdComponent` and update nickname from "WebToMd" to "Web2Md"
- Rename icon resource from `webtomd` to `Web2Md`
- Rename `WebToMdAsync` method to `Web2MdAsync` in web2md AI tool
- Update CHANGELOG.md and DEV.md references from WebToMdComponent to Web2MdComponent
- Update debug logging prefix from "WebToMd" to "Web2Md"
@marc-romu
feat(discourse): add generic Discourse forum components and AI tools …
1747877 
…alongside McNeel and Ladybug forum support

- Add generic `DiscourseSearch`, `DiscoursePostGet`, `DiscoursePostOpen`, `DiscoursePostDeconstruct` components
- Add generic `AIDiscoursePostSummarize` and `AIDiscourseTopicSummarize` components
@marc-romu
refactor(discourse): unify discourse-related aitools
04816ff
@marc-romu
fix(batch): use SetPersistentRuntimeMessage for batch validation warn…
5eeb2d5 
…ings instead of SurfaceMessagesFromReturn
@marc-romu
refactor(discourse): collapse multi-line JSON schema strings to singl…
9bc1e46 
…e-line format in DiscourseToolsBase

- Replace verbatim multi-line JSON schema strings with single-line interpolated strings
- Rename `baseUrlSchemaProperty` to `baseUrlProperty` for consistency
- Simplify schema formatting while maintaining identical structure and functionality
@marc-romu
feat(text2json): add error handling and null checks to batch processi…
4469013 
…ng with improved logging

- Add try-catch block around CallAiToolAsync to handle exceptions during batch item processing
- Add null check for toolResult with warning message when tool returns null
- Add error messages with batch item index when exceptions occur
- Update debug logging to use "batch item" terminology instead of "index" for consistency
- Add empty string outputs for failed batch items to maintain output alignment
@marc-romu
fix(batch): resolve HTTPClient timeout issues in batch operations by …
47b0f06 
…centralizing timeout configuration

- Add `CreateBatchHttpClient()` protected helper method to `AIProvider` base class with 5-minute default timeout (300 seconds)
- Refactor `OpenAIProvider`, `AnthropicProvider`, and `MistralAIProvider` to use centralized helper instead of default HttpClient
- Add timeout clamping (1-600 seconds) and debug logging for batch HTTP client creation
- Update CHANGELOG.md to document batch processing
@marc-romu
feat(network): add global HTTP timeout settings for all AI providers …
8fcda04 
…with configurable UI controls

- Add `HttpTimeoutSeconds` (default 120s) and `BatchHttpTimeoutSeconds` (default 300s) global settings in ProvidersSettingsPage under "Network Settings" section
- Add NumericStepper controls (1-600 seconds range) with descriptive labels and help text for both timeout types
- Update `AIProvider.CreateBatchHttpClient()` to read `BatchHttpTimeoutSeconds` from provider settings instead of hardcoded 300s default
@marc-romu
refactor(batch): restructure File2Md batch context from per-image to …
02a7ea2 
…per-file storage with ordered image slots

- Replace `ImageSentinelContext` with `FileBatchContext` containing base markdown and ordered `ImageSlot` list
- Rename `_sentinelContexts` to `_fileContexts` and key by representative sentinel ID (first image) instead of per-image
- Add `ImageSlot` class to store per-image metadata (index, sentinel ID, mode, context, MIME type, base64)
@marc-romu
refactor(network): read HTTP timeout settings from global settings in…
edb6ab0 
…stead of per-provider settings

- Update `CreateBatchHttpClient()` to read `BatchHttpTimeoutSeconds` from global settings instead of provider-specific settings
- Update `CreateHttpClient()` to read `HttpTimeoutSeconds` from global settings instead of provider-specific settings
- Use `SmartHopperSettings.Instance.GetSetting("Global", ...)` for both timeout configurations
- Maintain same default values (120s for HTTP, 300s for batch
@marc-romu
refactor(batch): improve batch processing debugging and prevent senti…
6a620ce 
…nel tree output during active batch operations

- Add debug logging throughout File2Md and file2md batch processing to track tool execution flow
- Override `RestorePersistentOutputs()` in AIStatefulAsyncComponentBase to skip sentinel tree output during active batch submission
- Update AIFile2MdComponent to extract image descriptions from AIInteractionToolResult instead of AIInteractionText
@marc-romu
refactor(file2md): add batch context persistence and unify image desc…
29f11d6 
…ription extraction across execute and batch paths

- Add `Write`/`Read` methods to serialize `_fileContexts` (base markdown + image slot metadata) so batch results survive Grasshopper file save/reload
- Add `_batchContextLost` flag to prevent `GatherInput` from resetting `_fileContextsInitialized` during active batch, fixing same-session context overwrite bug
- Update `OnBatchCompleted` to extract image descriptions from `AIInteractionText.Content` instead of
@marc-romu
refactor(batch): add HasActiveBatchSubmission property and fix metric…
0c570e4 
…s persistence after batch completion

- Add `HasActiveBatchSubmission` protected property to distinguish poll cycles from new runs in derived classes
- Call `SetMetricsOutput(null)` in `OnBatchCompleted` to persist aggregated metrics for `RestorePersistentOutputs` (since `OnSolveInstancePostSolve` is skipped during batch)
- Update `AIMetrics.CombineWith()` to skip overwriting Provider/Model with "Unknown" or empty values during
@github-actions github-actions Bot added provider: MistralAI Issues related to the MistralAI provider provider: OpenAI Issues related to the OpenAI provider provider: DeepSeek Issues related to the DeepSeek provider scope: UI Issues related to the UI features scope: Component Base Issues related to the Component Base provider: OpenRouter Issues related to the OpenRouter provider provider: Anthropic Issues related to the Anthropic provider scope: Security Issues related to Authenticode, SHA256, etc. ci documentation labels May 23, 2026
github-actions Bot and others added 4 commits May 23, 2026 07:41
@github-actions
chore(ci): update license headers
b97e42c
@marc-romu
chore(ci): add Gemini provider labels and update documentation labeler
8cfe495 
- Add "provider: Gemini" label configuration to issue-labeler, labeler, and labels files
- Remove '**/*.md' glob from documentation labeler to prevent over-labeling markdown files
@marc-romu
chore(ci): normalize BOM encoding across test and component files
312c9f8
@marc-romu
Merge branch 'feature/2.0.0-text2json' of https://github.com/architec…
0394b78 
…ts-toolkit/SmartHopper into feature/2.0.0-text2json
Copilot AI review requested due to automatic review settings May 23, 2026 07:46
Copilot AI reviewed May 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

marc-romu added 8 commits May 23, 2026 09:48
@marc-romu
chore(test): remove trailing whitespace from provider decode test com…
e8eb8e8 
…ponents
@marc-romu
refactor(test): migrate batch test components to use IAIBatchProvider…
a8d2796 
… API
@marc-romu
ci(provider-tests): add provider cancellation test components
e7afa61
@marc-romu
refactor(test): improve provider test components to use JSON parsing …
3fe4ad3 
…instead of string matching
@marc-romu
refactor(test): remove batch and vision test components for provider …
e926987 
…that do not support it
@marc-romu
refactor(openrouter): add vision input support via OpenAI-compatible …
c0548bc 
…image_url format
@marc-romu
feat(providers): default OpenAI to Responses API and cross-provider r…
075cd5e 
…easoning support

- Switch OpenAI default endpoint from /chat/completions to /responses
- Add ForceChatCompletions provider setting for legacy endpoint override
- Refactor OpenAI batch, encode, decode, standard, tools, and vision tests
  to validate both Responses and Chat Completions endpoints
- Add Anthropic thinking/redacted_thinking block decoding
- Add OpenRouter reasoning and reasoning_details field extraction
- Standardize test message agents from Context to User across providers
- Adjust batch polling timeouts to use call-specific defaults
@marc-romu
feat(openai): add Responses API streaming support and tool format con…
b8b3b9d 
…version

- Add `ConvertToolsToResponsesFormat` method to flatten Chat Completions tool definitions into Responses API format (strict functions at root level)
- Extend `OpenAIStreamingAdapter` to handle both `/v1/responses` and `/v1/chat/completions` streaming endpoints
- Implement Responses API SSE event handlers: `response.output_text.delta`, `response.content.delta`, `response.refusal.delta`, `response.output_item.added`, `response.
Copilot AI review requested due to automatic review settings May 24, 2026 17:47
Copilot AI reviewed May 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devin-ai-integration devin-ai-integration[bot] devin-ai-integration[bot] left review comments

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@marc-romu marc-romu

Labels

ci documentation provider: Anthropic Issues related to the Anthropic provider provider: DeepSeek Issues related to the DeepSeek provider provider: Gemini provider: MistralAI Issues related to the MistralAI provider provider: OpenAI Issues related to the OpenAI provider provider: OpenRouter Issues related to the OpenRouter provider scope: Component Base Issues related to the Component Base scope: Security Issues related to Authenticode, SHA256, etc. scope: UI Issues related to the UI features

Projects

None yet

Milestone

2.0.0-alpha

Development

Successfully merging this pull request may close these issues.

2 participants

@marc-romu