chore: fallback language skills v2#123
Open
sarahxsanders wants to merge 2 commits into
Open
Conversation
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Collaborator
Author
|
/wizard-ci all |
🧙 Wizard CI ResultsTrigger ID:
Configuration
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
loosening the reigns on the fallback language skillz. they're too constrained today, making them fail in some cases, or just implement posthog "meh"ly in others
sister PR: PostHog/wizard#431
changes
fallback-description.md- a separate skill template for non-framework languagesconfig.yamlto point the four fallback skills at the new templatedocs_urlsto fallback skills since they don't have example apps to teach patternswhy a new template?
framework skills have a specific example project to copy from. fallback skills don't have that luxury, the agent needs more freedom to explore the project and figure out where PostHog fits.
putting both sets of guidance in one file would mean every tweak to help vanilla JS projects risks changing behavior for Next.js or Django runs that already work well
this creates a clean blast radius so changes to either never impact the other
why disconnect example apps
I tested thoroughly with and without them. without them, the agent leaned on the SDK docs and produced more complete integrations across ALL four languages. the examples in this case were actually constraining the agent more than helping it
imo this is the right thing to do, as fallback language skills often get used in "odd" ways sometimes and we should let the agent do the thinking
testing
I've A/B tested this, see the wizard PR test results