Add generator script for Amazon Bedrock models#1431
Draft
athewsey wants to merge 1 commit intoanomalyco:devfrom
Draft
Add generator script for Amazon Bedrock models#1431athewsey wants to merge 1 commit intoanomalyco:devfrom
athewsey wants to merge 1 commit intoanomalyco:devfrom
Conversation
Add (and run) a generator script for updating models on Amazon Bedrock.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi folks! For up-front transparency, I work at AWS - though not in the actual Bedrock product team.
I came across models.dev recently and noticed a couple of gaps & inconsistencies in the Bedrock model listings... so rather than fixing them as a one-off, this PR attempts to introduce an automatic generator script for Bedrock similar to the ones already set up for other providers.
Keen to get your initial feedback before I do the last manual tidy-ups, as mentioned below 🙏
Implementation overview
This script:
generate-*scripts.Since it calls APIs, users will need an AWS Account/credentials set up to run the generator - but there should be no cost associated with doing so, as it's just fetching listings.
Known to-dos
There's some outstanding manual work to go through the new model files, especially, and add in some missing information that's not available through the APIs this script uses (e.g. audio/pdf modality support, reasoning & tool-calling support, etc) - and cases where the price mapping has come up empty. I'm OK to do that, but wanted to show & get feedback on the automated portion first. I also figured delaying those edits would help show the limitations of what the current script is(n't) able to populate.
Key call-outs & questions
(See also the docstring at the top of
generate-bedrock.ts)@aws-sdk/client-bedrockand@aws-sdk/client-pricinglibraries.:8kfor these endpoints... Not sure if there's appetite for this, or we need to consolidate?1_000_000), key ordering, etc. seemed to be inconsistently applied in the existing TOMLs... Of course it should be pretty easy to change how the script standardizes these if wanted.