feat: upgrade MiniMax default model to M3#11
Open
octo-patch wants to merge 1 commit into
Open
Conversation
Add MiniMax-M3 to the model registry and place it before the existing M2.7 / abab entries so it becomes the recommended default for new evaluations. M2.7 and M2.7-highspeed are retained as legacy options. - Add MiniMax-M3 to VLMEvalKit/vlmeval/config.py and VLMEvalKit_ov/vlmeval/config.py (mirror) - Insert M3 ahead of legacy MiniMax-M2.7 / M2.7-highspeed / abab entries - Reuse the same OpenAI-compatible GPT4V wrapper, api.minimax.io base URL, temperature=0.01 (API requires (0,1]), retry=10 - Extend tests/test_minimax_provider.py with M3 coverage: registration, api_base, model name, supported_VLM membership, GPT4V wrapper, retry count, ordering before legacy, key detection, and an integration sanity-check - No change to api/gpt.py: existing 'MiniMax-M' substring match already covers M3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrade the MiniMax model registration in NEO's evaluation kits to add
MiniMax-M3as the new default option, while keepingMiniMax-M2.7andMiniMax-M2.7-highspeedavailable for users who depend on the older entries.Changes
VLMEvalKit/vlmeval/config.pyMiniMax-M3partial entry (placed before legacy M2.7 / abab entries)VLMEvalKit_ov/vlmeval/config.pyVLMEvalKit/tests/test_minimax_provider.pyvlmeval/api/gpt.pyalready routes any model whose name containsMiniMax-Mto theMiniMax_API_KEY/MINIMAX_API_KEYenv var, so no code change is needed there for M3 to inherit the existing key-detection logic.Why
MiniMax-M3is the latest MiniMax flagship model (512K context, up to 128K output, image input support) and is the recommended default for new evaluations. Listing it ahead of the older entries surfaces it as the preferred option in the registry while preserving backward compatibility for any existing scripts that pinMiniMax-M2.7orMiniMax-M2.7-highspeed.Test plan
python -m unittest tests.test_minimax_provider.TestMiniMaxKeyDetection— newtest_m3_matches_conditionandtest_m3_detects_minimax_api_keypassMiniMax-M3registered withpartial(GPT4V, model="MiniMax-M3", api_base="https://api.minimax.io/v1/chat/completions", temperature=0.01, retry=10)in both kits, ordered before legacy entriesMiniMax-M3is accepted byhttps://api.minimax.io/v1/chat/completions