Skip to content

I64 mapping loaded incorrectly #40

@chadoe

Description

@chadoe

Hi, I noticed loading of models with I64 mapping , like potion-code-16M, results in incorrect mapping because it assumes the mapping is I32 in the token_mapping code. Ignoring the mapping dtype in safetensors header.

                let v: Vec<usize> = raw
                    .chunks_exact(4)
                    .map(|b| i32::from_le_bytes(b.try_into().unwrap()) as usize)
                    .collect();

Suggestion something like this to support loading both I32 and I64:

                let v: Vec<usize> = match t.dtype() {
                    Dtype::I64 => raw
                        .chunks_exact(8)
                        .map(|b| i64::from_le_bytes(b.try_into().unwrap()) as usize)
                        .collect(),
                    Dtype::I32 => raw
                        .chunks_exact(4)
                        .map(|b| i32::from_le_bytes(b.try_into().unwrap()) as usize)
                        .collect(),
                    other => return Err(anyhow!("unsupported mapping dtype {:?}", other)),
                };

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions