Skip to content

Fixed LSTM#219

Open
veblush wants to merge 1 commit into
ARM-software:mainfrom
veblush:lstm
Open

Fixed LSTM#219
veblush wants to merge 1 commit into
ARM-software:mainfrom
veblush:lstm

Conversation

@veblush
Copy link
Copy Markdown

@veblush veblush commented May 21, 2026

This PR fixes two critical issues in arm_lstm_unidirectional_s8 and s16 that prevent state persistence in streaming models and cause out-of-bounds reads during non-time-major inference. These issues are closely related to in tensorflow/tflite-micro#3564.

Problem:

  • State Wiping: By default, arm_lstm_unidirectional_* unconditionally sets hidden_in to NULL and memsets cell_state to 0. This discards the HiddenStateTensor and CellStateTensor that TFLM relies on to persist state across Invoke() calls for streaming models.
  • Striding Bug: In the time_major = false block of arm_lstm_unidirectional_*, CMSIS-NN attempts to jump between batches by passing batch_offset = params->time_steps to arm_nn_lstm_step_*. However, arm_nn_lstm_step_* forwards this batch_offset to arm_nn_vec_mat_mul_result_acc_s8_s16 for both the data_in and hidden_in pointers. Since the hidden_state buffer is contiguous (stride 1) and not strided like data_in, passing batch_offset = params->time_steps causes out-of-bounds reads on the hidden_in buffer at timestep t=0.

Solution:

  • Adding a hidden_state pointer to cmsis_nn_lstm_context.
  • Forwarding this hidden_state as hidden_in when present, skipping the cell_state wiping if so.
  • Explicitly iterating over the batch_size in the time_major = false case when computing step sizes, which forces batch_offset = 1 and avoids the buggy out-of-bounds stride entirely while writing to the final memory buffer sequentially.

@mansnils
Copy link
Copy Markdown
Contributor

Thank you Esun!
The PR looks good! Could you add a unit test?
I think this condition - time_major=false && batch > 1 && time_steps > 1 - should catch it?

Please update date/revisions. That would be the patch version to update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants