[Relax][Frontend][TFLite] Support sequence LSTM and RNN operators

[LudovicoYIN]

Summary

Add three TFLite sequence recurrent operators to the Relax frontend, all with
coupled input-forget gate (FULL kernel) and float32-only support.

• UNIDIRECTIONAL_SEQUENCE_LSTM
• BIDIRECTIONAL_SEQUENCE_RNN
• BIDIRECTIONAL_SEQUENCE_LSTM

From #19519.

Changes

• UNIDIRECTIONAL_SEQUENCE_LSTM: same layout as single-step LSTM, unrolls over
  time and stacks per-step hidden states. Supports time_major, cell_clip, proj_clip,
  and fused activation.
• BIDIRECTIONAL_SEQUENCE_RNN: separate fw/bw RNN cells, backward scans in
  reverse. Supports merge_outputs (concat fw + bw) and split outputs via Tuple.
• BIDIRECTIONAL_SEQUENCE_LSTM: 48-input operator with fw/bw LSTM cells sharing
  the same input tensor. States at indices 35-38.
• All converters propagate final states to exp_tab for multi-step correctness.
• Peephole, projection, layer norm, and aux input are not supported (raise
  OpNotImplemented).

Testing

• test_unidirectional_sequence_lstm_none_activation — output shape [batch, time, num_units]
• test_bidirectional_sequence_rnn_none_activation — merge_outputs=True, shape [batch, time, 2*num_units]
• test_bidirectional_sequence_lstm_none_activation — merge_outputs=True, shape [batch, time, 2*num_units]

python -m pytest tests/python/relax/test_frontend_tflite.py -k "sequence_lstm or sequence_rnn" -v

[gemini-code-assist]

Code Review

This pull request adds support for TFLite's BIDIRECTIONAL_SEQUENCE_LSTM and BIDIRECTIONAL_SEQUENCE_RNN operators, and enables the previously commented-out UNIDIRECTIONAL_SEQUENCE_LSTM operator in the TVM Relax frontend. It also adds corresponding unit tests. The feedback highlights performance and correctness issues in the LSTM conversion functions: specifically, calling relax.op.permute_dims and creating the constant 1.0 inside the timestep loops creates redundant nodes in the Relax AST, which can bloat the graph and slow down compilation. Additionally, the fused activation function in convert_unidirectional_sequence_lstm should be applied to the hidden state at each step inside the loop to ensure correct recurrent state activation.

[tlopex]

Thanks for adding these sequence recurrent converters. I think there are still several correctness issues before this should land.

1. The LSTM fused activation semantics do not match TFLite.

In TFLite LSTM, FusedActivationFunction is used as the internal cell activation: it is applied to the cell update gate and to the cell state when computing the output state. It is not a post-activation applied to the final hidden state.

The current converters always use tanh for the cell gate and tanh(c_new), then apply convert_fused_activation_function to h_new. This is incorrect for both NONE and TANH.

2. time_major=True returns the wrong layout.

The converters normalize time-major input to batch-major for unrolling, but always return [batch, time, ...]. TFLite preserves the sequence layout, so when time_major=True, the output should be [time, batch, ...].

This affects UNIDIRECTIONAL_SEQUENCE_LSTM, BIDIRECTIONAL_SEQUENCE_RNN, and BIDIRECTIONAL_SEQUENCE_LSTM.

3. Unsupported optional inputs are silently ignored.

The converters say peephole/projection/layer-norm/aux inputs are unsupported, but most of these are not rejected. If those optional tensors are present, the converter will produce incorrect IR instead of raising OpNotImplemented.

4. The new tests are too weak and partly non-representative.

The bidirectional RNN/LSTM test builders use shortened input lists, while the TFLite kernels expect 12 inputs for BIDIRECTIONAL_SEQUENCE_RNN and 48 for BIDIRECTIONAL_SEQUENCE_LSTM. The tests mostly check shapes, so they do not catch the numerical semantics above.

Please add numerical tests against TFLite, or at least structural tests that verify the actual equations, time_major=True, and rejection of unsupported optional inputs.

[LudovicoYIN]

Thanks for the detailed review. I updated the sequence recurrent converters to address the correctness issues you pointed out.

1. For both sequence LSTM converters, the fused activation is now applied on the internal cell path rather than as a post-activation on the final hidden state. I also fixed the time_major=True layout handling so the output preserves [time, batch, ...].

2. For unsupported optional inputs, the converters now explicitly raise OpNotImplemented instead of silently ignoring peephole / projection / layer-norm / aux-input variants.

I also strengthened the tests:

• BIDIRECTIONAL_SEQUENCE_RNN now uses the full 12-input form
• BIDIRECTIONAL_SEQUENCE_LSTM now uses the full 48-input form
• added time_major=True regression coverage
• added unsupported-input rejection tests
• added stronger equation/activation-level checks for the recurrent lowering

[tlopex]

Could you resolve the conflict so that we can merge it in? Thanks

[LudovicoYIN]

Hi, the conflicts are resolved now.