LSTM公式详细求导过程（只给出公式推导过程）

好的，这里给出LSTM公式的详细求导过程，仅包含公式推导：

### 1. 各个门的表达式

1. **遗忘门**：
   \[
   f_t = \sigma(W_f \cdot [h_{t-1}, x_t] + b_f)
   \]

2. **输入门**：
   \[
   i_t = \sigma(W_i \cdot [h_{t-1}, x_t] + b_i)
   \]

3. **候选状态**：
   \[
   \tilde{C_t} = \tanh(W_C \cdot [h_{t-1}, x_t] + b_C)
   \]

4. **输出门**：
   \[
   o_t = \sigma(W_o \cdot [h_{t-1}, x_t] + b_o)
   \]

5. **细胞状态更新**：
   \[
   C_t = f_t \cdot C_{t-1} + i_t \cdot \tilde{C_t}
   \]

6. **隐藏状态更新**：
   \[
   h_t = o_t \cdot \tanh(C_t)
   \]

### 2. 损失函数 \( L \) 对各个门参数的求导

#### **对遗忘门 \( W_f \) 的求导**：

1. 损失函数对细胞状态 \( C_t \) 的导数：
   \[
   \frac{\partial L}{\partial C_t} = \frac{\partial L}{\partial h_t} \cdot \frac{\partial h_t}{\partial C_t} + \frac{\partial L}{\partial C_{t+1}} \cdot \frac{\partial C_{t+1}}{\partial C_t}
   \]

2. 其中：
   \[
   \frac{\partial h_t}{\partial C_t} = o_t \cdot \text{sech}^2(C_t)
   \]
   \[
   \frac{\partial C_{t+1}}{\partial C_t} = f_{t+1}
   \]

3. 对遗忘门参数 \( W_f \) 求导：
   \[
   \frac{\partial L}{\partial W_f} = \frac{\partial L}{\partial C_t} \cdot \frac{\partial C_t}{\partial f_t} \cdot \frac{\partial f_t}{\partial W_f}
   \]
   \[
   \frac{\partial C_t}{\partial f_t} = C_{t-1}
   \]
   \[
   \frac{\partial f_t}{\partial W_f} = f_t \cdot (1 - f_t) \cdot [h_{t-1}, x_t]
   \]
   最终：
   \[
   \frac{\partial L}{\partial W_f} = \frac{\partial L}{\partial C_t} \cdot C_{t-1} \cdot f_t \cdot (1 - f_t) \cdot [h_{t-1}, x_t]
   \]

#### **对输入门 \( W_i \) 的求导**：

1. 损失函数对输入门 \( W_i \) 的求导：
   \[
   \frac{\partial L}{\partial W_i} = \frac{\partial L}{\partial C_t} \cdot \frac{\partial C_t}{\partial i_t} \cdot \frac{\partial i_t}{\partial W_i}
   \]

2. 其中：
   \[
   \frac{\partial C_t}{\partial i_t} = \tilde{C_t}
   \]
   \[
   \frac{\partial i_t}{\partial W_i} = i_t \cdot (1 - i_t) \cdot [h_{t-1}, x_t]
   \]
   最终：
   \[
   \frac{\partial L}{\partial W_i} = \frac{\partial L}{\partial C_t} \cdot \tilde{C_t} \cdot i_t \cdot (1 - i_t) \cdot [h_{t-1}, x_t]
   \]

#### **对候选状态 \( W_C \) 的求导**：

1. 损失函数对候选状态 \( W_C \) 的求导：
   \[
   \frac{\partial L}{\partial W_C} = \frac{\partial L}{\partial C_t} \cdot \frac{\partial C_t}{\partial \tilde{C_t}} \cdot \frac{\partial \tilde{C_t}}{\partial W_C}
   \]

2. 其中：
   \[
   \frac{\partial C_t}{\partial \tilde{C_t}} = i_t
   \]
   \[
   \frac{\partial \tilde{C_t}}{\partial W_C} = (1 - \tilde{C_t}^2) \cdot [h_{t-1}, x_t]
   \]
   最终：
   \[
   \frac{\partial L}{\partial W_C} = \frac{\partial L}{\partial C_t} \cdot i_t \cdot (1 - \tilde{C_t}^2) \cdot [h_{t-1}, x_t]
   \]

#### **对输出门 \( W_o \) 的求导**：

1. 损失函数对输出门 \( W_o \) 的求导：
   \[
   \frac{\partial L}{\partial W_o} = \frac{\partial L}{\partial h_t} \cdot \frac{\partial h_t}{\partial o_t} \cdot \frac{\partial o_t}{\partial W_o}
   \]

2. 其中：
   \[
   \frac{\partial h_t}{\partial o_t} = \tanh(C_t)
   \]
   \[
   \frac{\partial o_t}{\partial W_o} = o_t \cdot (1 - o_t) \cdot [h_{t-1}, x_t]
   \]
   最终：
   \[
   \frac{\partial L}{\partial W_o} = \frac{\partial L}{\partial h_t} \cdot \tanh(C_t) \cdot o_t \cdot (1 - o_t) \cdot [h_{t-1}, x_t]
   \]

Explore the detailed derivation process of LSTM formulas, focusing on gate equations such as forget gate, input gate, candidate state, and output gate. This guide provides a comprehensive breakdown for understanding LSTM networks.

Math Problem Statement

Solution

Ask a new question for Free

By Image

Math Problem Analysis

Mathematical Concepts

Formulas

Theorems

Suitable Grade Level

Related Recommendation