Chapter 6 Cyclic Neural Network (RNN)

1. 6.1 What is the difference between RNNs and FNNs? 218

2. 6.2 Typical characteristics of RNNs? 218

3. 6.3 What can RNNs do? 219

4. 6.4 Typical applications of RNNs in NLP? 220

5. 6.5 What are the similarities and differences between RNNs training and traditional ANN training? 220

6. 6.6 Common RNNs Extensions and Improvement Models 221

7. 6.6.1 Simple RNNs (SRNs) 221

8. 6.6.2 Bidirectional RNNs 221

9. 6.6.3 Deep(Bidirectional) RNNs 222

10. 6.6.4 Echo State Networks (ESNs) 222

11. 6.6.5 Gated Recurrent Unit Recurrent Neural Networks 224

12. 6.6.6 LSTM Netwoorks 224

13. 6.6.7 Clockwork RNNs (CW-RNNs) 225

P