Abstract:

Natural Language Processing (NLP) is a new interdisciplinary subject to study and process the natural language by computer. In recent years, NLP has developed very rapidly and it attracted great attention from the linguistic community. This paper discusses four types of neural network model in natural language processing: Feed-forward Neural Network model (FNN), Convolutional Neural Network model (CNN), Recurrent Neural Network model (RNN), and Pre-Training model (PT), including the basic principle, structure, algorithm, and mechanism of the model, highlights their application in NLP. The paper points out that although the neural network models have become the mainstream of NLP, but these models still lack interpretability and need to be supported by rule-based language models and statistics-based language models in the futur.

Key words: Natural Language Processing, Neural Network model, Feed-forward Neural Network model, Convolutional Neural Network model, Recurrent Neural Network model, Pre-Training model