FYP19082

Final Year Project

Optimization of sentiment analysis on Amazon Reviews

Introduction

Nowadays information is exploding that only few of those could be consumed by us. There is so much useful information that is generated but being ignored. It would be wonderful if there is a system can digest most of those, and then response to human needs automatically. With the rapid development of internet and NLP recently, there are enough data to collect, and sufficiently advanced technology to interpret those data, which make such a system possible now. This project focuses on sentiment analysis and its optimization, but at the end, a multi-class classification model is made, which suits for any kind of text data. It achieved great performance but is time-consuming and computationally expensive.

Nerual Network
Natural language processing
Amazon Reviews

Methods

A class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior. Derived from feedforward neural networks, RNNs can use their internal state (memory) to process variable length sequences of inputs.

RNN
CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons usually mean fully connected networks, that is, each neuron in one layer is connected to all neurons in the next layer. The "fully-connectedness" of these networks makes them prone to overfitting data. Typical ways of regularization include adding some form of magnitude measurement of weights to the loss function. CNNs take a different approach towards regularization: they take advantage of the hierarchical pattern in data and assemble more complex patterns using smaller and simpler patterns. Therefore, on the scale of connectedness and complexity, CNNs are on the lower extreme.

CNN
It stands for stands for Bidirectional Encoder Representations from Transformers, which is developed by Google. It is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. Therefore, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.

BERT

Results

Maximum text length

The higher the maximum text length, the higher the performance can be.
Learning rate

If it is too high, model will lose its ability to learn, so learning rate has its upper bound for the model. On the other hand, there is no lower bound for the model, the model will finally approach the best if it could. However, more epochs are needed to finish the model training.
Batch size

The smaller the batch size, the more training time it needs, while changing the batch size would not affect the performance of the model, since dataset of this project is big.
Train-valid-test split

If the number of samples is very big, difference in train-valid-test ratio did not make impact on performance of model. However, training efficiency was decreased according to train size.
Input text

Performance of the model does not depend on conciseness of text, but length of text. If the model gets more words, it will perform better.
Number of classes

If fine-grained sentiment is not needed, it should be transferred as binary sentiment.
Optimal model comparison

BERT outperformed CNN and RNN by achieving f1 score of 0.78, while the others only achieved 0.56.

Optimization of sentiment analysis on Amazon Reviews

Introduction

Nerual Network

Natural language processing

Amazon Reviews

Methods

Results

Maximum text length

Learning rate

Batch size

Train-valid-test split

Input text

Number of classes

Optimal model comparison

Documentations