The review column contains text for the review and the sentiment column contains sentiment for the review. The file contains 50,000 records and two columns: review and sentiment. If you download the dataset and extract the compressed file, you will see a CSV file. The dataset used in this article can be downloaded from this Kaggle link. Note: All the scripts in this article have been tested using Google Colab environment, with Python runtime set to GPU. In the next article I will explain how the BERT Tokenizer, along with BERT embedding layer, can be used to create even more efficient NLP models. In this article you will see how the BERT Tokenizer can be used to create text classification model. Rather we will see how to perform text classification using the BERT Tokenizer. In this article we will not go into the mathematical details of how BERT is implemented, as there are plenty of resources already available online. Just recently, Google announced that BERT is being used as a core part of their search algorithm to better understand queries. BERT was developed by researchers at Google in 2018 and has been proven to be state-of-the-art for a variety of natural language processing tasks such text classification, text summarization, text generation, etc. Like word embeddings, BERT is also a text representation technique which is a fusion of variety of state-of-the-art deep learning algorithms, such as bidirectional encoder LSTM and Transformers. If you have no idea of how word embeddings work, take a look at my article on word embeddings. BERT is a text representation technique like Word Embeddings. In this article we will study BERT, which stands for Bidirectional Encoder Representations from Transformers and its application to text classification. In the previous article of this series, I explained how to perform neural machine translation using seq2seq architecture with Python's Keras library for deep learning. This is the 23rd article in my series of articles on Python for NLP.

Tokenize pandas column