ICTACT Journal on Soft Computing

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Designing Effective Chatbot System Using GRU With Beam Search


Affiliations
1 Department of Computer Science and Information Technology, Rajarambapu Institute of Technology, India
2 Department of Computer Science, D.A.V. Rajarambapu Institute of Technology, India
     

   Subscribe/Renew Journal


Artificial Intelligence (AI) based Chatbot is a moderately new technology in the world. AI and Natural Language Processing (NLP) empowers a Chatbot to converse like a human being. Chatbots have become popular recently as they diminish human efforts by automating various tasks. AI-based Chatbot learns from the previous discussion and generates an appropriate response or action for the input given by the user. In the proposed research work we designed AI-based Chatbot system using the Sequence to Sequence (Seq2Seq) model. This system uses a Gated Recurrent Unit (GRU) for encoder and decoder. In the proposed model the GRU encoder accepts a query from the user. The GRU encoder uses an attention mechanism to consider only relevant information and convert it into the context vector form. A context vector is another input to the GRU decoder. The GRU decoder generates a response using the Beam search algorithm. The research work uses Cornell Movie Dialogue Corpus to train the proposed interactive Chatbot system. It is observed that the proposed model with the combination of GRU and Beam search gives better accuracy with the minimum loss for testing data. These experimental results are better than existing approaches that use LSTM Seq2Seq models to train Chatbot systems.

Keywords

Chatbot, Deep learning, RNN, GRU, Attention Mechanism, Beam Search
Subscription Login to verify subscription
User
Notifications
Font Size

  • A. Argal, S. Gupta, A. Modi, P. Pandey, S. Shim and C. Choo, “Intelligent Travel Chatbot for Predictive Recommendation in Echo Platform”, Proceedings of IEEE International Conference on Computing and Communication, pp. 176-183, 2018.

  • L. Chen, P. Chen and Z. Lin, “Artificial Intelligence in Education: A Review”, IEEE Access, Vol. 8, pp. 75264-75278, 2020.

  • Andry Chowanda and Alan Darmasaputra Chowanda, “Generative Indonesian Conversation Model using Recurrent Neural Network with Attention Mechanism”, Proceedings of IEEE International Conference on Computer Science and Computational Intelligence, pp. 433-440, 2018.

  • Cristian Danescu-Niculescu-Mizil and Lillian Lee, “Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs”, Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pp. 1-12, 2011.

  • G. Dzakwan and A. Purwarianti, “Comparative Study of Topology and Feature Variants for Non-task-oriented Chatbot using Sequence to Sequence Learning”, Proceedings of IEEE International Conference on Advanced Informatics: Concept Theory and Applications, pp. 135-140, 2018.

  • Hisham El-Amir and Mahmoud Hamdy, “A Tour Through the Deep Learning Pipeline”, Oxford Press, 2020.

  • H. Honda and M. Hagiwara, “Question Answering Systems with Deep Learning-based Symbolic Processing”, IEEE Access, Vol. 7, pp. 152368-152378, 2019.

  • B. Kohli, T. Choudhury, S. Sharma, and P. Kumar, “A Platform for Human-Chatbot Interaction using Python”, Proceedings of IEEE International Conference on Green Computing and Internet of Things, pp. 439-444, 2018.

  • P. Kumar, M. Sharma, S. Rawat and T. Choudhury, “Designing and Developing a Chatbot using Machine Learning”, Proceedings of IEEE International Conference on System Modeling Advancement in Research Trends, pp 87-91, 2018.

  • R.B. Mathew, S. Varghese, S.E. Joy and S.S. Alex, “Chatbot for Disease Prediction and Treatment Recommendation using Machine Learning”, Proceedings of IEEE International Conference on Trends in Electronics and Informatics, pp.851-856, 2019.

  • T. Nguyen and M. Shcherbakov, “A Neural Network Based Vietnamese Chatbot”, Proceedings of IEEE International Conference on System Modeling Advancement in Research Trends, pp. 147-149, 2018.

  • A. Nursetyo, D.R.I.M. Setiadi and E.R. Subhiyakto, “Smart Chatbot System for E-Commerce Assitance Based on AIML”, Proceedings of IEEE International Conference on Research of Information Technology and Intelligent Systems, pp. 641-645, 2018.

  • M. Nuruzzaman and O. K. Hussain., “A Survey on Chatbot Implementation in Customer Service Industry Through Deep Neural Networks”, Proceedings of IEEE International Conference on E-Business Engineering, pp. 54-61, 2018.

  • E. Ozsarfati, E. Sahin, C.J. Saul and A. Yilmaz, “Genre Classification Based on Titles with Comparative Machine Learning Algorithms”, Proceedings of IEEE International Conference on Computer and Communication Systems, pp. 14-20, 2019.

  • K. Palasundram and A. Azman, “Enhancements to The Sequence-to-Sequence-based Natural Answer Generation Models”, IEEE Access, Vol. 8, pp. 45738-45752, 2020.

  • S. Prasomphan, “Improvement of Chatbot in Trading System for SMEs by using Deep Neural Network”, Proceedings of IEEE International Conference on Cloud Computing and Big Data Analysis, pp. 517-522, 2019.

  • D. Ren, Y. Cai, W. H. Chan and Z. Li, “A Clustering Based Adaptive Sequence-to-Sequence Model for Dialogue Systems”, Proceedings of IEEE International Conference on Big Data and Smart Computing, pp. 775-781, 2018.

  • P. Rivas, K. Holzmayer, C. Hernandez and C. Grippaldi. “Excitement and Concerns about Machine Learning-based Chatbots and Talkbots: A Survey”, Proceedings of IEEE International Conference on Technology and Society, pp. 156-162, 2018.

  • Yashvardhan Sharma and Sahil Gupta, “Deep Learning Approaches for Question Answering System”, Proceedings of IEEE International Conference on Computational Intelligence and Data Science, pp. 785-794, 2018.

  • M. Su, C. Wu, K. Huang, Q. Hong and H. Wang, “A Chatbot using LSTM-based Multi-layer Embedding for Elderly Care”, Proceedings of IEEE International Conference on Orange Technologies, pp. 70-74, 2017.

  • M. Tabiaa and A. Madani, “The Deployment of Machine Learning in E-Banking: A Survey”, Proceedings of IEEE International Conference on Intelligent Computing in Data Sciences, pp. 1-7, 2019.

  • Q. Yang, Z. He, F. Ge and Y. Zhang, “Sequence-to-Sequence Prediction of Personal Computer Software by Recurrent Neural Network”, Proceedings of IEEE International Conference on Neural Networks, pp. 934-940, 2017.


Abstract Views: 78

PDF Views: 2




  • Designing Effective Chatbot System Using GRU With Beam Search

Abstract Views: 78  |  PDF Views: 2

Authors

Sandeep A. Thorat
Department of Computer Science and Information Technology, Rajarambapu Institute of Technology, India
Vishakha D. Jadhav
Department of Computer Science, D.A.V. Rajarambapu Institute of Technology, India
Dadaso T. Mane
Department of Computer Science and Information Technology, Rajarambapu Institute of Technology, India

Abstract


Artificial Intelligence (AI) based Chatbot is a moderately new technology in the world. AI and Natural Language Processing (NLP) empowers a Chatbot to converse like a human being. Chatbots have become popular recently as they diminish human efforts by automating various tasks. AI-based Chatbot learns from the previous discussion and generates an appropriate response or action for the input given by the user. In the proposed research work we designed AI-based Chatbot system using the Sequence to Sequence (Seq2Seq) model. This system uses a Gated Recurrent Unit (GRU) for encoder and decoder. In the proposed model the GRU encoder accepts a query from the user. The GRU encoder uses an attention mechanism to consider only relevant information and convert it into the context vector form. A context vector is another input to the GRU decoder. The GRU decoder generates a response using the Beam search algorithm. The research work uses Cornell Movie Dialogue Corpus to train the proposed interactive Chatbot system. It is observed that the proposed model with the combination of GRU and Beam search gives better accuracy with the minimum loss for testing data. These experimental results are better than existing approaches that use LSTM Seq2Seq models to train Chatbot systems.

Keywords


Chatbot, Deep learning, RNN, GRU, Attention Mechanism, Beam Search

References