Paper Abstract
"Chatbots play a central role in modern software systems regarding customer services. They have the potential of applicability for different purposes. A key open research challenge is how humanizing and evaluating chatbots during their development. Humanized chatbots carry out a fluid and pleasant conversation with the user, demonstrating empathy and personality. In this article, we aim to study and develop an evaluation method that indicates the level of humanization of a chatbot under analysis. Our framework comprises two objectives and questionnaires, which are applied to establish an assessment adaptable to the different objectives of using chatbots (commercial or therapeutic, for example). We combine these questionnaires to generate evaluation metrics, which provide a humanization score for the evaluated software. Our method helps designers identify specific factors that affect users' experience interacting with chatbots. We carry out a case study on the application of our framework and reveal the main findings about its applicability."

The full paper is available through SpringerLink, provided that you have proper access rights.