Examining the Determinants of Perceived Effectiveness in AI-powered Conversational Interfaces: A Study Based on Higher Education Sector in Sri Lanka

Abstract

This study was carried out with the main purpose of identifying the determinants of perceived effectiveness in AI-powered conversational interfaces in higher education in Sri Lanka. There were minimal research being done to analyze the determinants of perceived effectiveness in AI-powered conversational interfaces which reflected a clear literature gap. Further, none of the studies were focused on the AI-powered conversational interfaces in Sri Lankan education institutions. This was identified as the main research problem to carry out this study. A comprehensive literature review was done to identify the main independent variables. These were recognized as Usage Frequency of AI Interfaces, Quality of AI Interaction, Training and Familiarity with AI, and Institutional Support for AI Integration. The dependent variable was identified as the perceived effectiveness in AI-powered conversational interfaces. The theories such as Constructivism and Social Constructivism, Cognitive Load Theory, Connectivism and Behaviorism and Feedback Mechanisms were examined under theoretical review of this study. This study is based on the positivism philosophy and deductive approach to achieve the objectives. Main research instrument used in this study is the survey questionnaire based on a Likert scale. Based on the literature, conceptual framework and hypotheses were developed. A questionnaire was distributed among university students and 100 answers were considered as the sample size. The collected data was analyzed using SPSS software. First, reliability and validity was tested to ensure that the data set is accurate and reliable. A thorough analysis of variables was done using descriptive statistics, and correlation analysis was used to analyze the relationship between the variables. The results indicated that all the identified independent variables have a significant positive correlation with the perceived effectiveness in AI-powered conversational interface. Main implications of these findings are investment in training to enhance students' and faculty's familiarity with AI technologies, improving quality of the interface, support for the integration of AI technologies into the curriculum and encouraging regular usage of AI interfaces. Future researchers may do a longitudinal study covering larger sample sizes to gain better understanding about the perceived effectiveness in AI-powered conversational interfaces in educational institutions.