We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Benchmark of public intent recognition services.
- Authors
Lorenc, Petr; Marek, Petr; Pichl, Jan; Konrád, Jakub; Šedivý, Jan
- Abstract
Most of the conversational AI platforms use the natural language understanding pipeline. An essential algorithm of the pipeline is intent recognition. It is responsible for classifying input messages into classes (intents), and this way, controlling the dialog flow. These days, many intent recognition services are currently also available as web services or as open-source alternatives. However, there is a limited number of datasets to compare their performance in a scenario of noised input. Additionally, many factors, such as CPU and memory requirements, make selecting the right approach challenging in practice. This paper presents a novel CIIRC dataset for evaluating the impact of noise (sentence disfluencies and filler sentences) on intent recognition algorithms. The data set focuses on command control as well as on social topics. We suggest criteria for selecting the best intent recognition algorithm. Finally, we use the suggested criteria and the new CIIRC dataset to compare the selected public intent recognition services with popular open-source algorithms.
- Subjects
WEB services; NATURAL language processing; NATURAL languages; SOCIAL control
- Publication
Language Resources & Evaluation, 2022, Vol 56, Issue 3, p1023
- ISSN
1574-020X
- Publication type
Article
- DOI
10.1007/s10579-021-09563-3