We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
The Janes project: language resources and tools for Slovene user generated content.
- Authors
Fišer, Darja; Ljubešić, Nikola; Erjavec, Tomaž
- Abstract
The paper presents the results of the Janes project, which aimed to develop language resources and tools for Slovene user generated content. The paper first describes the 200 million word Janes corpus, containing tweets, forum posts, news comments, user and talk pages from Wikipedia, and blogs and blog comments, where each text is accompanied by rich metadata. The developed processing tools for Slovene user generated content are presented next, which include a tokeniser, word-normaliser, part-of-speech tagger and lemmatiser, and a named entity recogniser. A set of manually annotated datasets was also produced, both for tool training as well as for linguistic research. The developed resources and tools are made publicly available under Creative Commons licences in the repository of the CLARIN.SI research infrastructure and on GitHub, while the corpora are also available through the CLARIN.SI concordancers.
- Subjects
WIKIPEDIA; ONLINE comments; USER-generated content; LANGUAGE &; languages
- Publication
Language Resources & Evaluation, 2020, Vol 54, Issue 1, p223
- ISSN
1574-020X
- Publication type
Article
- DOI
10.1007/s10579-018-9425-z