: ..,
..
:
: 109
:
: 2024
: .., .. // . 109. .: , 2024. .67-88. DOI: https://doi.org/10.25728/ubs.2024.109.4
: , , , ,
(.): anomaly detection, topic modeling, probabilities of rare events, repeatability of rare events, anomalies in text chats
: , . : , , () N- 堖, , . , , , , -. , Telegram, , . , 堖. . , , . , . , , . .
(.): The aim of the work is to develop a new method for detecting anomalies in text chats that does not use text corpora. Tasks: a brief presentation of the statistical description of the recurrence of anomalies developed in the authors' previous works, the introduction of the method of paired (generalized) N-grams, the synthesis of these methods into a new method for detecting anomalies in short message exchange systems, the method testing. A new method for detecting anomalies in the flow of text messages is proposed, which does not use a corpus of texts for learning, and, in addition, allows online learning. The material for the work was chats, groups and channels in Telegram, to which one of the authors of the work is subscribed. The volume of text material was about 50 MB, which corresponds to about 2 million words collected over 5 years. The method uses a statistical distribution of the repetition of anomalous events, as well as a method of thematic modeling based on the statistics of noun-verb pairs. Both methods were proposed earlier in the authors' works. The experiment showed that the results predicted by the proposed method correspond to the actually registered anomalies. The application of the proposed method can be useful in research and analysis of the appearance of anomalies in complex social systems, the interaction in which is reflected in communications through social networks and messengers. Such tasks are relevant both for government agencies and for business, and can help to smooth out acute social and industrial problems. The proposed method is seemed especially useful for the journalism because it allows you to determine the time of the most likely appearance of significant social phenomena.
PDF
: 377, : 1364, : 13.