I did similar calculations with satirical and regular German news (a corpus of about 60.000 articles). If I tested on news from sites which had articles within the trainings data I got a precision of 98% and a recall of 95%.


On articles from sites completely unknown to the model the performance dropped to accuracy of 88% and an F1 score of 76%.


University of Applied Sciences Upper Austria / School of Informatics, Communications and Media http://www.stoeckl.ai/profil/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store