I did similar calculations with satirical and regular German news (a corpus of about 60.000 articles). If I tested on news from sites which had articles within the trainings data I got a precision of 98% and a recall of 95%.

https://medium.com/@andreasstckl/detecting-satire-and-fake-news-with-machine-learning-435695749f79

On articles from sites completely unknown to the model the performance dropped to accuracy of 88% and an F1 score of 76%.

https://arxiv.org/abs/1810.00593

University of Applied Sciences Upper Austria / School of Informatics, Communications and Media http://www.stoeckl.ai/profil/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store