Transactions on Data Analysis in Social Science

Transactions on Data Analysis in Social Science

Separating Representation from Decision: A Controlled Comparison of Classifier Heads on Frozen ParsBERT Representations for Persian News Topic Classification

Document Type : Original Article

Author
M. Mehrafrouz
Department of Computer Engineering, Payame Noor University, International Unit (Overseas Campuses Unit in Asia)
10.22034/tdas.2026.255885
Abstract
Automatic topic classification of news is difficult in Persian because of rich morphology, inconsistent orthography and topical overlap between categories. Pretrained transformers such as ParsBERT provide strong contextual representations, but it is unclear how much of the task is solved by the representation and how much by the classifier placed on top of it. This study holds a frozen ParsBERT encoder fixed and compares three heads with different inductive biases: logistic regression (LR), a multi-layer perceptron (MLP) and a convolutional network (CNN) over the token matrix. On a seven-class corpus of 500 annotated news articles (Cohen’s κ = .82) with a stratified held-out test set of 100 articles, LR performed best (accuracy 0.79, 95% CI [0.70, 0.86]; macro-F1 0.78), followed by the CNN (0.77; macro-F1 0.66) and the MLP (0.68; macro-F1 0.49); all exceeded a majority baseline (0.31). Exact McNemar tests on the paired predictions showed no difference between LR and the CNN (p = .83); LR exceeded the MLP (p = .035), but not after Holm correction. Repeated stratified 5-fold cross-validation with five seeds confirmed the pattern (accuracy: LR 0.78 ± 0.03, CNN 0.76 ± 0.04, MLP 0.70 ± 0.04): LR outperformed the MLP (corrected resampled t-test, p = .002) but not the CNN (p = .38), class weighting reduced the MLP’s collapse on minority classes, and full fine-tuning of ParsBERT added only 2.5 accuracy points. Frozen ParsBERT representations thus make Persian news topics largely linearly separable, and added decision-layer capacity brings no benefit when labelled data are scarce and imbalanced.
Keywords
Persian Natural Language Processing
News Topic Classification
Parsbert
Transfer Learning
Classifier Head
Logistic Regression
Convolutional Neural Network
Low-Resource Languages
[1]      Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019). Text classification algorithms: A survey. Information, 10(4), Article 150. https://doi.org/10.3390/info10040150
[2]      Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2021). Deep learning-based text classification: A comprehensive review. ACM Computing Surveys, 54(3), Article 62. https://doi.org/10.1145/3439726
[3]      McCallum, A., & Nigam, K. (1998). A comparison of event models for Naive Bayes text classification. In AAAI-98 workshop on learning for text categorization, 752(1), 41–48.
[4]      Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. In Machine learning: ECML-98 (Lecture Notes in Computer Science, Vol. 1398, pp. 137–142). Springer. https://doi.org/10.1007/BFb0026683
[5]      Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (pp. 1746–1751). Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1181
[6]      Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
[7]      Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681. https://doi.org/10.1109/78.650093
[8]      Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (Vol. 30, pp. 5998–6008).
[9]      Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171–4186). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
[10]   Drouchas, T., & Tsalidis, C. (2021). The challenges of Persian user-generated textual content: A machine learning-based approach (arXiv:2101.08087). arXiv. https://arxiv.org/abs/2101.08087
[11]   Kashefi, O., Rahmati, M., & Sharif, H. (2024). Investigating the challenges and opportunities in Persian language information retrieval through standardized data collections and deep learning. Computers, 13(8), Article 212. https://doi.org/10.3390/computers13080212
[12]   AleAhmad, A., Amiri, H., Darrudi, E., Rahgozar, M., & Oroumchian, F. (2009). Hamshahri: A standard Persian text collection. Knowledge-Based Systems, 22(5), 382–387. https://doi.org/10.1016/j.knosys.2009.05.002
[13]   Feizi-Derakhshi, M. R., Mottaghinia, Z., & Asgari-Chenaghlu, M. (2022). Persian text classification based on deep neural networks. Soft Computing Journal, 11(1), 120–139.
[14]   Farahani, M., Gharachorloo, M., Farahani, M., & Manthouri, M. (2021). ParsBERT: Transformer-based model for Persian language understanding. Neural Processing Letters, 53(6), 3831–3847. https://doi.org/10.1007/s11063-021-10528-4
[15]   Ghayoomi, M., & Mousavian, M. (2022). Deep transfer learning for COVID-19 fake news detection in Persian. Expert Systems, 39(8), Article e13008. https://doi.org/10.1111/exsy.13008
[16]   Samadi, M., Mousavian, M., & Momtazi, S. (2021). Persian fake news detection: Neural representation and classification at word and text levels. ACM Transactions on Asian and Low-Resource Language Information Processing, 21(1), Article 7. https://doi.org/10.1145/3472620
[17]   Jahanbakhsh-Nagadeh, Z., Feizi-Derakhshi, M. R., & Sharifi, A. (2021). A deep content-based model for Persian rumor verification. ACM Transactions on Asian and Low-Resource Language Information Processing, 21(1), Article 16. https://doi.org/10.1145/3487289
[18]   Kebriaei, E., Homayouni, A., Faraji, R., Razavi, A., Shakery, A., Faili, H., & Yaghoobzadeh, Y. (2024). Persian offensive language detection. Machine Learning, 113(7), 4359–4379. https://doi.org/10.1007/s10994-023-06370-5
[19]   Peters, M. E., Ruder, S., & Smith, N. A. (2019). To tune or not to tune? Adapting pretrained representations to diverse tasks. In Proceedings of the 4th Workshop on Representation Learning for NLP (pp. 7–14). Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-4302
[20]   Sun, C., Qiu, X., Xu, Y., & Huang, X. (2019). How to fine-tune BERT for text classification? In Chinese computational linguistics (Lecture Notes in Computer Science, Vol. 11856, pp. 194–206). Springer. https://doi.org/10.1007/978-3-030-32381-3_16
[21]   Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 8440–8451). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.747
[22]   Ethayarajh, K. (2019). How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (pp. 55–65). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1006
[23]   Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56), 1929–1958.
[24]   Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations.
[25]   Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427–437. https://doi.org/10.1016/j.ipm.2009.03.002
[26]   Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429–449. https://doi.org/10.3233/IDA-2002-6504
[27]   Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895–1923. https://doi.org/10.1162/089976698300017197
[28]   Nadeau, C., & Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52(3), 239–281. https://doi.org/10.1023/A:1024068626366
Transactions on Data Analysis in Social Science
Volume 8, Issue 1
Winter 2026
Pages 74-88

  • Receive Date 03 December 2025
  • Revise Date 21 January 2026
  • Accept Date 04 March 2026