Algorithms for assignment of external reviewers for PhD-thesis defense
DOI:
https://doi.org/10.20535/SRIT.2308-8893.2025.4.08Keywords:
external reviewers, reviewer assignment problem, categorization, optimization, brute force algorithm, greedy algorithm, assignment in isolation, PhD-thesis, Dimensions, ANZSRC 2020, research groupAbstract
We propose an approach to assigning external reviewers. In the proposed ap-proach, only the semantic similarity between applications and reviewers is tak-en into account; the similarity indices are assessed, and the necessary number of reviewers is assigned to ensure the maximum suitability level of the reviewers with the application, according to some criteria. We also perform a comparative analysis of various optimization algorithms using the criterion of “assignment quality–optimization time”. Experiments on the dataset showed that a reasona-ble balance between the “assignment quality” and “optimization time” criteria for the assignment of external reviewers can be achieved using a greedy algo-rithm without elitism or brute-force search on a truncated set of candidates. An application of the proposed algorithms improves the average quality of PhD committees by 13–34% across the entire dataset, depending on the algorithm used.
References
F. Wang, N. Shi, B. Chen, “A comprehensive survey of the reviewer assignment problem,” In-ternational Journal of Information Technology and Decision Making, 9(4), pp. 645–668, 2010. doi: https://doi.org/10.1142/S0219622010003993
M. Aksoy, S. Yanik, M.F. Amasyali, “Reviewer assignment problem: A systematic review of the literature,” Journal of Artificial Intelligence Research, vol. 76, 2023. doi: https://doi.org/10.1613/JAIR.1.14318
S. Tan, Z. Duan, S. Zhao, J. Chen, Y. Zhang, “Improved reviewer assignment based on both word and semantic features,” Information Retrieval Journal, 24(3), pp. 175–204, 2021. doi: https://doi.org/10.1007/s10791-021-09390-8
D. Yarowsky, R. Florian, “Taking the load off the conference chairs: Towards a digital paper-routing assistant,” Proceedings of the 1999 Joint SIGDAT Conference on Empirical Meth-ods in Natural Language Processing and Very Large Corpora, EMNLP 1999, pp. 220–230.
M. Karimzadehgan, C.X. Zhai, G. Belford, “Multi-aspect expertise matching for review as-signment,” Proceedings of International Conference on Information and Knowledge Man-agement, pp. 1113–1122, 2008. doi: https://doi.org/10.1145/1458082.1458230
M. Mirzaei, J. Sander, E. Stroulia, “Multi-aspect review-team assignment using latent research areas,” Information Processing and Management, 56(3), pp. 858–878, 2019. doi: https://doi.org/10.1016/j.ipm.2019.01.007
E. Ekinci, S.I. Omurca, “NET-LDA: A novel topic modeling method based on semantic doc-ument similarity,” Turkish Journal of Electrical Engineering and Computer Sciences, 28(4), pp. 2244–2260, 2020. doi: https://doi.org/10.3906/ELK-1912-62
O. Anjum, H. Gong, S. Bhat, J. Xiong, W.M. Hwu, “Pare: A paper-reviewer matching approach using a common topic space,” EMNLP-IJCNLP 2019 – 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natu-ral Language Processing, Proceedings of the Conference, pp. 518–528. doi: https://doi.org/10.18653/v1/d19-1049
C. Sun, K.T.J. Ng, P. Henville, R. Marchant, “Hierarchical word mover distance for collabora-tion recommender system,” Communications in Computer and Information Science, vol. 996, pp. 289–302. Springer Verlag, 2019. doi: https://doi.org/10.1007/978-981-13-6661-1_23
X. Kong, H. Jiang, Z. Yang, Z. Xu, F. Xia, A. Tolba, “Exploiting publication contents and col-laboration networks for collaborator recommendation,” PLOS One, 11(2): e0148492, 2016. doi: https://doi.org/10.1371/journal.pone.0148492
B. Bhaisare, R. Bharati, “Advancing Peer Review Integrity: Automated Reviewer Assignment Techniques with a Focus on Deep Learning Applications,” in A.K. Bairwa, V. Tiwari, S.K. Vishwakarma, M. Tuba, T. Ganokratanaa, (eds) Computation of Artificial Intelligence and Machine Learning. ICCAIML 2024. Communications in Computer and Information Science, vol 2184. Springer, Cham, 2024. doi: https://doi.org/10.1007/978-3-031-71481-8_25
Y. Zhao, J. Tang, Z. Du, “EFCNN: A restricted convolutional neural network for expert find-ing,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial In-telligence and Lecture Notes in Bioinformatics), vol. 11440 LNAI, pp. 96–107. Springer Ver-lag, 2019. doi: https://doi.org/10.1007/978-3-030-16145-3_8
S. Shtovba, M. Petrychko, “Topic modeling of researchers based on their interests from Google Scholar,” System Research and Information Technologies, no. 2, pp. 113–129, 2021. doi: https://doi.org/10.20535/SRIT.2308-8893.2021.2.09
S. Shtovba, M. Petrychko, “Jaccard index-based assessing the similarity of research fields in dimensions,” CEUR Workshop Proceedings, vol. 2533, pp. 117–128, 2019. Available: https://ceur-ws.org/Vol-2533/paper11.pdf
S. Shtovba, M. Petrychko, O. Shtovba, “Similarity metric оf categorical distributions for topic modeling problems with akin categories,” CEUR Workshop Proceedings, vol. 3392 “The Sixth International Workshop on Computer Modeling and Intelligent Systems”, pp. 76–85, 2023. doi: https://doi.org/10.32782/cmis/3392-7
M. Petrychko, S. Shtovba, “Dataset for PhD theses reviewers’ assignments,” ResearchGate, 2024. doi: http://dx.doi.org/10.13140/RG.2.2.23147.35362
