gpntbНаучные и технические библиотекиScientific and Technical Libraries1027-36892686-8601Russian National Public Library for Science and Technology10.33186/1027-3689-2025-1-56-78gpntb-1447Research ArticleИНФОРМАЦИОННО-ПОИСКОВЫЕ ЯЗЫКИINFORMATION RETRIEVAL LANGUAGESКлассификация научных исследований целей устойчивого развития ООН: проблемы, подходы и перспективы использования генеративного искусственного интеллектаClassifying the UN SDGs research: The problems, approaches and prospects for generative artificial intelligencehttps://orcid.org/0000-0001-8805-7631СеливановаИ. В.SelivanovaI. V.

Селиванова Ирина Вячеславовна – научный сотрудник лаборатории наукометрии и научных коммуникаций

Москва

Irina V. Selivanova – Researcher, Scientometrics and Scientific Communications Laboratory

Moscow

i-seli@yandex.ruhttps://orcid.org/0000-0002-8990-2742БлиновП. Ю.BlinovP. Y.

Блинов Павел Юрьевич – старший научный сотрудник лаборатории наукометрии и научных коммуникаций

Москва

Pavel Y. Blinov – Senior Researcher, Scientometrics and Scientific Communications Laboratory

Moscow

p.blinov@riep.ruhttps://orcid.org/0000-0003-2552-3943МалышеваА. В.MalyshevaA. V.

Малышева Александра Валерьевна – младший научный сотрудник лаборатории наукометрии и научных коммуникаций

Москва

Alexandra V. Malysheva – Junior Researcher, Scientometrics and Scientific Communications Laboratory

Moscow

bag_bala@mail.ruhttps://orcid.org/0000-0002-0495-9898КосяковД. В.KosyakovD. V.

Косяков Денис Викторович – научный сотрудник лаборатории информационных технологий и искусственного интеллекта

Новосибирск

Denis V. Kosyakov – Researcher, Information Technologies and Artificial Intelligence Laboratory, Institute of Computational Mathematics and Mathematical Geophysics

Novosibirsk

kosyakov@sciencepulse.ruРоссийский научно-исследовательский институт экономики, политики и права в научно-технической сфереРоссияRussian Research Institute of Economics, Politics and Law in Science and TechnologyRussian FederationИнститут вычислительной математики и математической геофизики Сибирского отделения РАНРоссияInstitute of Computational Mathematics and Mathematical Geophysics, Russian Academy of Sciences Siberian BranchRussian Federation202531012025015678Copyright © Селиванова И.В., Блинов П.Ю., Малышева А.В., Косяков Д.В., 20252025Селиванова И.В., Блинов П.Ю., Малышева А.В., Косяков Д.В.Selivanova I.V., Blinov P.Y., Malysheva A.V., Kosyakov D.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ntb.gpntb.ru/jour/article/view/1447

Тематическая классификация научных публикаций улучшает навигацию в потоке научной литературы, обеспечивает возможность библиометрического анализа, разноуровневой оценки результативности научных исследований. Универсальный характер повестки устойчивого развития ООН, внимание к целям устойчивого развития (ЦУР), значимость научных исследований, направленных на их достижение, а также комплексный и многоаспектный характер ЦУР обеспечивают высокий интерес к проблеме соотнесения научных публикаций и ЦУР со стороны библиографов, наукометрического сообщества, международных научных баз данных ( МНБД). В Web of Science, Scopus, Dimensions, а также у отдельных групп исследователей приняты различные подходы к классификации статей о ЦУР, каждый из которых имеет сильные и слабые стороны. Разница в результатах классификации требует анализа и совершенствования подходов и методов. Развитие технологий генеративного искусственного интеллекта и больших языковых моделей открывает новые возможности тематической классификации научных текстов, в том числе в разрезе ЦУР ООН. Цель исследования – анализ методов классификации, используемых для отнесения научных публикаций к ЦУР, демонстрация применимости для этой задачи больших языковых моделей на примере ChatGPT.

The subject classification of research publications enhances navigation in the flow of science literature, enables bibliometric analysis, multitier assessment of research performance. The universal character of the UN agenda of sustainable development and importance of sustainable development goals (SDGs) and scientific research to achieve them, and the complex and multiaspect SDGs stir high interest of bibliographers, scientometrics community, international science databases, in the problem of correlating science publications and SDGs. The Web of Science, Scopus, Dimensions, as well as the individual researchers apply various approaches to classifying the articles on SDGs, and these classifications have their strengths and weaknesses. The differences in the resulting classifications calls for the analysis and improvement of methods and approaches. The evolving generative artificial intelligence technologies and big language models open up new possibilities for the subject classification of science texts including those related to the UN SDGs. The authors analyze the methods used to classify publications as SDG-related, and demonstrate the applicability of big language models as exemplified by ChatGPT

цели устойчивого развитиятематическая классификация научных текстовбиблиографический поискмашинное обучениеграфы цитированиягенеративный искусственный интеллектбольшие языковые моделиScopusWeb of ScienceDimensionssustainable development goalssubject classification of scientific textsbibliographic searchmachine learningcitation graphsgenerative artificial intelligencebig language modelsScopusWeb of ScienceDimensionsReferencesUnited Nations Environment Programme (UNEP) // Year Book of International Cooperation on Environment and Development / ed. Bergesen H.O., Parmann G., Thommessen O. B. : Routledge, 1998. 3 p.United Nations Environment Programme (UNEP) // Year Book of International Cooperation on Environment and Development / ed. Bergesen H.O., Parmann G., Thommessen O. B. : Routledge, 1998. 3 p.Leal Filho W. et al. Towards a common future: revising the evolution of universitybased sustainability research literature // International Journal of Sustainable Development & World Ecology. Taylor & Francis. 2021. Vol. 28, № 6. P. 503–517. https://doi.org/10.1080/13504509.2021.1881651.Leal Filho W. et al. Towards a common future: revising the evolution of universitybased sustainability research literature // International Journal of Sustainable Development & World Ecology. Taylor & Francis. 2021. Vol. 28, № 6. P. 503–517. https://doi.org/10.1080/13504509.2021.1881651.Коптюг В. А. Повестка дня на ХХI век. Концепция устойчивого развития и социально-политические движения // Наука из первых рук. 2011. № 2 (38). P. 36–51.Koptiug V. A. Povestka dnia na KHKHI vek. Kontceptciia ustoi`chivogo razvitiia i sotcial`no-politicheskie dvizheniia // Nauka iz pervy`kh ruk. 2011. № 2 (38). P. 36–51.Ларионова М. В. Вызовы достижения Целей развития тысячелетия (ЦРТ) // Вестник Международных Организаций: Образование, Наука, Новая Экономика. 2020. Vol. 15, № 1. P. 155–176. https://doi.org/10.17323/1996-7845-2020-01-07.Larionova M. V. Vy`zovy` dostizheniia TCelei` razvitiia ty`siacheletiia (TCRT) // Vestneyk Mezhdunarodny`kh Organizatcii`: Obrazovanie, Nauka, Novaia E`konomika. 2020. Vol. 15, № 1. P. 155–176. https://doi.org/10.17323/1996-7845-2020-01-07.Parotto E., Pablos-Méndez A. From MDGs to SDGs // Global Health Essentials / ed. Raviglione M. C. B. et al. Cham: Springer International Publishing, 2023. P. 463–468. https://doi.org/10.1007/978-3-031-33851-9_71.Parotto E., Pablos-Méndez A. From MDGs to SDGs // Global Health Essentials / ed. Raviglione M. C. B. et al. Cham: Springer International Publishing, 2023. P. 463–468. https://doi.org/10.1007/978-3-031-33851-9_71.Цели устойчивого развития: отчёт по России. URL: https://icss.ru/vokrugstatistiki/tseli-ustoychivogo-razvitiya-otchet-po-rossii (дата обращения: 20.12.2024).TCeli ustoi`chivogo razvitiia: otchyot po Rossii. URL: https://icss.ru/vokrugstatistiki/tseli-ustoychivogo-razvitiya-otchet-po-rossii (дата обращения: 20.12.2024).Добровольный национальный обзор. URL: https://rosstat.gov.ru/folder/94692 (дата обращения: 20.12.2024).Dobrovol`ny`i` natcional`ny`i` obzor. URL: https://rosstat.gov.ru/folder/94692 (data obrashcheniia: 20.12.2024).Вандышева А. и др. 2020–2030: Десятилетие действий для ЦУР в России. Вызовы и решения / под ред. Н. Рахимовой. Москва, 2020. 142 с.Vandy`sheva A. i dr. 2020–2030: Desiatiletie dei`stvii` dlia TCUR v Rossii. Vy`zovy` i resheniia / pod red. N. Rahimovoi`. Moskva, 2020. 142 s.Alfirević N., Malešević Perović L., Mihaljević Kosor M. Productivity and Impact of Sustainable Development Goals (SDGs)-Related Academic Research: A Bibliometric Analysis: 9 // Sustainability. Multidisciplinary Digital Publishing Institute. 2023. Vol. 15, № 9. P. 7434. https://doi.org/10.3390/su15097434.Alfirević N., Malešević Perović L., Mihaljević Kosor M. Productivity and Impact of Sustainable Development Goals (SDGs)-Related Academic Research: A Bibliometric Analysis:Sianes A. et al. Impact of the Sustainable Development Goals on the academic research agenda. A scientometric analysis // PLoS One. 2022. Vol. 17, № 3. P. e0265409. https://doi.org/10.1371/journal.pone.0265409.// Sustainability. Multidisciplinary Digital Publishing Institute. 2023. Vol. 15, № 9. P. 7434. https://doi.org/10.3390/su15097434.Filho W.L. et al. The role of universities in accelerating the sustainable development goals in Europe // Sci Rep. Nature Publishing Group2024. Vol. 14, № 1. P. 15464. https://doi.org/10.1038/s41598-024-65820-9.Sianes A. et al. Impact of the Sustainable Development Goals on the academic research agenda. A scientometric analysis // PLoS One. 2022. Vol. 17, № 3. P. e0265409. https://doi.org/10.1371/journal.pone.0265409.Fonseca L. M., Domingues J. P., Dima A. M. Mapping the Sustainable Development Goals Relationships: 8 // Sustainability. Multidisciplinary Digital Publishing Institute. 2020. Vol. 12, № 8. P. 3359. https://doi.org/10.3390/su12083359Filho W.L. et al. The role of universities in accelerating the sustainable development goals in Europe // Sci Rep. Nature Publishing Group2024. Vol. 14, № 1. P. 15464. https://doi.org/10.1038/s41598-024-65820-9.Armitage C. S., Lorenz M., Mikki S. Mapping scholarly publications related to the Sustainable Development Goals: Do independent bibliometric approaches get the same results? // Quantitative Science Studies. 2020. Vol. 1, № 3. P. 1092–1108. https://doi.org/10.1162/qss_a_00071.Fonseca L. M., Domingues J. P., Dima A. M. Mapping the Sustainable Development Goals Relationships: 8 // Sustainability. Multidisciplinary Digital Publishing Institute. 2020. Vol. 12, № 8. P. 3359. https://doi.org/10.3390/su12083359.Jayabalasingham B. et al. Identifying research supporting the United Nations Sustainable Development Goals. Elsevier Data Repository, 2019. Vol. 1. https://doi.org/10.17632/87txkw7khs.1.Armitage C. S., Lorenz M., Mikki S. Mapping scholarly publications related to the Sustainable Development Goals: Do independent bibliometric approaches get the same results? // Quantitative Science Studies. 2020. Vol. 1, № 3. P. 1092–1108. https://doi.org/10.1162/qss_a_00071.Search Queries for «Mapping Research Output to the Sustainable Development Goals (SDGs)». URL: https://aurora-network-global.github.io/sdg-queries/ (дата обращения: 21.12.2024).Jayabalasingham B. et al. Identifying research supporting the United Nations Sustainable Development Goals. Elsevier Data Repository, 2019. Vol. 1. https://doi.org/10.17632/87txkw7khs.1.Duran-Silva N. et al. A controlled vocabulary defining the semantic perimeter of Sustainable Development Goals. Zenodo, 2019. https://doi.org/10.5281/zenodo.3567769.Search Queries for «Mapping Research Output to the Sustainable Development Goals (SDGs)». URL: https://aurora-network-global.github.io/sdg-queries/ (Accessed: 21.12.2024Kashnitsky Y. et al. Evaluating approaches to identifying research supporting the United Nations Sustainable Development Goals // Quantitative Science Studies. 2024. Vol. 5, № 2. P. 408–425. https://doi.org/10.1162/qss_a_00304Duran-Silva N. et al. A controlled vocabulary defining the semantic perimeter of Sustainable Development Goals. Zenodo, 2019. https://doi.org/10.5281/zenodo.3567769.Improving the Scopus and Aurora queries to identify research that supports the United Nations Sustainable Development Goals (SDGs) 2021. URL: https://elsevier.digitalcommonsdata.com/datasets/9sxdykm8s4/4 (accessed: 17.12.2024).Kashnitsky Y. et al. Evaluating approaches to identifying research supporting the United Nations Sustainable Development Goals // Quantitative Science Studies. 2024. Vol. 5, № 2. P. 408–425. https://doi.org/10.1162/qss_a_00304SDG Research Mapping Initiative – SEO Metadata // www.elsevier.com. URL: https://www.elsevier.com/about/sustainability/sdg-research-mapping-initiative (accessed: 20.12.2024).Improving the Scopus and Aurora queries to identify research that supports the United Nations Sustainable Development Goals (SDGs) 2021. URL: https://elsevier.digitalcommonsdata.com/datasets/9sxdykm8s4/4 (Accessed: 17.12.2024).Садовская Л. Л. и др. Обработка текстов на естественном языке: обзор публикаций // Искусственный интеллект и принятие решений. 2021. № 3. P. 66–86. https://doi.org/10.14357/20718594210306/SDG Research Mapping Initiative – SEO Metadata // www.elsevier.com. URL: https://www.elsevier.com/about/sustainability/sdg-research-mapping-initiative (Accessed: 20.12.2024).Selivanova E. V. et al. Expert, Journal, and Automatic Classification of Full Texts and Annotations of Scientific Articles // Automatic Documentation and Mathematical Linguistics. 2021. Vol. 55, № 4. P. 178–189. https://doi.org/10.3103/S0005105521040075.Sadovskaia L. L. i dr. Obrabotka tekstov na estestvennom iazy`ke: obzor publikatcii` // Iskusstvenny`i` intellekt i priniatie reshenii`. 2021. № 3. P. 66–86. https://doi.org/10.14357/20718594210306/The South African SDG Hub. URL: https://sasdghub.up.ac.za/home/ (accessed: 21.12.2024).Selivanova E. V. et al. Expert, Journal, and Automatic Classification of Full Texts and Annotations of Scientific Articles // Automatic Documentation and Mathematical Linguistics. 2021. Vol. 55, № 4. P. 178–189. https://doi.org/10.3103/S0005105521040075.Hook D. W., Porter S. J., Herzog C. Dimensions: Building Context for Search and Evaluation // Front. Res. Metr. Anal. Frontiers. 2018. Vol. 3. https://doi.org/10.3389/frma.2018.00023.The South African SDG Hub. URL: https://sasdghub.up.ac.za/home/ (Accessed: 21.12.2024).Sustainable Development Goals Classification. URL: https://www.digitalscience.com/resource/sustainable-development-goals-classification/ (accessed: 17.12.2024).Hook D. W., Porter S. J., Herzog C. Dimensions: Building Context for Search and Evaluation // Front. Res. Metr. Anal. Frontiers. 2018. Vol. 3. https://doi.org/10.3389/frma.2018.00023.Juergen Wastl et al. Contextualizing Sustainable Development Research: Using Dimensions to explore the global landscape of research on Sustainable Development Goals. 2020. URL: https://www.digital-science.com/resource/contextualizing-sustainabledevelopment-research/ (accessed: 17.12.2024).Sustainable Development Goals Classification. URL: https://www.digitalscience.com/resource/sustainable-development-goals-classification/ (Accessed: 17.12.2024).Yelmen I., Gunes A., Zontul M. Multi-Class Document Classification Using Lexical Ontology-Based Deep Learning // Applied Sciences. 2023. Vol. 13, № 10. P. 6139. https://doi.org/10.3390/app13106139.Juergen Wastl et al. Contextualizing Sustainable Development Research: Using Dimensions to explore the global landscape of research on Sustainable Development Goals. 2020. URL: https://www.digital-science.com/resource/contextualizing-sustainabledevelopment-research/ (Accessed: 17.12.2024).Li Q. et al. A Survey on Text Classification: From Traditional to Deep Learning // ACM Trans. Intell. Syst. Technol. 2022. Vol. 13, № 2. P. 1–41. https://doi.org/10.1145/3495162Yelmen I., Gunes A., Zontul M. Multi-Class Document Classification Using Lexical Ontology-Based Deep Learning // Applied Sciences. 2023. Vol. 13, № 10. P. 6139. https://doi.org/10.3390/app13106139.Discover and analyse research in context of the United Nations Sustainable Development Goals. URL: https://www.dimensions.ai/webinars/discover-and-analyseresearch-in-context-of-the-united-nations-sustainable-development-goals/ (accessed: 17.12.2024).Li Q. et al. A Survey on Text Classification: From Traditional to Deep Learning // ACM Trans. Intell. Syst. Technol. 2022. Vol. 13, № 2. P. 1–41. https://doi.org/10.1145/3495162.LaFleur M. T. Art is long, life is short: An SDG Classification System for DESA Publications: 159 // Working Papers. United Nations, Department of Economics and Social Affairs, 2019.Discover and analyse research in context of the United Nations Sustainable Development Goals. URL: https://www.dimensions.ai/webinars/discover-and-analyseresearch-in-context-of-the-united-nations-sustainable-development-goals/ (Accessed: 17.12.2024).LaFleur M. T. SDGClassy: Shell. 2022. URL: https://github.com/SeaCelo/SDGclassy (accessed: 17.12.2024).LaFleur M. T. Art is long, life is short: An SDG Classification System for DESA Publications: 159 // Working Papers. United Nations, Department of Economics and Social Affairs, 2019.Traag V. A., Waltman L., Van Eck N. J. From Louvain to Leiden: guaranteeing wellconnected communities // Sci Rep. 2019. Vol. 9. № 1. P. 5233. https://doi.org/10.1038/s41598-019-41695-z.LaFleur M. T. SDGClassy: Shell. 2022. URL: https://github.com/SeaCelo/SDGclassy (Accessed: 17.12.2024).Boyack K. W., Klavans R. A comparison of large-scale science models based on textual, direct citation and hybrid relatedness // Quantitative Science Studies. 2020. Vol. 1, № 4. P. 1570–1585. https://doi.org/10.1162/qss_a_00085.Traag V. A., Waltman L., Van Eck N. J. From Louvain to Leiden: guaranteeing wellconnected communities // Sci Rep. 2019. Vol. 9. № 1. P. 5233. https://doi.org/10.1038/s41598-019-41695-z.A more sustainable future for all: Introducing the UN Sustainable Development Goals in InCites. URL: https://clarivate.com/academia-government/blog/a-more-sustainable-futurefor-all-introducing-the-un-sustainable-development-goals-in-incites/ (accessed: 17.12.2024).Boyack K. W., Klavans R. A comparison of large-scale science models based on textual, direct citation and hybrid relatedness // Quantitative Science Studies. 2020. Vol. 1, № 4. P. 1570–1585. https://doi.org/10.1162/qss_a_00085.Törnberg P. Large Language Models Outperform Expert Coders and Supervised Classifiers at Annotating Political Social Media Messages // Social Science Computer Review. SAGE Publications Inc, 2024. P. 08944393241286471. https://doi.org/10.1177/08944393241286471.A more sustainable future for all: Introducing the UN Sustainable Development Goals in InCites. URL: https://clarivate.com/academia-government/blog/a-more-sustainable-futurefor-all-introducing-the-un-sustainable-development-goals-in-incites/ (Accessed: 17.12.2024).Stavropoulos A., Crone D., Grossmann I. Shadows of wisdom: Classifying meta-cognitive and morally-grounded narrative content via Large Language Models. OSF, 2023. https://doi.org/10.31234/osf.io/x2f4a.Törnberg P. Large Language Models Outperform Expert Coders and Supervised Classifiers at Annotating Political Social Media Messages // Social Science Computer Review. SAGE Publications Inc, 2024. P. 08944393241286471. https://doi.org/10.1177/08944393241286471.Flores Villanueva D. Application of neural language models for research article classification into sustainable development goals: Master of Science in Engineering. Pontificia Universidad Católica de Chile, 2022. https://doi.org/10.7764/tesisUC/ING/66404.Stavropoulos A., Crone D., Grossmann I. Shadows of wisdom: Classifying meta-cognitive and morally-grounded narrative content via Large Language Models. OSF, 2023. https://doi.org/10.31234/osf.io/x2f4a.Yin H., Aryani A., Nambiar N. Evaluating the Performance of Large Language Models for SDG Mapping (Technical Report): arXiv:2408.02201. arXiv, 2024. https://doi.org/10.48550/arXiv.2408.02201.Flores Villanueva D. Application of neural language models for research article classification into sustainable development goals: Master of Science in Engineering. Pontificia Universidad Católica de Chile, 2022. https://doi.org/10.7764/tesisUC/ING/66404.Raman R. et al. ChatGPT: Literate or intelligent about UN sustainable development goals? // PLOS ONE. Public Library of Science. 2024. Vol. 19, № 4. P. e0297521. https://doi.org/10.1371/journal.pone.0297521.Yin H., Aryani A., Nambiar N. Evaluating the Performance of Large Language Models for SDG Mapping (Technical Report): arXiv:2408.02201. arXiv, 2024. https://doi.org/10.48550/arXiv.2408.02201.Kosyakov D. V. Anatomy of the Abnormal Growth in the Number of Russian Publications in Conference Proceedings in Scopus // Sci. Tech. Inf. Proc. 2023. Vol. 50, № 2. P. 96–108. https://doi.org/10.3103/S0147688223020028.Raman R. et al. ChatGPT: Literate or intelligent about UN sustainable development goals? // PLOS ONE. Public Library of Science. 2024. Vol. 19, № 4. P. e0297521. https://doi.org/10.1371/journal.pone.0297521.Kosyakov D., Guskov A. Disciplinary and institutional shifts: decomposing deviations in the country-level proportions of conference papers in Scopus // Scientometrics. 2024. Vol. 129. P. 1697–1717. https://doi.org/10.1007/s11192-024-04943-2.Kosyakov D. V. Anatomy of the Abnormal Growth in the Number of Russian Publications in Conference Proceedings in Scopus // Sci. Tech. Inf. Proc. 2023. Vol. 50, № 2. P. 96–108. https://doi.org/10.3103/S0147688223020028.Kosyakov D., Guskov A. Disciplinary and institutional shifts: decomposing deviations in the country-level proportions of conference papers in Scopus // Scientometrics. 2024. Vol. 129. P. 1697–1717. https://doi.org/10.1007/s11192-024-04943-2.Kosyakov D., Guskov A. Disciplinary and institutional shifts: decomposing deviations in the country-level proportions of conference papers in Scopus // Scientometrics. 2024. Vol. 129. P. 1697–1717. https://doi.org/10.1007/s11192-024-04943-2.

The authors declare that there are no conflicts of interest present.