Development of recommendation systems to improve the efficiency of regulated procurement in the electric power industry
Abstract
This article considers ways to improve the efficiency of the regulated procurement market by implementing recommender systems into the existing procurement IT infrastructure. Using state, municipal and commercial procurement of electric power products as an example, the article considers promising classes of recommender systems for implementation, proposes a methodology for developing such services, and discloses algorithms for processing, configuring and interpreting data necessary for their operation. The difference between the author’s approach to creating services and previously published works is substantiated, testing and A/B testing are carried out, and an assessment of the effectiveness is presented. The results obtained have scientific novelty (the methodology of using neural networks in relation to the procurement industry has been substantiated) and practical significance (the customer’s time saved on searching for suppliers by up to 40%; the pool of potential suppliers has been expanded; supplier risks have been diversified by selecting relevant procedures from new areas and from new customers; suppliers have been provided with the opportunity to find up to 2–3 new customers for 1 recommendation mailing with a frequency of 1–2 times a week). We proposed to implement the developments in the practice of the operator of public procurement tenders. The authors see further development of recommendation services and solutions for the procurement industry in improving the analysis of semantic (text, logical) content of procurement documents, as well as the behavioral strategies of suppliers. The risks and limitations are associated with the high cost of maintaining a staff of developers-practitioners in neural networks, possible hallucinations of neural networks and their high sensitivity to errors and original data sets.
Downloads
References
Garant System (2014) Resolution of the Government of the Russian Federation of April 15, 2014 No. 320 “On approval of the state program of the Russian Federation “Public Finance Management and Regulation of Financial Markets” (with amendments and additions). Available at: https://base.garant.ru/70644234 (accessed 10 October 2024) (in Russian).
Denisova A.I., Sozaeva D.A., Gonchar K.V., Aleksandrov G.A. (2024) Analysis of factors for failure to close tenders for public procurement of drugs. Problems of Forecasting, no. 2, pp. 178–191 (in Russian). https://doi.org/10.47711/0868-6351-203-178-191
Ministry of Finance of Russia (2024) Consolidated analytical report on the results of monitoring the procurement of goods, works, services to meet state and municipal needs, as well as the procurement of goods, works, services by certain types of legal entities for 2023. Available at: https://minfin.gov.ru/common/upload/library/2024/05/main/Svodnyy_2023.pdf (accessed 10 October 2024) (in Russian).
Ministry of Finance of Russia (2024) Quarterly report on the results of monitoring the procurement of goods, works, services to meet state and municipal needs, as well as the procurement of goods, works, services by certain types of legal entities for the second quarter of 2024. Available at: https://minfin.gov.ru/common/upload/library/2024/07/main/II_kvartal_2024_g.pdf (accessed 10 October 2024) (in Russian).
Kuznetsov M. (2024) What awaits the public procurement market in 2024. Rossiyskaya Gazeta – Economy of the Central District, no. 4(9246). Available at: https://rg.ru/2024/01/11/reg-cfo/delo-za-malym.html (accessed 10 October 2024) (in Russian).
Yakovlev A.A., Balaeva O.N., Rodionova Yu.D., Tkachenko A.V. (2020) Problems and efficiency of public procurement through the eyes of their participants. All-Russian economic journal ECO, no. 11, pp. 83–103 (in Russian). https://doi.org/10.30680/ECO0131-7652-2020-11-83-103
Falk K. (2020) Recommender systems in practice. Moscow: DMK Press (in Russian).
Baranov V.V. (2005) Processes of making management decisions motivated by interests. Moscow: Fizmatlit (in Russian).
Kravchenko T.K., Isaev D.V. (2024) Decision support systems: textbook and practical training for universities. 2nd edition, revised and additional edition. Moscow: Yurait (in Russian).
Keeney R.L., Raiffa H. (1976) Decisions with multiple objectives: Preferences and value tradeoffs. John Wiley & Sons.
Thaler R. (2022) New behavioral economics. Why people break the rules of traditional economics. Moscow: Eksmo (in Russian).
García Rodríguez M.J., Montequín V., Ortega-Fernández F., Balsera J. (2020) Bidders recommender for public procurement auctions using machine learning: data analysis, algorithm, and case study with tenders from Spain. Complexity, article 8858258. https://doi.org/10.1155/2020/8858258
Hickok M. (2024) Public procurement of artificial intelligence systems: new risks and future proofing. AI & Society, vol. 39, pp. 1213–1227. https://doi.org/10.1007/s00146-022-01572-2
Seliverstov Y.A., Komissarov A.A., Lesovodskaya A.A., et al. (2022) Development and research of models of multi-class classifiers for a recommendrd system for preparing applications on the e-procurement. Computing, Telecommunications and Control, vol. 15, no. 2, pp. 43–62 (in Russian). https://doi.org/10.18721/JCSTCS.15204
Chen J., Dong H., Wang X., et al. (2023) Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems, vol. 41, no. 3, article 67. https://doi.org/10.1145/3564284
Ko H., Lee S., Park Y., Choi A. (2022) A survey of recommendation systems: Recommendation models, techniques, and application fields. Electronics, vol. 11, no. 1, article 141. https://doi.org/10.3390/electronics11010141
Andrianov N.A., Atakhodjaeva M.R., Borodin E.I. (2022) Mathematical modeling of a recommender system and data processing of a telecommunications company using machine learning models. Bulletin of the South Ural State University. Series: Computer technologies, management, radio electronics, vol. 22, no. 2, pp. 17–28 (in Russian). https://doi.org/10.14529/ctcr220202
Denisova A.I., Sozaeva D.A., Gonchar K.V., Aleksandrov G.A. (2023) Improving the methodology for assessing the economic efficiency of public e-procurement of medicines. Financial Journal, vol. 15, no. 4, pp. 63–81 (in Russian). https://doi.org/10.31107/2075-1990-2023-4-63-81
Denisova A.I., Sozaeva D.A., Gonchar K.V., Aleksandrov G.A. (2024) Analysis of factors for non-closure of tenders for public procurement of medicines. Problems of Forecasting, no. 2, pp. 178–191 (in Russian). https://doi.org/10.47711/0868-6351-203-178-191
ConsultantPlus (2024) Resolution of the Government of the Russian Federation of 08.06.2018 N 656 (as amended on 23.12.2024) "On the requirements for operators of electronic platforms, operators of specialized electronic platforms, electronic platforms, specialized electronic platforms and the functioning of electronic platforms, specialized electronic platforms, confirmation of compliance with such requirements, on the loss of the status of an operator of an electronic platform, operator of a specialized electronic platform by a legal entity". Available at: https://www.consultant.ru/document/cons_doc_LAW_300361/ (accessed 10 October 2024) (in Russian).
scikit-learn: Machine Learning in Python (2025) RandomForestClassifier. Available at: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html (accessed 11 February 2025).
CatBoost library (2025) CatBoostClassifier. Available at: https://catboost.ai/docs/en/concepts/python-reference_catboostclassifier (accessed 11 February 2025).
PyTorch (2025) torch.nn. Available at: https://pytorch.org/docs/stable/nn.html (accessed 11 February 2025).
scikit-learn: Machine Learning in Python (2025) Ensembles: Gradient boosting, random forests, bagging, voting, stacking. Available at: https://scikit-learn.org/stable/modules/ensemble.html (accessed 11 February 2025).
Lin T.-Y., Goyal P., Girshick R., He K., Dollár P. (2017) Focal loss for dense object detection. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2999–3007. https://doi.org/10.1109/ICCV.2017.324
Statista (2025) Content available on Disney's new streaming service's (Disney+) compared to Netflix in the United States as of April 2019. Available at: https://www.statista.com/statistics/1041104/disney-catalog-netflix-comparison-us/ (accessed 10 October 2024).
Safronov A. (2019) Netflix US subscribers fall for first time in 8 years. RBC Investments. Available at: https://quote.ru/news/article/5d3013649a794752c4a31fee (accessed 10 October 2024).
Morley M. (2025) Netflix library by the numbers 2024: 589 new originals released and library swells to over 7,000 titles. What's on Netflix. Available at: https://www.whats-on-netflix.com/news/netflix-library-by-the-numbers-2024/ (accessed 10 May 2025).
Moore K. (2025) Netflix added over 10,000 episodes of TV in 2024. What's on Netflix. Available at: https://www.whats-on-netflix.com/news/netflix-added-over-10000-episodes-of-tv-in-2024/ (accessed 10 May 2025).
Statista (2025) Number of Netflix paying streaming subscribers worldwide as of 4th quarter 2024, by region. Available at: https://www.statista.com/statistics/483112/netflix-subscribers/ (accessed 10 May 2025).
Alliance in the field of artificial intelligence (2020) Recommender system in the field of electronic procurement. Available at: https://ai-russia.ru/library/roseltorg-system (accessed 10 October 2024).
