Advanced Methods for Data Retrieval and Analysis through Internet APIs, Web Scraping, and Machine Learning: Development of a Software Solution for Comparative Price Analysis

Abstract

The report examines the use of Internet APIs, web scraping, and machine learning for automated data extraction and processing. It presents the principles of REST APIs, tools for web scraping such as Scrapy and Selenium, as well as machine learning algorithms, including neural networks and classification models. The report discusses the challenges of handling dynamic content, managing
large data volumes, and the ethical aspects of web scraping. It also analyzes the prospects for data automation and the enhancement of analytical models. The conclusion emphasizes the importance of integrating these technologies for more efficient information management across various industries. 
During the preparation of the report, a software solution was developed to automate the comparative analysis of prices for identical products across multiple online retailers.