Web Scraping + File Integration with API -- 2

Project: Data scraping/extracting and importing into API Background: We are a dropshipping business that sells products on the Internet - on our own ecommerce website as well as Amazon. We need to ensure that the inventory levels for products are current. We source products from multiple suppliers that provide inventory data in different ways and in different frequencies. We need to gather all that data and then import it into our inventory software system. That system integrates with our e-commerce website and Amazon to ensure the latest inventory levels are correct. The separate data sources that need to be scraped/extracted include: 2 websites. Scrape search pages and product pages. Up to 3K products. Site formatting can be inconsistent and some pages require logic for variants. Data should be scraped on a short interval basis - every 30 minutes - and then pushed to database. Emailed CSV file. 1 file. Hundreds of products. Need app/macro to open email, download sheet and then scrape data. Email is G Mail so experience with G Mail scripts is helpful. 2 Google sheets. 50 to 100 products each. Need app/macro/solution to open sheets, extract data and then send to database. Experience with Google API is helpful. Needs to be done once per day. 1 txt file on FTP site. Hundreds of products. Need app/macro to open sheets, pull data and send to API. On an ongoing basis. FTP site updated every 15 minutes. Should be pulled every 30 minutes. All of this data will then need to be consolidated in a database or spreadsheet (Google Sheet0. It will be preferably for a spreadsheet as some of the data needs to be formatted/modified before being imported into Ecomdash. Once that data is updated, it needs to then be linked to Ecomdash through their API. Scope -Create app(s)/scripts that can perform the data scraping/extraction. -Ensure data is properly imported into tabs into Google Sheets. -Ensure formatted/consolidated data is then properly imported into Ecomdash via API. (Note: client will handle formatting/modifying data in Google Sheets). -Provide support in hosting app on appropriate site (i.e. Heroku) Experience needed: - Web scraping - App/Scripting for extracting data from Google Sheets, emailed CSV files in G Mail and txt files on FTP sites. - API integrations with Google Sheets - App hosting