Bulk website crawler needed

Hi, do you want the script, or you want results? I can run this on my side if needed .

Hi I can develop a desktop application in C# that can crawl any "site" and extract the "email/phone" The tool can be multi-threading for fast processing. The tool can be implemented in 3 days and it will costs 600 USD Can work on a demo if you like. No prior payment is required. Thanks

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I will use Python . Check out my profile, portfolio and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Best regards, Miljan

Hi. So, as input you will have a list of already deinfed urls and we need to to check that url and extract any email from the page? Are urls from different domains or the same? If from the same domain then we may need a proxy if we want to process this really fast. I want todo this in python with scrapy. Let me know if you are interested . Thx.

Hello, Hope you are doing well. I can help with you in your project Bulk website crawler. I can assure you the quality job. I have good experience in C Programming, Python, Scrapy, Web Crawling, Web Scraping. We have worked on several similar projects before! We have worked on 400+ Projects. Please check the profile reviews. I can deliver your job with in your deadline. Please ping me for more discussion. I can assure the 100% job satisfaction. Thanks,

hello there, i would like to help you with this project, i will write in python. but please can u give me about urls? the structure of the urls are the same#? or i should check some of them first to unnderstand pattern and then i can start to code.. please let me know and we can discuss further cheers Amadeus

Hi! My name is Guillermo, I represent Aurora Studio and we would like to help you with your project! I would develop the crawler in C#, if you are looking to work on a Linux environment you can use mono for cross-platform. The latency to process a webpage is almost nonexistent unless they are extremely big, processing time is mostly based on bandwidth and latency to the remote server, therefor the time constraint is out of the hands of the programmer. A parallel approach could be created to maximize the use of bandwidth but you can prone to packet loss (slower scrapping time) if you exceed your own bandwidth limit. I'm in the chat if you need me. If you have any questions, feel free to ask! Guillermo Andrade

Sir, I am well versed in these kind of jobs and can do your project as per requirement. **I am ready to start Waiting to hear from you. with thanks and regards Relevant Skills and Experience Python, scrapy

bulk website crawler needed yes we can start please intitate message High Quality + Fast Speed = Excellent Result + Business Success, this is my working style. I have gone through your Job post and I can understand your job requirement thoroughly. I have a total of 15 years of experience in Web Designing and Development and had completed a number of projects with some great graphics and User Interface so far. I have all the required skills and experience you need for the above Job. I have strong command over: * WordPress, PHP, Wordpress themeing, Plugin Development * Android and IOS all kind of mobile apps development *Responsive theme Design * HTML5, CSS3 , Jquery, Bootsrtap, Git, * Widget Development * Other CMS: Magento, Joomla, Expression Engine, Drupal etc. * I’m honest & trustworthy, dependable & fast learner. * I’ve over 7 years experience in Wordpress Website designing/development. * I am available 40 hours a week for your job. You can be assured of a quality communication and the quality of the work provided from my end. I’m looking forward to hearing from you soon. Thank you for considering my cover letter.

Hello there, Myself Prakhar, i am working in python for last 3 years. I have read your description thoroughly and i am confident that i can do this easily. Let's discuss further in personal chat. Regards Prakhar.

Python Language Scrapy Library We can do this with Python, if you provide a linux server it will be very easy to run the script. I have experience in web scraping of Election Commision Website of India . more than 300 Million data scraped Where we need to store the data? as CSV or Mysql ? Please come for a quick chat.. Im online Now

Hello, Kindly send me a message in order to discuss more details about your project. I can't pretend that I m an expert if I don t have enough data to start with. I d like to writr my scripts in Python but I can get around other languages as well. Thank you!

Hello, read your description and want you to know that I can help you with the task. I'm a professional computer scientist with expertise in web crawling. we can build this tasks in C# and can use multi programming to make the program faster. I'm sure to provide you quality work. your satisfaction is guaranteed. We can discuss further details in pm. Looking forward to hear back from you. kind regards, Zeeshan Ahmed

I am confident I am the right candidate for this project as I have done many similar projects in the past. With years of experience in this field, I believe this project will be very easy for me. I will be using C# to create the crawler for you.

Greetings! I will like to work on this project. I am Web application and software developer having many years of experience. In past i have worked in various project so i have gain knowledge about implementing various libraries, apis and debugging application. Skill Java, php, javascript,react.js, node.js, C++ C# and different other web technologies such as css html node.js ajax json and various software development techniques and methodology. Pm for more detail and budget discussion. Thank You!Have a good day

Hi, I am a linux administrator and programmer. I am thinking to use wget and shell or nodejs. Shell or nodejs is the controlling software and spawns wget, extract email address from caught contents. Thank you.

Hi, I can do it as soon as possible. I have good prior experience in scraping. I already have a scrapy framework that can perform this task in minimum time. Thanks Amit

I offer you a crawler in parallel python. May not be as fast as C, but parallel. Can try it out yourself on your 4-core laptop, or choose 35-100 concurrent processes to run. Don’t know how super fast it is going to be, so that’s why I am offering my crawler to you for cheap. So you can may be hire me and use the remaining funds to hire a C person to write another crawler and do a horse race between the two crawlers.

HI, I have many scrapy project to use scrapy: Project A.30 Vedio web site Spders 1. Use Scrapy and PhantomJS and Selenium to crawl 15 web site about video, such as youtube, insquire, sina, CCTV. 2. Use Python Django to save data to mysql. 3. Use different skills to defend the forbid of these web site, such as multi ips, http proxy, cookies settings. Project B. Five real-time information Spiders. 1. Use Scrapy to crawl five real-time information web site, such horse match, weather, flight aware, who score. 2. Use different skills to resolve the defender of these web site. 3. Use python django framework to save the data to mysql. 4. Run in the aws round by round Thanks