web-crawler-example

This repository contains examples that show how to apply web crawlers to scrape web pages.

Examples

Scrapy

Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.

• Craigslist Vacation House Data Crawling
  In this example we use Scrapy collect data from Craigslist.com. Specifically, the data under craigslist.org/Seattle/housing/vacation rentals. You can find the page under the link: https://seattle.craigslist.org/d/vacation‐rentals/search/vac

  In the example, I collected following information:

  1. Title
  2. Posted Date
  3. Rental Price
  4. Number of bedrooms
  5. Neighborhood
  6. Description