Web scraping that supports JavaScript, proxies and cookies.
Requirements:
Python (>= 2.7 is preferable)
requests
selenium
BeautifulSoup4
Usage:
If you would like to test this package, please try
python main.py
You can obtain an output named 'output.json' storing the title and description of http://mainichi.jp.
The file main.py provides a simple example how to use the class pyScraping.
References:
The Japanese description available online:
http://qiita.com/_akisato/items/2daafdbc3de544cf6c92