I’m little further down the road in my training and want to take that Python Web Scraper script written on my Windows PC using Python 3 and run it on a Kali Linux VM which apparently is running Python 2.7.
My Original Python 2.7 Web Scraper Script
I fired up an old Kali VM that had my original python web scraper code and was able to publish it on my Github.
You can now get the original code and update it using the rest of this article.
So as with all things tech, it won’t run without some tweaking.
This is how I got it to work!
First, there is a difference in how you execute Python scripts in Linux. With Kali Linux, you seem to have several versions installed by default. Looks like python 2.7 is the directory where most of the goodies including the BeautifulSoup package my scraper needs were so I’m using this one.
Example of how to call python 2.7 then pass the script that is located in the “Desktop” folder to it.
The only code I had to update one line of code and that was the reference to the urllib.
Why? Because the urllib and urllib2 packages have been split into urllib.request , urllib.parse and urllib.error packages in Python 3.x. The latter packages do not exist in Python 2.x
We are going backwards in version and
Old code: from urllib.request import urlopen as uReq
New Python Library Reference
from urllib2 import urlopen as uReq
