"pip install" installs browsermob-proxy 0.8.0, not 2.0 or 2.1.5

[tslever]

I'm trying to load webpages, wait until they've finished loading, scroll to their bottoms, and wait for until they've finished loading. Because the load times are indeterminate, I am using "browsermob-proxy.Client.wait_for_traffic_to_stop" to wait for traffic to stop before scrolling. I am using Python via Anaconda and Jupyter-Notebook. Given a quiet period of 0.1 seconds and a timeout of 20 seconds, my code consistently prints below my Jupyter-Notebook cell a duration of about 19.9 seconds, which is the difference between the wait time limit and the length of the quiet period, which I think is basically the amount of time wait_for_traffic_to_stop is allowed to wait.

That all being said, in Anaconda Prompt, I executed the command "pip install git+https://github.com/AutomatedTester/browsermob-proxy-py.git" to try to install browsermob-proxy 2.0. Despite browsermob-proxy-py's documentation seeming to imply that my command would install browsermob-proxy 2.0, my command installed browsermob-proxy 0.8.0. Given a post in the thread at "lightbody/browsermob-proxy#212", it seems that the above time-out issue was fixed in browsermob-proxy 2. How do I install browsermob-proxy 2 or later via Anaconda Prompt?

[mctwynne]

@tslever I think you might be mixing up browsermob-proxy-py, the Python package, and browsermob-proxy, the proxy itself. If you're looking for version 2.0+ of browsermob-proxy you can find and download it here.

[jayden-xie]

The last release of Python package is 2 years ago, https://pypi.org/project/browsermob-proxy/#history
@AutomatedTester When will you push the latest code to pip? Including fix for killing Java process? Thanks.

[tslever]

Hi Jayden.

Thanks for getting back to me.

I would like to "pip install" what I believe needs to be a browsermob-proxy Python-3 package via the most up-to-date Anaconda Prompt. Please see below Python-3 module which today outputted a "duration", equal to the allowed time to wait of 19.916 seconds, when waiting for a quiet period from Google.com.

# Find_Time_to_Load_Web_Page.py
#
# Created: 01/09/19 by Tom Lever
# Updated: 04/16/19 by Tom Lever
#
# This program attempts to find the time it takes to load a web page.
# Unfortunately, this program relies on browsermob-proxy which,
# in waiting for data traffic associated with web-page loading to stop,
# times out for, likely, "https://" web pages.
#
# Inputs: None
# Dependencies: browsermobproxy.Server, selenium.webdriver, time
# Outputs: At this time, to screen, the time-out value.

# Enable creating browsermobproxy.Server class instances.
from browsermobproxy import Server
`# Enable creating selenium.webdriver class instances.` `from selenium import webdriver`
# Enable creating time class instances.
import time

# Initialize the path to a proxy server batch file for creating a Server class instance.
path_to_server_batch = "C:/Users/thoma/Python/Web_Scraping/browsermob-proxy-2.1.4/bin/browsermob-proxy.bat"
`# Initialize the path to the chromedriver executable.` `path_to_chromedriver = "C:/Users/thoma/Python/Web_Scraping/chromedriver.exe"`
# Create the server class instance.
server = Server(path_to_server_batch)
`# "Start" the server.` `server.start()`
# Create a proxy.
proxy = server.create_proxy()
`# Have the proxy server load a Chrome browser.` `chrome_options = webdriver.ChromeOptions()` `chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy))` `driver = webdriver.Chrome(executable_path=path_to_chromedriver, options=chrome_options)`
# Have the proxy server have Chrome load a specific webpage.
driver.get("https://google.com/")
`# Record the present time.` `start_time = time.time()`
# Wait for traffic to stop.
status_code = proxy.wait_for_traffic_to_stop(100, 20000)
`# Print the time between load beginning and traffic end, or wait time limit.` `duration = time.time() - start_time` `print(duration)`
# Shut down use of the server. Erase the server class instance.
server.stop()
``
# Close the Google-loaded Chrome browser.
`driver.quit()
`# driver.close()`

[jayden-xie]

Killing the Java process of proxy is quite a problem now, on both Windows and Linux, because we have tests running in parallel. If the Java process got ability to self-destruct after 5 minutes idle time, it would be nice!
Thank you.