Problem with deploying my python script using Selenium

I m trying to deploy my webpage but while i cannot figure out how i manage the buildpacks from heroku if i want to deploy my page here.
Here is the code i used: chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(“–headless”)
chrome_options.add_argument(“–disable-dev-shm-usage”)
chrome_options.add_argument(“–no-sandbox”)
chrome_options.binary_location = os.environ.get(“GOOGLE_CHROME_BIN”)
service = ChromeService(executable_path=os.environ.get(“CHROMEDRIVER_PATH”))
driver = webdriver.Chrome(service=service, options=chrome_options)
# Navigate to the webpage
driver.get(
https://www.vanguard.co.uk/professional/product/etf/equity/9679/ftse-all-world-ucits-etf-usd-accumulating’).
Also about the environment variables for google chrome and chromedriver, should i put the path that they exist in my local environment?

Hi there,

You most likely used the Chrome buildpack on Heroku to add Chrome to Heroku’s stack image. You will need to do something similar on Render by adding a build script something like this one: https://gist.github.com/BigAlRender/41f4c4d87df3e25770e3db8db728443e. If you add Chrome to the path, you should need to use the environment variables.

You will need to include a similar thing for the Chrome Driver. To find the locations of Chrome Driver downloads, look here: https://chromedriver.chromium.org/downloads.

Regards,

Keith
Render Support, UTC+10 :australia:

I got this working (Django+Celery).

For the last 6 months or so, selenium automatically locates and sets up the driver automatically, you don’t need to download or setup anything in the code (for the Webdriver).

You just need to make sure your Render VM has chrome installed and in the PATH

I have a build script for my worker VM:

#!/usr/bin/env bash
# exit on error
set -o errexit

#Chrome
STORAGE_DIR=/opt/render/project/.render

if [[ ! -d $STORAGE_DIR/chrome ]]; then
  echo "...Downloading Chrome"
  mkdir -p $STORAGE_DIR/chrome
  cd $STORAGE_DIR/chrome
  wget -P ./ https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
  dpkg -x ./google-chrome-stable_current_amd64.deb $STORAGE_DIR/chrome
  rm ./google-chrome-stable_current_amd64.deb
  cd $HOME/project/src # Make sure we return to where we were
else
  echo "...Using Chrome from cache"
fi

#requirements
pip install -r requirements.txt

Then my python code:

options = webdriver.ChromeOptions()
    options.add_argument('--headless=new')
    options.add_argument('--ignore-certificate-errors')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--disable-gpu')

    driver = webdriver.Chrome(options=options)
    driver.set_page_load_timeout(90)

    driver.get('https://www.xe.com/currencyconverter/convert/?Amount=1&From=USD&To=CAD')

    min_exchange_rate = 1.3
    max_exchange_rate = 1.4

    element = driver.find_element(by=By.XPATH, value='//*[@id="__next"]/div[3]/div[2]/section/div[2]/div/main/div/div[2]/div[1]/p[2]')

    exchange_rate = float(element.text.split(' ')[0])

    driver.quit()  # maybe close instead?

Hello Mowbray.
I would love to have some more infos, if you could.
How do you set up your enviroments on settings?
Don’t you need to import those on your code?
How do you start your .sh file?

thankss

Hey Luan Paganucci,

The only environment setting that needs to be made is adding the chrome binary to the $PATH. You don’t have to do anything special in the code, you can just import and use selenium.

My worker VM start.sh (I put it in the root of my git repo).

#!/usr/bin/env bash
# exit on error
set -o errexit

# add Chrome's location to the PATH as part of your Start Command
export PATH="/opt/render/project/.render/chrome/opt/google/chrome:$PATH"
echo "Current PATH: $PATH"

celery -A mydjangoproject worker --beat --scheduler django --loglevel=info --concurrency 4

To tell Render to run this script, you can either use the render.yaml or the web interface for the VM.

  - type: worker
    name: mice-prod-worker
    region: ohio
    env: python
    buildCommand: "./render-worker-build.sh"
    startCommand: "./render-worker-start.sh"