How to Set Proxy in Selenium

As the title suggests, this article will guide QA engineers on how they can set Proxy in Selenium.

Let’s begin by addressing the most fundamental question.

What is a proxy?

A proxy is an intermediary between client requests and server responses. Proxies are primarily used to ensure privacy and encapsulation between numerous interactive systems.

A proxy can also provide an added layer of security by operating as a firewall between client and web servers. This is especially useful when the websites that clients use have to be labeled as allowed or blocked based on the website content. Often, websites will block IPs that make too many requests, and proxies are a way to get around this.

Testing with Proxy Servers

Proxy servers are most useful for executing localization tests. Let’s say a tester wants to open an E-commerce website and check that the proper language settings and currency appear for users from a specific country.

An easy way to verify this is to access the website as a user would from a target location. Obviously, like most tests, it would be easier to automate this activity, especially when a website has to be checked from multiple locations.

Selenium is the most widely used tool for running automated browser tests. Essentially, developers can use Selenium to monitor browser and website behavior without opening and executing a full browser instance. This article will detail how to set up a proxy server and use it to access the website via Selenium.

Setting up a Proxy Server

Many free proxy servers are unauthenticated, which simply means that a username and password are not required.

An unauthenticated proxy server in Selenium can be set up with the following steps:

• Import Selenium WebDriver from the package
• Define the proxy server (IP:PORT)
• Set ChromeOptions()
• Add the proxy server argument to the options
• Add the options to the Chrome() instance

from selenium import webdriver
PROXY = "11.456.448.110:8080"
chrome_options = WebDriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(chrome_options=chrome_options)
chrome.get("https://www.google.com")

Use this Chrome WebDriver instance to execute tests that incorporate the proxy server. For example, the following code snippet tests to ensure that a search field shows the user’s current city as the default location.

def testUserLocationZurich(self):
self.chrome.get(self.url)
search = self.chrome.find_element_by_id('user-city')
self.assertIn('Zurich', search.text)

In order to test a website from multiple locations by making this code reusable across separate tests, define a method that takes the proxy IP address as an argument.

Testers can run tests using an unauthenticated server. However, if they wish to use an authenticated server, they can follow the procedure below.

Authenticated Proxy

Authenticated proxy servers can be tedious to use in automated tests as there’s no built-in way to pass along proxy server credentials in Selenium. As of now, there are two options to handle authenticated proxies. The right choice depends on the testers’ requirements. It depends on factors like the version of Selenium in use and the headless browser being used in tests.

Let’s learn how to use it simply. The best way to integrate authenticated proxies with Selenium is by using PhantomJS as a headless browser instead of the Chrome WebDriver.

However, it is also possible to add a browser extension that does the authentication for Selenium. While this approach is more complex, it can be used with the latest version of Selenium, which may be a requirement for some development teams.

The first step is creating a Chrome extension by including two files in an archive, named proxy.zip:

Background.js

var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: "YOUR_PROXY_ADDRESS",
port: parseInt(PROXY_PORT)
},
bypassList: ["foobar.com"]
}
};
chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});
function callbackFn(details) {
return {
authCredentials: {
username: "PROXY_USERNAME",
password: "PROXY_PASSWORD"
}
};
}
chrome.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
);

manifest.js

{
"version": "1.0.0",
"manifest_version": 3,
"name": "Chrome Proxy",
"permissions": [
"Proxy",
"Tabs",
"unlimitedStorage",
"Storage",
"<all_urls>",
"webRequest",
"webRequestBlocking"
],
"background": {
"scripts": ["background.js"]
},
"Minimum_chrome_version":"76.0.0"
}

The Chrome extension can be added to Selenium using the add_extension method:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_extension("proxy.zip")
driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
driver.get("http://google.com")
driver.close()

This example uses a single proxy server in the extension. To add more proxy servers, the tester would have to make further modifications to the chrome.proxy API. In case the tester is using a CI/CD server, they would have to be sure that the build machine has Chrome installed and the relevant browser extension added.