Scraping Browser FAQs - Bright Data Docs

How do I configure Scraping Browser to work from a specific country?

When using the Scraping Browser, the same country-targeting parameter is available to use as in other Bright Data proxy products.Specific countryWhen setting up your script, add the -country flag, after your USER credentials within the Bright Data endpoint, followed by the 2-letter ISO code for that country.For example, Scraping Browser using Puppeteer in the USA:

const SBR_WS_ENDPOINT = `wss://${USERNAME-country-us:PASSWORD}@brd.superproxy.io:9222`;

EU regionEU region You can target the entire European Union region in the same manner as “Country” above by adding “eu” after “country” in your request: “-country-eu”. Requests sent using -country-eu, will use IPs from one of the countries below which are included automatically within “eu”: AL, AZ, KG, BA, UZ, BI, XK, SM, DE, AT, CH, UK, GB,IE, IM, FR, ES, NL, IT, PT, BE, AD, MT, MC, MA, LU, TN, DZ, GI, LI, SE, DK, FI, NO, AX, IS, GG, JE, EU, GL, VA, FX, FO.Need Scraping Browser to target a specific geographical radius of proxies?Check out our Proxy.setLocation/scraping-automation/scraping-browser/features/proxy-location) feature.

Which programming languages, libraries, and browser automation tools are supported by Scraping Browser?

Bright Data’s Scraping Browser is compatible with a wide variety of programming languages, libraries, and browser automation tools, offering full native support for Node.js, Python, and Java/C# using puppeteer, playwright, and selenium respectively.Other languages can usually be integrated as well via the third-party libraries listed below, enabling you to incorporate Scraping Browser directly into your existing tech stack.

Language/Platform	puppeteer	playwright	selenium
Python	N/A	*playwright-python	*Selenium WebDriver
JS / Node	*Native	*Native	*WebDriverJS
Java	Puppeteer Java	Playwright for Java	*Native
Ruby	Puppeteer-Ruby	playwright-ruby-client	Selenium WebDriver for Ruby
C#	*.NET: Puppeteer Sharp	Playwright for .NET	*Selenium WebDriver for .NET
Go	chromedp	playwright-go	Selenium WebDriver for Go
*Full support

How can I debug what's happening behind the scenes during my Scraping Browser session?

Where do I find the Scraping Browser Debugger?

How to automatically launch devtools locally to view your live browser session?

If you would like to automatically launch devtools on every session to view your live browser session, you can integrate the following code snippet:

NodeJS - Puppeteer

// Node.js Puppeteer - launch devtools locally  

const {
    exec
} = require('child_process');
const chromeExecutable = 'google-chrome';

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const openDevtools = async (page, client) => {
    // get current frameId  
    const frameId = page.mainFrame()._id;
    // get URL for devtools from scraping browser  
    const {
        url: inspectUrl
    } = await client.send('Page.inspect', {
        frameId
    });
    // open devtools URL in local chrome  
    exec(`"${chromeExecutable}" "${inspectUrl}"`, error => {
        if (error)
            throw new Error('Unable to open devtools: ' + error);
    });
    // wait for devtools ui to load  
    await delay(5000);
};

const page = await browser.newPage();
const client = await page.target().createCDPSession();
await openDevtools(page, client);
await page.goto('http://example.com');

Debugger WalkthroughCheck out the Scraping Browser Debugger in action below<inser-video-here>

How can I see a visual of what's happening in the browser?

You can easily trigger a screenshot of the browser at any time by adding the following to your code:

NodeJS

// node.js puppeteer - Taking screenshot to file screenshot.png 
await page.screenshot({ path: 'screenshot.png', fullPage: true });

To take screenshots on Python and C# see here.

Why does the initial navigation for certain pages take longer than others?

There is a lot of “behind the scenes” work that goes into unlocking your targeted site.Some sites will take just a few seconds for navigation, while others might take even up to a minute or two as they require more complex unlocking procedures. As such, we recommend setting your navigation timeout to “2 minutes” to give the navigation enough time to succeed if needed.You can set your navigation timeout to 2 minutes by adding the following line in your script before your “page.goto” call.

// node.js puppeteer - Navigate to site with 2 min timeout  
page.goto('<https://example.com>', { timeout: 2*60*1000 });

What are the most Common Error codes?


Error Code	Meaning	What can you do about it?
Unexpected server response: 407	An issue with the remote browser’s port	Please check your remote browser’s port. The correct port for Scraping Browser is port:9222
Unexpected server response: 403	Authentication Error	Check authentication credentials (username, password) and check that you are using the correct “Browser API” zone from Bright Data control panel
Unexpected server response: 503	Service Unavailable	We are likely scaling browsers right now to meet demand. Try to reconnect in 1 minute.

I can't seem to establish a connection with Scraping Browser, do I have a connection issue?

If you’re experiencing connection issues, you can test your local Scraping Browser connection with a simple curl to the following endpoint:

Shell

curl -v -u SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD https://brd.superproxy.io:9222/json/protocol

If a JSON is returned within the response:

Your authentication and connection to Scraping Browser are successful. Confirm you are using these exact configurations in your script.
If you are still facing issues connecting to Scraping Browser, contact support for further assistance.

If a JSON is not returned within the response:

Check your authentication details from your Scraping Browser zone, and ensure the USER and PASS values are correct.
Verify network Configuration: Confirm your network and/or firewall is not blocking outbound connections to https://brd.superproxy.io:9222.
If the issue persists, please contact support for further assistance.

How to Integrate Scraping Browser with .NET Puppeteer Sharp?

Integration with the Scraping browser product with C# requires patching the PuppeteerSharp library to add support for websocket authentication. This can be done like the following:

C# PuppeteerSharp

using PuppeteerSharp;  
using System.Net.WebSockets;  
using System.Text;  
  
// Set the authentication credentials  
var auth = "SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD";  
// Construct the WebSocket URL with authentication  
var ws = $"wss://{auth}@brd.superproxy.io:9222";  
// Custom WebSocket factory function  
async Task<WebSocket> ws_factory(Uri url, IConnectionOptions options, CancellationToken cancellationToken)  
  
{  
    // Create a new ClientWebSocket instance
    var socket = new ClientWebSocket();  
    // Extract the user information (username and password) from the URL  
    var user_info = url.UserInfo;  
    if (user_info != "")  
    {  
        // Encode the user information in Base64 format  
        var auth = Convert.ToBase64String(Encoding.Default.GetBytes(user_info));  
        // Set the "Authorization" header of the WebSocket options with the encoded credentials  
        socket.Options.SetRequestHeader("Authorization", $"Basic {auth}");  
    }  
  
    // Disable the WebSocket keep-alive interval  
    socket.Options.KeepAliveInterval = TimeSpan.Zero;  
    // Connect to the WebSocket endpoint  
    await socket.ConnectAsync(url, cancellationToken);  
    return socket;  
}  
  
// Create ConnectOptions and configure the options  
var options = new ConnectOptions()  
  
{  
    // Set the BrowserWSEndpoint to the WebSocket URL  
    BrowserWSEndpoint = ws,  
    // Set the WebSocketFactory to the custom factory function  
    WebSocketFactory = ws_factory,  
};  
  
// Connect to the browser using PuppeteerSharp  
Console.WriteLine("Connecting to browser...");  
  
using (var browser = await Puppeteer.ConnectAsync(options))  
{  
    Console.WriteLine("Connected! Navigating...");  
    // Create a new page instance  
    var page = await browser.NewPageAsync();  
    // Navigate to the specified URL  
    await page.GoToAsync("https://example.com");  
    Console.WriteLine("Navigated! Scraping data...");  
    // Get the content of the page  
    var content = await page.GetContentAsync();  
    Console.WriteLine("Done!");  
    Console.WriteLine(content);  
}

Which coding languages does Scraping Browser support?

Bright Data’s Scraping Browser supports a wide range of programming languages and libraries. We currently have full native support for Node.js and Python using puppeteer, playwright, and selenium, and other languages can be integrated as well using the other libraries below, giving you the flexibility to integrate Scraping Browser right into your current tech stack.

Language/Platform	puppeteer	playwright	selenium
Python	N/A	playwright-python	Selenium WebDriver
JS / Node	Native support	Native support	WebDriverJS
Java	Puppeteer Java	Playwright for Java	Native support
Ruby	Puppeteer-Ruby	playwright-ruby-client	Selenium WebDriver for Ruby
C#	.NET: Puppeteer Sharp	Playwright for .NET	Selenium WebDriver for .NET
Go	chromedp	playwright-go	Selenium WebDriver for Go