Custom CDP Functions - Bright Data Docs

Captcha Solver

When navigating a page with Scraping Browser, our integrated CAPTCHA solver automatically solves all CAPTCHAs by default. You can monitor this auto-solving process in your code with the following custom CDP functions.

If you would like to disable CAPTCHA solver entirely through the Control Panel see our feature for Disable Captcha Solver

Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default.

CAPTCHA Solver - Automatic Solve

Captcha.solve

Use this command to return the status after the captcha was solved, failed, or not detected. Use this command to return the status after the captcha was solved, failed, or not detected.

Captcha.solve({
    detectTimeout?: number // Detect timeout in millisecond for solver to detect captcha  
    options?: CaptchaOptions[] // Configuration options for captcha solving  
}) : SolveResult

SolveResult : {  
  status: SolveStatus // Detect and solve status  
  type?: string // Detected captcha type  
  error?: string // Error if captcha was not solved  
}

SolveStatus : string enum {  
  "not_detected" // Captcha was not detected  
  "solve_finished" // Captcha successfully solved  
  "solve_failed" // Captcha detected, but was not solved  
  "invalid" // Something goes wrong  
}

Examples

const page = await browser.newPage();
const client = await page.target().createCDPSession();
await page.goto('[https://site-with-captcha.com](https://site-with-captcha.com/)');  

// Note 1: If no captcha was found it will return not_detected status after detectTimeout   
// Note 2: Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default   

const client = await page.target().createCDPSession();  
const {status} = await client.send('Captcha.solve', {detectTimeout: 30*1000});   
console.log(`Captcha solve status: ${status}`)

page = await browser.new_page()   
client = await page.context.new_cdp_session(page)   
await page.goto('[https://site-with-captcha.com](https://site-with-captcha.com/)')  
# Note 1: If no captcha was found it will return not_detected status after detectTimeout   
# Note 2: Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default  
client = await page.context.new_cdp_session(page)  
solve_result = await client.send('Captcha.solve', { 'detectTimeout': 30*1000 })   
status = solve_result['status']   
print(f'Captcha solve status: {status}') 

If CAPTCHA-solving fails, please attempt a retry. If the issue persists, submit a support reques detailing the specific problem you encountered.

Custom CDP commands for CAPTCHA status

Use the commands below to pinpoint a more specific stage in the CAPTCHA solving flow:


`Captcha.detected`	Scraping Browser has encountered a CAPTCHA and has begun to solve it
`Captcha.solveFinished`	Scraping Browser successfully solved the CAPTCHA
`Captcha.solveFailed`	Scraping Browser failed in solving the CAPTCHA
`Captcha.waitForSolve`	Scraping Browser waits for CAPTCHA solver to finish

Examples

Asynchronous
Synchronous

The following code sets up a CDP session, listens for CAPTCHA events, and handles timeouts:

// Node.js - Puppeteer - waiting for CAPTCHA solving events  
const client = await page.target().createCDPSession();   
await new Promise((resolve, reject)=>{   
  client.on('Captcha.solveFinished', resolve);   
  client.on('Captcha.solveFailed', ()=>reject(new Error('Captcha failed')));   
  setTimeout(reject, 5 * 60 * 1000, new Error('Captcha solve timeout'));  
});

# Python - Playwright - waiting for CAPTCHA solving events  
client = await page.context.new_cdp_session(page)  
client.on('Captcha.detected', lambda c: print('Captcha detected', c))   
client.on('Captcha.solveFinished', lambda _: print('Captcha solved!'))   
client.on('Captcha.solveFailed', lambda _: print('Captcha failed!'))

Selenium doesn’t support asynchronous server-driven events like Puppeteer and Playwright.

The Captcha.waitForSolve command waits for Scraping Browser’s CAPTCHA solver to finish.

Python - Selenium

# Python Selenium - Waiting for Captcha to auto-solve after navigate  
driver.execute('executeCdpCommand', {  
    'cmd': 'Captcha.waitForSolve',  
    'params': {},  
})

CAPTCHA Solver - Manual Control

If you would like to either manually configure or fully disable our default CAPTCHA solver and instead call the solver manually or solve on your own, see the following CDP commands and functionality.

Captcha.setAutoSolve

This command is used to control the auto-solving of a CAPTCHA. You can disable auto-solve or configure algorithms for different CAPTCHA types and manually trigger this:

Captcha.setAutoSolve({  
  autoSolve: boolean // Whether to automatically solve captcha after navigate  
  options?: CaptchaOptions[] // Configuration options for captcha auto-solving  
}) : void

CaptchaOptions : {  
  type: string // Captcha type  
  disabled?: boolean // Disable detect and solve for specified captcha  
  ... // additinal options related to captcha type  
}

Examples of CDP commands to disable auto-solver completely within the session:

// Node.js Puppeteer - Disable Captcha auto-solver completely  
const page = await browser.newPage();  
const client = await page.target().createCDPSession();  
await client.send('Captcha.setAutoSolve', { autoSolve: false })

# Python Playwright - Disable Captcha auto-solver completely  
page = await browser.new_page()  
client = await page.context.new_cdp_session(page)  
await client.send('Captcha.setAutoSolve', {'autoSolve': False}):

# Python Selenium - Disable Captcha auto-solver completely  
driver.execute('executeCdpCommand', {  
    'cmd': 'Captcha.setAutoSolve',  
    'params': {'autoSolve': False},  
})

Disable auto-solver for a specific CAPTCHA type only - Examples

// Node.js Puppeteer - Disable Captcha auto-solver for ReCaptcha only  
const page = await browser.newPage();  
const client = await page.target().createCDPSession();  
await client.send('Captcha.setAutoSolve', {  
    autoSolve: true,  
    options: [{  
        type: 'usercaptcha',  
        disabled: true,  
    }],  
});

# Python Playwright - Disable Captcha auto-solver for ReCaptcha only  
page = await browser.new_page()  
client = await page.context.new_cdp_session(page)  
await client.send('Captcha.setAutoSolve', {  
    'autoSolve': True,  
    'options': [{  
        'type': 'usercaptcha',  
        'disabled': True,  
    }],  
})

Manually solving CAPTCHAs - Examples

// Node.js Puppeteer - manually solving CAPTCHA after navigation  
const page = await browser.newPage();  
const client = await page.target().createCDPSession();  
await client.send('Captcha.setAutoSolve', { autoSolve: false });  
await page.goto('https://site-with-captcha.com', { timeout: 2*60*1000 });  
const {status} = await client.send('Captcha.solve', { detectTimeout: 30*1000 });  
console.log('Captcha solve status:', status);

# Python Playwright- manually solving CAPTCHA after navigation  
page = await browser.new_page()  
client = await page.context.new_cdp_session(page)  
await client.send('Captcha.setAutoSolve', {'autoSolve': False})  
await page.goto('https://site-with-captcha.com', timeout=2*60_000)  
solve_result = await client.send('Captcha.solve', {'detectTimeout': 30_000})  
print('Captcha solve status:', solve_result['status'])

# Python Selenium - manually solving CAPTCHA after navigation  
driver.execute('executeCdpCommand', {  
    'cmd': 'Captcha.setAutoSolve',  
    'params': {'autoSolve': False},  
})  
driver.get('https://site-with-captcha.com')  
solve_result = driver.execute('executeCdpCommand', {  
    'cmd': 'Captcha.solve',  
    'params': {'detectTimeout': 30_000},  
})  
print('Captcha solve status:', solve_result['value']['status'])

CaptchaOptions for other Captcha types

For the following three CAPTCHA types we support the following additional options to control and configure our auto-solving algorithm.

CF Challenge
HCaptcha
usercaptcha (reCAPTCHA)

CF Challenge

timeout: 40000  
selector: '#challenge-body-text, .challenge-form'  
check_timeout: 300  
error_selector: '#challenge-error-title'  
success_selector: '#challenge-success[style*=inline]'  
check_success_timeout: 300  
btn_selector: '#challenge-stage input[type=button]'  
cloudflare_checkbox_frame_selector: '#turnstile-wrapper iframe'  
checkbox_area_selector: '.ctp-checkbox-label .mark'  
wait_timeout_after_solve: 500  
wait_networkidle: {timeout: 500}

HCaptcha

detect_selector:  
  '#cf-hcaptcha-container, #challenge-hcaptcha-wrapper .hcaptcha-box, .h-captcha'  
pass_proxy: true  
submit_form: true  
submit_selector: '#challenge-form body > form[action*="internalcaptcha/captchasubmit"]  
value_selector: '.h-captcha textarea[id^="h-captcha-response"]'  
  

UserCaptcha (reCAPTCHA)

{ // configuration keys and default values for reCAPTCHA (type=usercaptcha)  
  type: 'usercaptcha',  
  // selector to retrieve sitekey and/or action  
  selector: '.g-recaptcha, .recaptcha',  
  // attributes to search for sitekey  
  sitekey_attributes: ['data-sitekey', 'data-key'],  
  // attributes to search for action  
  action_attributes: ['data-action'],  
  // detect selectors  
  detect_selector: `  
    .g-recaptcha[data-sitekey] > *,  
    .recaptcha > *,  
    iframe[src*="[www.google.com/recaptcha/api2](http://www.google.com/recaptcha/api2)"],  
    iframe[src*="[www.recaptcha.net/recaptcha/api2](http://www.recaptcha.net/recaptcha/api2)"],  
    iframe[src*="[www.google.com/recaptcha/enterprise](http://www.google.com/recaptcha/enterprise)"]`,  
  // element to type response code into  
  reponse_selector: '#g-recaptcha-response, .g-recaptcha-response',  
  // should solver submit form automatically after captcha solved  
  submit_form: true,  
  // selector for submit button  
  submit_selector: '[type=submit]',  
}

Emulation Functions

Emulation.getSupportedDevices

Use this command to get a list of all possible devices that can be emulated. This method returns an array of device options that can be used with the setDevice command.

Example

Emulation.getSupportedDevices().then(devices => { console.log(devices);});

Emulation.setDevice

Once you’ve received the list above of supported devices, you can emulate a specific device using the Emulation.setDevice command. This command changes the screen width, height, userAgent, and devicePixelRatio to match the specified device.

Emulation.setDevice({device: '[device_name]'});

Emulation.setDevice({device: 'iPhone X'});

Landscape mode

If you wish to change the orientation to landscape (for devices that support it), add the string landscape after the device_name.

Example

Emulation.setDevice({device: 'iPhone X landscape'});

Custom Client SSL/TLS Certificates

Use this command to install custom client SSL/TLS certificates where required for specific domain authentication. These certificates are applied for the duration of a single Scraping Browser session and are automatically removed once the session ends.

Browser.addCertificate

Browser.addCertificate(params: {
  cert: string // base64 encoded certificate file
  pass: string // password for the certificate
}) : void

Code examples

Replace placeholder values SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD with your valid Scraping Browser credentials.
Replace client.pfx with the actual path to your certificate file. This file should be a valid SSL/TLS client certificate in .pfx format.

Replace secret with the actual password for the certificate.

const puppeteer = require('puppeteer-core');
const fs = require('fs/promises');
const {
  AUTH = 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD',
  TARGET_URL = 'https://example.com',
  CERT_FILE = 'client.pfx',
  CERT_PASS = 'secret',
} = process.env;

async function scrape(url = TARGET_URL, file = CERT_FILE, pass = CERT_PASS) {
  if (AUTH == 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD') {
    throw new Error(`Provide Scraping Browsers credentials in AUTH`
        + ` environment variable or update the script.`);
  }
  console.log(`Connecting to Browser...`);
  const browserWSEndpoint = `wss://${AUTH}@brd.superproxy.io:9222`;
  const browser = await puppeteer.connect({ browserWSEndpoint });
  try {
    console.log(`Connected! Installing ${file} certificate...`);
    const page = await browser.newPage();
    const client = await page.createCDPSession();
    const cert = (await fs.readFile(CERT_FILE)).toString('base64');
    await client.send('Browser.addCertificate', { cert, pass });
    console.log(`Installed! Navigating to ${url}...`);
    await page.goto(url, { timeout: 2 * 60 * 1000 });
    console.log(`Navigated! Scraping page content...`);
    const data = await page.content();
    console.log(`Scraped! Data: ${data}`);
  } finally {
    await browser.close();
  }
}

scrape();

from os import environ
from base64 import standard_b64encode
from selenium.webdriver import Remote, ChromeOptions as Options
from selenium.webdriver.chromium.remote_connection import ChromiumRemoteConnection as Connection

AUTH = environ.get('AUTH', default='SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD')
TARGET_URL = environ.get('TARGET_URL', default='https://example.com')
CERT_FILE = environ.get('CERT_FILE', default='client.pfx')
CERT_PASS = environ.get('CERT_PASS', default='secret')

def scrape(url=TARGET_URL, file=CERT_FILE, pswd=CERT_PASS):
    if AUTH == 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD':
    raise Exception('Provide Scraping Browsers credentials in AUTH '
                    'environment variable or update the script.')
    print('Connecting to Browser...')
    server_addr = f'https://{AUTH}@brd.superproxy.io:9515'
    connection = Connection(server_addr, 'goog', 'chrome')
    driver = Remote(connection, options=Options())
    try:
        print(f'Connected! Installing {file} certificate...')
        with open(file, 'rb') as f:
            cert = standard_b64encode(f.read()).decode()
        driver.execute('executeCdpCommand', {
            'cmd': 'Browser.addCertificate',
            'params': {'cert': cert, 'pass': pswd},
        })
        print(f'Installed! Navigating to {url}...')
        driver.get(url)
        print('Navigated! Scraping page content...')
        data = driver.page_source
        print(f'Scraped! Data: {data}')
    finally:
        driver.quit()

scrape()

​Captcha Solver

​CAPTCHA Solver - Automatic Solve

​CAPTCHA Solver - Manual Control

​Emulation Functions

​Landscape mode

​Custom Client SSL/TLS Certificates

Captcha Solver

CAPTCHA Solver - Automatic Solve

CAPTCHA Solver - Manual Control

Emulation Functions

Landscape mode

Custom Client SSL/TLS Certificates