headless mode is not working when setting-up by ChromeOptions add_argument('--headless')

[yangkwch]

Sample code as below:

import undetected_chromedriver as uc
from webdriver_manager.core.os_manager import OperationSystemManager, ChromeType

option = uc.ChromeOptions()

option.add_argument('--ignore-ssl-errors')
option.add_argument('--disable-blink-features=AutomationControlled')
option.add_argument('--headless=new') 
option.add_argument('--no-sandbox') 
option.add_argument("--window-size=1920x1080")
option.add_argument('--disable-dev-shm-usage')
option.add_argument("--disable-gpu")

# Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0
option.add_argument('user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15"')

br_ver = OperationSystemManager().get_browser_version_from_os(ChromeType.GOOGLE)
ver_main=int(br_ver.split('.')[0])

driver = uc.Chrome(options=option, driver_executable_path=r"D:\\Program Files\\chromedriver-win64\\chromedriver.exe", headless=True, version_main=ver_main)

driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
    "source": """
        Object.defineProperty(navigator, 'webdriver', {
        get: () => undefined
        })
    """
})

driver.get(site_url)
driver.implicitly_wait(10)
print(driver.page_source)

if the site has both mobile version and desktop version, the mobile version will be returned.

The possible root cause is in undetected_chromedriver/init.py, the user agent is set incorrectly by the function get_wrapped when called by _configure_headless (Line #1522)

        self.execute_cdp_cmd(
            "Network.setUserAgentOverride",
            {
                "userAgent": self.execute_script(
                    "return navigator.userAgent"
                ).replace("Headless", "")
            },
        )

A temporary fix is to re-set the user agent by calling execute_cdp_cmd before calling get():

        driver.execute_cdp_cmd("Network.setUserAgentOverride", {
            "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15"
        })