BrowserConfig

conatus.utils.browser.config ¶

Configuration class for the FullBrowser and the classes it calls.

The way this works is that we have a class that defines the configuration for the FullBrowser class. We also have a typed dictionary that defines the configuration for the FullBrowser class.

We have this class to simplify the handling of configuration values. We also use Pydantic, so that the user can pass a dictionary of configuration values and we can validate them.

{
    "headless": "bool",
    "context": {
        "initial_viewport": "(int, int)",
        "device_scale_factor": "int | float",
        "device_type": "str"
    },
    "page": {
        "hard_timeout": "int",
        "soft_timeout": "int",
        "time_delta": "float",
        "stability_threshold": "float"
    },
    "step": {
        "save_screenshot": "bool",
        "save_screenshot_b64": "bool",
        "save_html": "bool"
    }
}

Confused about the terminology used here? Check out the documentation for the FullBrowser class.

DEFAULT_HARD_TIMEOUT `module-attribute` ¶

DEFAULT_HARD_TIMEOUT = 20

The hard timeout for the page, as encoded in Playwright.

If the page takes longer than this time to load, we throw an error. Defaults to 20 seconds.

DEFAULT_SOFT_TIMEOUT `module-attribute` ¶

DEFAULT_SOFT_TIMEOUT = 10

The soft timeout for the page, which is a looser constraint that we manage.

If the page takes longer than this time to load, we treat it as if it has finished loading. Defaults to 10 seconds.

DEFAULT_NEW_PAGE_TIMEOUT `module-attribute` ¶

DEFAULT_NEW_PAGE_TIMEOUT = 1

The timeout for a new page.

This is the time we wait expecting a new page to load. After that time, new pages that are launched (say, by clicking a link) will not be waited for. Defaults to 1 second.

DEFAULT_TIME_DELTA `module-attribute` ¶

DEFAULT_TIME_DELTA = 0.5

The time delta at which we check for page stability.

This is basically the interval at which we check if the page is loaded. We can't do that every millisecond, because we need to run some expensive-ish DOM operations. Defaults to 0.5 seconds.

DEFAULT_STABILITY_THRESHOLD `module-attribute` ¶

DEFAULT_STABILITY_THRESHOLD = 0.99

The threshold for page stability.

We consider the page stable if the DOM hasn't changed by this amount of time. Defaults to 0.99 seconds.

StepConfig ¶

Bases: BaseModel

Default configuration for the Step class.

ATTRIBUTE	DESCRIPTION
`draw_boxes`	Whether to draw boxes around the nodes that can be clicked, is inputable, or can be interacted with. TYPE: `bool`
`take_screenshots`	Whether to take screenshots of the page. Defaults to `True`, but you can set it to `False` for performance gains. TYPE: `bool`
`save_screenshot`	Whether to save a screenshot of the page. Will not take effect if `take_screenshots` is `False`. TYPE: `bool`
`save_screenshot_b64`	Whether to save a base64 encoded screenshot of the page. Will not take effect if `take_screenshots` is `False`. TYPE: `bool`
`save_html`	Whether to save the HTML of the page. TYPE: `bool`

StepConfigTD ¶

Bases: TypedDict

Typed dictionary for the StepConfig class.

ATTRIBUTE	DESCRIPTION
`draw_boxes`	Whether to draw boxes around the nodes that can be clicked, inputable, or can be interacted with. TYPE: `bool`
`take_screenshots`	Whether to take screenshots of the page. Defaults to `True`, but you can set it to `False` for performance gains. TYPE: `bool`
`save_screenshot`	Whether to save a screenshot of the page. Will not take effect if `take_screenshots` is `False`. TYPE: `bool`
`save_screenshot_b64`	Whether to save a base64 encoded screenshot of the page. Will not take effect if `take_screenshots` is `False`. TYPE: `bool`
`save_html`	Whether to save the HTML of the page. TYPE: `bool`

PageConfig ¶

Bases: BaseModel

Default configuration for the Page class.

ATTRIBUTE	DESCRIPTION
`hard_timeout`	The hard timeout for the page, as encoded in Playwright. If the page takes longer than this time to load, we throw an error. Encoded in seconds. TYPE: `float`
`soft_timeout`	The soft timeout for the page, which is a looser constraint that we manage. If the page takes longer than this time to load, we treat it as if it has finished loading. Encoded in seconds. TYPE: `float`
`new_page_timeout`	The timeout for a new page. This is the time we wait expecting a new page to load. After that time, new pages that are launched (say, by clicking a link) will not be waited for. Encoded in seconds. TYPE: `float`
`time_delta`	The time delta at which we check for page stability. This is basically the interval at which we check if the page is loaded. We can't do that every millisecond, because we need to run some expensive-ish DOM operations. Encoded in seconds. TYPE: `float`
`stability_threshold`	The threshold for page stability. We consider the page stable if the DOM hasn't changed for this amount of time. TYPE: `float`

PageConfigTD ¶

Bases: TypedDict

Typed dictionary for the PageConfig class.

ATTRIBUTE	DESCRIPTION
`hard_timeout`	The hard timeout for the page, as encoded in Playwright. If the page takes longer than this time to load, we throw an error. Encoded in seconds. TYPE: `float`
`soft_timeout`	The soft timeout for the page, which is a looser constraint that we manage. If the page takes longer than this time to load, we treat it as if it has finished loading. Encoded in seconds. TYPE: `float`
`new_page_timeout`	The timeout for a new page. This is the time we wait expecting a new page to load. After that time, new pages that are launched (say, by clicking a link) will not be waited for. Encoded in seconds. TYPE: `float`
`time_delta`	The time delta at which we check for page stability. This is basically the interval at which we check if the page is loaded. We can't do that every millisecond, because we need to run some expensive-ish DOM operations. Encoded in seconds. TYPE: `float`
`stability_threshold`	The threshold for page stability. We consider the page stable if the DOM hasn't changed for this amount of time. TYPE: `float`

BrowserContextConfig ¶

Bases: BaseModel

Default configuration for the BrowserContext class.

initial_viewport `class-attribute` `instance-attribute` ¶

initial_viewport: tuple[int, int] = DEFAULT_VIEWPORT

The initial size of the viewport.

device_scale_factor `class-attribute` `instance-attribute` ¶

device_scale_factor: int | float = DEFAULT_SCALE_F

The device scale factor.

device_type `class-attribute` `instance-attribute` ¶

device_type: str = DEFAULT_DEVICE_TYPE

The device type.

BrowserContextConfigTD ¶

Bases: TypedDict

Typed dictionary for the BrowserContextConfig class.

ATTRIBUTE	DESCRIPTION
`initial_viewport`	The initial size of the viewport. TYPE: `tuple[int, int]`
`device_scale_factor`	The device scale factor. TYPE: `int \| float`
`device_type`	The device type. TYPE: `str`

FileWriterTD ¶

Bases: TypedDict

Typed dictionary for the FileWriter class.

ATTRIBUTE	DESCRIPTION
`base_folder`	The base folder to log run outputs to. TYPE: `Path \| str \| None`
`run_folder`	The folder to write the outputs of a specific run to. TYPE: `Path \| str \| None`
`log_folder`	The folder to write logs to. TYPE: `Path \| str \| None`
`out_folder`	The folder to write outputs to. TYPE: `Path \| str \| None`
`debug_folder`	The folder to write debug information to. TYPE: `Path \| str \| None`
`run_id`	The ID of the run. Defaults to a Unix UTC timestamp. TYPE: `str \| None`

BrowserConfigDict ¶

Bases: TypedDict

Typed dictionary for the BrowserConfig class.

ATTRIBUTE	DESCRIPTION
`headless`	Whether to run the browser in headless mode. TYPE: `bool`
`context`	The configuration of `BrowserContext`. TYPE: `BrowserContextConfigTD`
`page`	The configuration of `Page`. TYPE: `PageConfigTD`
`step`	The configuration of `Step`. TYPE: `StepConfigTD`
`writer`	The file writer that handles writing logs and outputs. TYPE: `FileWriterTD`
`run_id`	The ID of the run. Defaults to a Unix UTC timestamp. TYPE: `FileWriterTD`

BrowserConfig ¶

BrowserConfig(
    data_as_dict: BrowserConfigDict | None = None,
)

Bases: BaseModel

Default configuration for the FullBrowser class.

ATTRIBUTE	DESCRIPTION
`headless`	Whether to run the browser in headless mode. TYPE: `bool`
`context`	The configuration of `BrowserContext`. TYPE: `BrowserContextConfig`
`page`	The configuration of `Page`. TYPE: `PageConfig`
`step`	The configuration of `Step`. TYPE: `StepConfig`
`writer`	The file writer that handles writing logs and outputs. TYPE: `FileWriter`

Although you could use this class directly, it is intended to be used with a dictionary of configuration values.

from conatus.utils.browser.config import BrowserConfig

browser_config = BrowserConfig({
    "headless": True,
    "context": {
        "initial_viewport": (1100, 700),
        "device_scale_factor": 1.0,
        "device_type": "Desktop Chrome HiDPI"
    },
    "page": {
        "hard_timeout": 20,
        "soft_timeout": 10,
        "time_delta": 0.5,
        "stability_threshold": 0.99
    },
    "step": {
        "save_screenshot": False,
        "save_screenshot_b64": False,
        "save_html": False
    }
})

# You can access the values like this
assert browser_config.headless

PARAMETER	DESCRIPTION
`data_as_dict`	The configuration as a dictionary. Defaults to None. TYPE: `dict` DEFAULT: `None`

Source code in conatus/utils/browser/config.py

def __init__(
    self,
    data_as_dict: BrowserConfigDict | None = None,
) -> None:
    """Initialize the configuration.

    Although you could use this class directly, it is intended to be used
    with a dictionary of configuration values.

    ```python
    from conatus.utils.browser.config import BrowserConfig

    browser_config = BrowserConfig({
        "headless": True,
        "context": {
            "initial_viewport": (1100, 700),
            "device_scale_factor": 1.0,
            "device_type": "Desktop Chrome HiDPI"
        },
        "page": {
            "hard_timeout": 20,
            "soft_timeout": 10,
            "time_delta": 0.5,
            "stability_threshold": 0.99
        },
        "step": {
            "save_screenshot": False,
            "save_screenshot_b64": False,
            "save_html": False
        }
    })

    # You can access the values like this
    assert browser_config.headless
    ```

    Args:
        data_as_dict (dict, optional): The configuration as a dictionary.
            Defaults to None.
    """
    if data_as_dict is None:
        data_as_dict = {}
    super().__init__(**data_as_dict)

options: inherited_members: true members: - BrowserConfig - BrowserContextConfig - PageConfig - StepConfig

BrowserConfig

conatus.utils.browser.config ¶

DEFAULT_HARD_TIMEOUT module-attribute ¶

DEFAULT_SOFT_TIMEOUT module-attribute ¶

DEFAULT_NEW_PAGE_TIMEOUT module-attribute ¶

DEFAULT_TIME_DELTA module-attribute ¶

DEFAULT_STABILITY_THRESHOLD module-attribute ¶

StepConfig ¶

StepConfigTD ¶

PageConfig ¶

PageConfigTD ¶

BrowserContextConfig ¶

initial_viewport class-attribute instance-attribute ¶

device_scale_factor class-attribute instance-attribute ¶

device_type class-attribute instance-attribute ¶

BrowserContextConfigTD ¶

FileWriterTD ¶

BrowserConfigDict ¶

BrowserConfig ¶

DEFAULT_HARD_TIMEOUT `module-attribute` ¶

DEFAULT_SOFT_TIMEOUT `module-attribute` ¶

DEFAULT_NEW_PAGE_TIMEOUT `module-attribute` ¶

DEFAULT_TIME_DELTA `module-attribute` ¶

DEFAULT_STABILITY_THRESHOLD `module-attribute` ¶

initial_viewport `class-attribute` `instance-attribute` ¶

device_scale_factor `class-attribute` `instance-attribute` ¶

device_type `class-attribute` `instance-attribute` ¶