[Bug]:

[mkaesler44]

crawl4ai version

0.7.8

Expected Behavior

When performing full page screenshots with a scroll_delay, the scrolls between screenshots should respect the scroll_delay parameter and wait that amount of time before screenshotting

Current Behavior

Currently there is a hardcoded asyncio.sleep(0.01) between each of the screenshots in the take_screenshot_scroller() function, thus each screenshot only waits 0.01 seconds before scrolling for the next screenshot

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

macOS

Python version

3.12.0

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

[Ahmed-Tawfik94]

Root Cause Analysis

• Primary cause:
  The take_screenshot_scroller() method at line 1725 in crawl4ai/async_crawler_strategy.py contains a hardcoded await asyncio.sleep(0.01) instead of using the configurable scroll_delay parameter.

• Contributing factors:

  1. Missing parameter propagation: The take_screenshot() function is called at lines 1016-1018 and 1584-1586 without passing scroll_delay from the config
  2. Incomplete implementation: The scroll_delay parameter exists in CrawlerRunConfig (default: 0.2) and is properly used in _handle_full_page_scan() for regular crawling, but was not implemented for the screenshot scrolling logic
  3. Inconsistent behavior: Users expect scroll_delay to work consistently across all scrolling operations, but screenshots bypass this configuration

• Evidence:

  • Line 1725: await asyncio.sleep(0.01) # wait for render (hardcoded)
  • Lines 1016-1018: take_screenshot() called with only screenshot_height_threshold, missing scroll_delay
  • Lines 1584-1586: Second call site also missing scroll_delay parameter
  • Line 1279: scroll_delay: float = 0.2 defined in CrawlerRunConfig but not utilized for screenshots

Suggested Fixes (Backward-Compatible)

Fix 1: Pass scroll_delay to take_screenshot() (Lines 1016-1018)

# Before:
screenshot_data = await self.take_screenshot(
    page, screenshot_height_threshold=config.screenshot_height_threshold
)

# After:
screenshot_data = await self.take_screenshot(
    page,
    screenshot_height_threshold=config.screenshot_height_threshold,
    scroll_delay=config.scroll_delay
)

Fix 2: Pass scroll_delay to take_screenshot() (Lines 1584-1586)

# Before:
screenshot_data = await self.take_screenshot(
    page, screenshot_height_threshold=screenshot_height_threshold
)

# After:
screenshot_data = await self.take_screenshot(
    page,
    screenshot_height_threshold=screenshot_height_threshold,
    scroll_delay=getattr(config, 'scroll_delay', 0.2)
)

Fix 3: Use scroll_delay in take_screenshot_scroller() (Line 1725)

# Add at the beginning of take_screenshot_scroller():
scroll_delay = kwargs.get('scroll_delay', 0.2)

# Replace at line 1725:
# Before:
await asyncio.sleep(0.01)  # wait for render

# After:
await asyncio.sleep(scroll_delay)  # wait for render