Skip to content

Scrappage

flowtask.components.ScrapPage

Scrapping a Web Page Using Selenium + ChromeDriver + BeautifulSoup.

    Example:

    ```yaml
    ScrapPage:
      url: https://www.consumeraffairs.com/insurance/assurant-phone-insurance.html?page=2#sort=recent
      rotate_ua: true
      use_selenium: true
      use_proxy: true
      paid_proxy: true
      as_mobile: false
      timeout: 60
      wait_until:
      - - class_name
        - js-rvw
      screenshot:
        filename: reviews_page.png
        directory: /home/ubuntu/symbits/ejemplo/screenshots/
      outputs:
      - scraped_content
      - screenshot_path
    ```

ScrapPage 

ScrapPage(loop=None, job=None, stat=None, **kwargs)

Bases: SeleniumService, HTTPService, FlowComponent

ScrapPage. Scrapping a Web Page using Selenium.

run async 

run()

Run the Scrapping Tool.

run_http async 

run_http()

Run the Scrapping Tool Using HTTPx.

run_selenium async 

run_selenium()

Run the Scrapping Tool Using Selenium.