Metadata-Version: 2.4
Name: selenium-crawler
Version: 0.1.0
Summary: Creates a simple Selenium Webdriver, optimized for web scraping.
Home-page: https://github.com/jo-hoe/selenium-crawler
Author: Jo-Hoe
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: selenium>=4.0.0
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Selenium Crawler

A simple, opinionated Selenium WebDriver setup optimized for web scraping with Chrome.
Designed for quick, lightweight data extraction, especially in Dockerized environments.

## Features

- Chrome WebDriver configuration for scraping
- Headless mode and image loading control
- Optimizations for Docker, Windows, Linux, and MacOS
- Utilities for element selection and waiting

## Requirements

- Google Chrome browser
- ChromeDriver (compatible with your Chrome version)

## Usage

Import and use the WebDriver creator and scraping tools in your scripts:

```python
from selenium_crawler.webdrivercreator import create_webdriver
from selenium_crawler.scraping_tools import get_element
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

driver = create_webdriver(headless=True)
try:
    driver.get('https://example.com')
    element = get_element(driver, (By.XPATH, '//h1'), timeout=10)
    print(element.text)
except TimeoutException:
    print('Element not found within the timeout period.')
finally:
    driver.quit()
```

## Testing

Run all tests with:

```sh
make test
```

Or manually:

```sh
python -m unittest discover -s tests -v
```

## Notes

- This project is not intended to bypass all bot detection mechanisms.
- Focuses on ease of use and quick setup for scraping tasks.
- For advanced anti-bot evasion, consider additional tools or services.
