Advanced Web Scraper Pro

A simple web scraping tool that extracts text from websites and YouTube videos, then summarizes it using AI.

Setup

1. Install Python
2. Clone this repository
3. Run pip install -r requirements.txt
4. Get a Google AI API key from https://makersuite.google.com/
5. Create a .env file and add: GEMINI_API_KEY=your_api_key_here

Usage

1. Run streamlit run app.py
2. Enter a URL in the web interface
3. Click "Start Scraping"

Features

• Web scraping
• YouTube transcript extraction
• AI summarization
• Basic text statistics

Troubleshooting

• Ensure all dependencies are installed
• Check your API key in the .env file

Contributing

Feel free to submit pull requests or open issues for improvements.