Metadata-Version: 2.1
Name: uglypy
Version: 0.0.36
Summary: A Python package for aggregating and processing RSS feeds with LLM-enhanced content rewriting.
Home-page: https://github.com/fabriziosalmi/UglyFeed
Author: Fabrizio Salmi
Author-email: fabrizio.salmi@gmail.com
License: AGPL-3.0
Project-URL: Bug Tracker, https://github.com/fabriziosalmi/UglyFeed/issues
Project-URL: Documentation, https://github.com/fabriziosalmi/UglyFeed/blob/main/docs
Project-URL: Source Code, https://github.com/fabriziosalmi/UglyFeed
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: beautifulsoup4 ==4.12.3
Requires-Dist: EbookLib ==0.18
Requires-Dist: feedgen ==1.0.0
Requires-Dist: feedparser ==6.0.11
Requires-Dist: jiwer ==3.0.4
Requires-Dist: joblib ==1.4.0
Requires-Dist: langdetect ==1.0.9
Requires-Dist: lexical-diversity ==0.1.1
Requires-Dist: nltk ==3.8.1
Requires-Dist: numpy ==1.24.4
Requires-Dist: openai ==1.30.5
Requires-Dist: pandas ==2.2.2
Requires-Dist: psutil ==5.9.8
Requires-Dist: PyYAML ==6.0.1
Requires-Dist: requests ==2.32.3
Requires-Dist: scikit-learn ==1.4.2
Requires-Dist: sentence-transformers ==2.7.0
Requires-Dist: spacy ==3.7.0
Requires-Dist: textblob ==0.18.0.post0
Requires-Dist: textstat ==0.7.3
Requires-Dist: tqdm ==4.66.3
Requires-Dist: typer ==0.9.0
Requires-Dist: streamlit
Requires-Dist: schedule

# <img src="https://raw.githubusercontent.com/fabriziosalmi/UglyFeed/main/docs/UglyFeed.png" alt="UglyFeed" height="32" width="32"> UglyFeed

UglyFeed is a **simple** application designed to **retrieve**, **aggregate**, **filter**, **rewrite**, **evaluate** and **serve** content (RSS feeds) written by a large language model. This repository provides the [code](https://github.com/fabriziosalmi/UglyFeed.git), the **[documentation](https://github.com/fabriziosalmi/UglyFeed/blob/main/docs/README.md)**, a [FAQ](https://github.com/fabriziosalmi/UglyFeed/blob/main/docs/faq.md) page and some optional scripts to evaluate the generated content.

![GitHub last commit](https://img.shields.io/github/last-commit/fabriziosalmi/UglyFeed) ![GitHub Issues or Pull Requests](https://img.shields.io/github/issues-raw/fabriziosalmi/UglyFeed) [![Pylint](https://github.com/fabriziosalmi/UglyFeed/actions/workflows/pylint.yml/badge.svg)](https://github.com/fabriziosalmi/UglyFeed/actions/workflows/pylint.yml) [![CodeQL](https://github.com/fabriziosalmi/UglyFeed/actions/workflows/github-code-scanning/codeql/badge.svg)](https://github.com/fabriziosalmi/UglyFeed/actions/workflows/github-code-scanning/codeql) ![Docker Pulls](https://img.shields.io/docker/pulls/fabriziosalmi/uglyfeed) ![PyPI - Downloads](https://img.shields.io/pypi/dm/uglypy?label=uglypy)


## Features

- 📡 Retrieve RSS feeds
- 🧮 Aggregate feeds items by similarity
- ✨ Rewrite content using LLM API
- 💾 Save rewritten feeds to JSON files
- 🔁 Convert JSON to valid RSS feed
- 🌐 Serve XML feed via HTTP server
- 🌎 Deploy XML feed to GitHub or GitLab
- 📈 Evaluate generated content
- 🖥️ Web UI based on Streamlit
- 📰 [RSS test feeds available](https://github.com/fabriziosalmi/uglyfeed-cdn)
- 🤖 Same codebase for all releases
- 🛑 Simple post-filter moderation 

## Get it now

- 💾 [Source code](https://github.com/fabriziosalmi/UglyFeed/releases/latest)
- 🐳 [Docker installable package](https://hub.docker.com/r/fabriziosalmi/uglyfeed)
- 🐍 [Pip installable package](https://pypi.org/project/uglypy/)
- ⚙️ [Github actions workflows](https://github.com/fabriziosalmi/UglyFeed/blob/main/docs/README.md#%EF%B8%8F-installation-and-automated-runs-github-actions)
  
## Quick start

### Prerequisites

- 🌎 Internet connection
- 🐳 Docker
- ✨ LLM API
- 📲 RSS reader
  
**Supported API and models**

- OpenAI API (`gpt-3.5-turbo`, `gpt-4`, `gpt-4o`)
- Ollama API (all models like `llama3`, `phi3`, `qwen2`)
- Groq API (`llama3-8b-8192`, `llama3-70b-8192`, `gemma-7b-it`, `mixtral-8x7b-32768`)
- Anthropic API (`claude-3-haiku-20240307`, `claude-3-sonnet-20240229`, `claude-3-opus-20240229`)

> You can use your own models by running a compatible OpenAI LLM server. You must change the OpenAI API url parameter.
 
### Running the Container

To start the UglyFeed app, use the following `docker run` command:

```bash
docker run -p 8001:8001 -p 8501:8501 -v /path/to/local/feeds.txt:/app/input/feeds.txt -v /path/to/local/config.yaml:/app/config.yaml fabriziosalmi/uglyfeed:latest
```

### Configure the application
In the **Configuration** page (or by manually editing the `config.yaml` file) you will find all configuration options. You must change at least the source feeds you want to aggregate, the LLM API and model to use to rewrite the aggregated feeds. You can then retrieve the final `uglyfeed.xml` feed in many ways: 

- local filesystem
- download from web UI
- HTTP server url
- HTTPS GitHub CDN url

> You can easily extend it to send it to cms, notification or messaging systems.

### Execute the application scripts
Execute all scripts in the **Run scripts** page easily by clicking on the button **Run `main.py`, `llm_processor.py`, `json2rss.py` sequentially**.
You can check for logs, errors and informational messages.

### Serve the final rewritten XML feed via HTTP
Once all scripts completed go to the **View and Serve XML** page where you can view and download the generated XML feed. If you start the HTTP server you can access to the XML url at `http://container_ip:8001/uglyfeed.xml`

### Deploy the final rewritten XML feed to GitHub/GitLab
Once all scripts completed go to the **Deploy** page where you can push the final rewritten XML file to the configured GitHub/GitLab repository, the public XML URL to use by RSS readers is returned for each enabled platform.

## Documentation

Please refer to the extended [documentation](https://github.com/fabriziosalmi/UglyFeed/blob/main/docs/README.md) to better understand how to get the best from this application.


## Use cases

The project can be easily customized to fit several use cases:

- **Smart Content Curation**: Create bespoke newsfeeds tailored to niche interests, blending articles from diverse sources into a captivating, engaging narrative.
- **Dynamic Blog Generation**: Automate blog post creation by rewriting and enhancing existing articles, optimizing them for readability and SEO.
- **Interactive Educational Tools**: Develop AI-driven study aids that summarize and rephrase academic papers or textbooks, making complex topics more accessible and fun.
- **Personalized Reading Experiences**: Craft custom reading lists that adapt to user preferences, offering fresh perspectives on favorite topics.
- **Brand Monitoring**: Aggregate and summarize brand mentions across the web, providing concise, actionable insights for marketing teams.
- **Multilingual Content Delivery**: Automatically translate and rewrite content from international sources, broadening the scope of accessible information.
- **Enhanced RSS Feeds**: Offer enriched RSS feeds that summarize, evaluate, and filter content, providing users with high-quality, relevant updates.
- **Creative Writing Assistance**: Assist writers by generating rewritten drafts of their work, helping overcome writer's block and sparking new ideas.
- **Content Repurposing**: Transform long-form content into shorter, more digestible formats like infographics, slideshows, and social media snippets.
- **Fake News Detection Datasets**: Generate datasets by rewriting news articles for use in training models to recognize and combat fake news.

## Contribution

Feel free to open issues or submit pull requests. Any contributions are welcome!


## Roadmap

> I started this project some weeks ago to experiment, get fun, learn and contribute to the open source community on my free time. I am so grateful to those who already made me empowering this pathway in a so short timeframe 🙏

Here some improvements I am still working on:

- overall code improvements and tests
- generate media from rewritten content
- here something i forgot 😅

## Disclaimer

> _It is crucial to acknowledge the potential misuse of AI language models by this tool. The use of adversarial prompts and models can easily lead to the creation of misleading content. This application should not be used with the intent to deceive or mislead others. Be a responsible user and prioritize ethical practices when utilizing language models and AI technologies._

## License

This project is licensed under the AGPL3 License.


