mirror/ai-marketplace-monitor
1
0
Fork 0
mirror of https://github.com/BoPeng/ai-marketplace-monitor.git synced 2025-12-23 14:18:11 -05:00
Code Issues Packages Projects Releases Wiki Activity

Read the docs (#245)

Browse Source 
* move docs to readthedocs
This commit is contained in:
Bo
2025-08-08 10:34:47 -05:00
committed by GitHub
parent 07642d8fdb
commit 7c2be169fc
21 changed files with 3046 additions and 2009 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
.readthedocs.yml
View File

@@ -4,11 +4,19 @@ build:
os: ubuntu-22.04
tools:
python: "3.12"
jobs:
post_create_environment:
# Install uv for faster dependency resolution
- pip install uv
post_install:
# Install the package with documentation dependencies
- uv pip install -e .[docs]
formats: all
sphinx:
configuration: docs/conf.py
fail_on_warning: false
python:
install:

4
CHANGELOG.md
View File

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.9.7]
- Add support for telegram [PR 231](https://github.com/BoPeng/ai-marketplace-monitor/pull/231). thanks to @adawalli
## [0.9.6]
- Fix searching across regions.

905
README.md
View File

@@ -17,6 +17,8 @@
An intelligent tool that monitors Facebook Marketplace listings using AI to help you find the best deals. Get instant notifications when items matching your criteria are posted, with AI-powered analysis of each listing.
**📚 [Read the Full Documentation](https://ai-marketplace-monitor.readthedocs.io/)**
![Search In Action](docs/search_in_action.png)
Example notification from PushBullet:
@@ -29,40 +31,16 @@ https://facebook.com/marketplace/item/1234567890
AI: Great deal; A well-priced, well-maintained camera meets all search criteria, with extra battery and charger.
```
**Table of content:**
**Table of Contents:**
- [✨ Key Features](#-key-features)
- [Usage](#usage)
- [Before the prerequisites](#before-the-prerequisites)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Set up a notification method (optional)](#set-up-a-notification-method-optional)
- [Sign up with an AI service or build your own (optional)](#sign-up-with-an-ai-service-or-build-your-own-optional)
- [Configuration](#configuration)
- [Run the program](#run-the-program)
- [Updating search](#updating-search)
- [Cost of operations](#cost-of-operations)
- [Advanced features](#advanced-features)
- [Setting up email notification](#setting-up-email-notification)
- [Setting Up PushOver Notifications](#setting-up-pushover-notifications)
- [Setting Up Telegram Notifications](#setting-up-telegram-notifications)
- [Adjust prompt and notification level](#adjust-prompt-and-notification-level)
- [Advanced Keyword-based filters](#advanced-keyword-based-filters)
- [Searching multiple cities and regions](#searching-multiple-cities-and-regions)
- [Searching across regions with different currencies](#searching-across-regions-with-different-currencies)
- [Support for non-English languages](#support-for-non-english-languages)
- [Check individual listing](#check-individual-listing)
- [Multiple marketplaces](#multiple-marketplaces)
- [First and subsequent searches](#first-and-subsequent-searches)
- [Showing statistics](#showing-statistics)
- [Self-hosted Ollama Model](#self-hosted-ollama-model)
- [Cache Management](#cache-management)
- [Support for different layouts of facebook listings](#support-for-different-layouts-of-facebook-listings)
- [Searching Anonymously with a Proxy Server](#searching-anonymously-with-a-proxy-server)
- [Contributing](#contributing)
- [License](#license)
- [Support](#support)
- [Credits](#credits)
- [🚀 Quick Start](#-quick-start)
- [💡 Example Usage](#-example-usage)
- [📚 Documentation](#-documentation)
- [🤝 Contributing](#-contributing)
- [📜 License](#-license)
- [💬 Support](#-support)
- [🙏 Credits](#-credits)
## ✨ Key Features
@@ -94,801 +72,109 @@ AI: Great deal; A well-priced, well-maintained camera meets all search criteria,
- Customizable search radius
- Flexible seller location filtering
## Usage
## 🚀 Quick Start
### Before the prerequisites
_AI Marketplace Monitor_ is a tool designed to assist users in monitoring online marketplaces, with a focus on leveraging AI technologies to filter out spam and irrelevant listings. **This project was developed for personal, hobbyist use only**.
However, it is crucial to understand that **Facebook's [EULA](https://www.facebook.com/terms/)** explicitly prohibits the use of automated tools to collect or access data without prior authorization:
> You may not access or collect data from our Products using automated means (without our prior permission) or attempt to access data you do not have permission to access, regardless of whether such automated access or collection is undertaken while logged-in to a Facebook account.
By using _AI Marketplace Monitor_, you acknowledge and agree that **you are solely responsible for ensuring compliance with Facebook’s (Meta’s) terms of service, as well as any applicable laws and regulations**. If you intend to use this tool — particularly for commercial or for-profit purposes — you **must** obtain explicit permission from Meta (and any other marketplaces that this tool may support in the future) before proceeding.
Unauthorized use of this tool may result in account suspension, legal consequences, or other penalties. **The developers and contributors of _AI Marketplace Monitor_ disclaim any liability for misuse, violations of platform policies, or any resulting consequences**. Use this tool at your own risk and ensure compliance with relevant terms and regulations before deployment.
### Prerequisites
- Python 3.x installed.
- Internet connection
> **⚠️ Legal Notice**: Facebook's EULA prohibits automated data collection without authorization. This tool was developed for personal, hobbyist use only. You are solely responsible for ensuring compliance with platform terms and applicable laws.
### Installation
Install the program by
```sh
```bash
pip install ai-marketplace-monitor
```
Install a browser for Playwright using the command:
```sh
playwright install
```
For more detailed instructions, please refer to
### Basic Configuration
- [linux (pipx)](INSTALL.linux.md)
### Set up a notification method (optional)
If you would like to receive notification from your phone
- Sign up for [PushBullet](https://www.pushbullet.com/), [PushOver](https://pushover.net/), [Ntfy](https://ntfy.sh/), or use [Telegram](https://telegram.org/)
- Install the app on your phone
- Go to the respective website and obtain necessary token(s), or for Telegram, create a bot using @BotFather
If you would like to receive email notification, obtain relevant SMTP settings from your email provider. See [Setting up email notification](#setting-up-email-notification) for details.
### Sign up with an AI service or build your own (optional)
You can sign up for an AI service (e.g. [OpenAI](https://openai.com/) and [DeepSeek](https://www.deepseek.com/)) by
- Sign up for an account
- Go to the API keys section of your profile, generate a new API key, and copy it
You can also connect to any other AI service that provides an OpenAI compatible API, or host your own large language model using Ollama (see [Self-hosted Ollama Model](#self-hosted-ollama-model) for details.)
### Configuration
One or more configuration file in [TOML format](https://toml.io/en/) is needed. The following example ([`minimal_config.toml`](docs/minimal_config.toml)) shows the absolute minimal number of options, namely which city you are searching in, what item you are searching for, and how you get notified with matching listings.
Create `~/.ai-marketplace-monitor/config.toml`:
```toml
[marketplace.facebook]
search_city = 'houston'
[item.name]
search_phrases = 'Go Pro Hero 11'
[user.user1]
pushbullet_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
```
The configuration file needs to be put as `$HOME/.ai-marketplace-monitor/config.toml`, or be specified via option `--config`.
A more realistic example using openAI would be
```toml
[ai.openai]
api_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
[marketplace.facebook]
search_city = 'houston'
username = 'your@email.com'
seller_locations = [
"sugar land",
"stafford",
"missouri city",
"pearland"
]
[item.name]
search_phrases = 'Go Pro Hero 11'
description = '''A new or used Go Pro version 11, 12 or 13 in
good condition. No other brand of camera is acceptable.
Please exclude sellers who offer shipping or asks to
purchase the item from his website.'''
keywords = "('Go Pro' OR gopro) AND (11 OR 12 OR 13)"
min_price = 100
max_price = 200
[item.name2]
search_phrases = 'something rare'
description = '''A rare item that has to be searched nationwide and be shipped.
listings from any location are acceptable.'''
search_region = 'usa'
delivery_method = 'shipping'
seller_locations = []
[user.user1]
email = 'you@gmail.com'
smtp_password = 'xxxxxxxxxxxxxxxx'
```
For a complete list of options, please see the [configuration documentation](docs/README.md).
### Run the program
```sh
ai-marketplace-monitor
```
or use option `--config` for a non-standard configuration file.
Use `Ctrl-C` to terminate the program.
**NOTE**
1. You need to keep the terminal running to allow the program to run indefinitely.
2. You will see a browser firing up. **You may need to manually enter username and/or password (if unspecified in config file), and answer any prompt (e.g. CAPTCHA) to login**. You may want to click "OK" to save the password, etc.
3. If you fail to login to facebook, _AI Marketplace Monitor_ will continue to operate. However, Facebook will not be able to provide results related to your user profile and will display a login screen over all search pages.
### Updating search
It is recommended that you **check the log messages and make sure that it includes and excludes listings as expected**. Modify the configuration file to update search criteria if needed. The program will detect changes and restart the search automatically.
### Cost of operations
1. **Licensing Costs**: None.
2. **External Service Costs**: Usage-dependent costs for notification services (e.g., PushBullet, SMTP) and AI platforms (e.g., OpenAI, DeepSeek).
3. **Infrastructure Costs**: Requires a PC, server, or cloud hosting (e.g., AWS t3.micro at ~$10/month) for 24/7 operation.
4. **Maintenance and Support**: Open-source support via GitHub; Active subscribers to our Pro or Business Plans get priority email support.
## Advanced features
### Setting up email notification
To send email notifications, you need to specify recipient email addresses in the `email` of a `user` or a notification setting. You can configure multiple users with individual or multiple email addresses like this:
```toml
[user.user1]
email = 'user1@gmail.com'
[user.user2]
email = ['user2@gmail.com', 'user2@outlook.com']
```
An SMTP server is required for sending emails, for which you will need to know `smtp_server`, `smtp_port`, `smtp_username` and `smtp_password`. Generally speaking, you will need to create a notification section with the information obtained from your email service provider.
```toml
[notification.myprovider]
smtp_username = 'username@EMAIL.COM' # default to email
smtp_server = 'smtp.EMAIL.COM' # default to smtp.EMAIL.COM
smtp_port = 587 # default for most providers
smtp_password = 'mypassword'
```
`ai-marketplace-monitor` will try to use `email` if `smtp_username` is unspecified, and determine `smtp_username` and `smtp_server` automatically from the sender email address. For example, your Gmail setup could be as simple as:
```toml
[notification.gmail]
smtp_password = 'abcdefghijklmnop'
```
You can specify `smtp_password` directly in the `user` section if you are not sharing the `notification` setting with other users.
```toml
[user.me]
email = 'myemail@gmail.com'
smtp_password = 'abcdefghijklmnop'
```
**Note:**
- **Gmail Users**; Your will need to create a separate app password for your Google account as `smtp_password`.
- **Commercial Users**: If you are a subscriber to our Pro or Business Plans, detailed instructions on configuring the SMTP service we provide will be sent to you via email.
### Setting Up PushOver Notifications
To enable PushOver notifications, follow these steps:
1. **Install the PushOver app** on your mobile device.
2. **Create a PushOver account** at [pushover.net](https://pushover.net). After registration, you will find your **User Key** labeled as `Your User Key` — this is your `pushover_user_key`.
3. **Create a new application** (you can name it `AIMarketplaceMonitor`). After creation, you will receive an **API Token/Key**, referred to as `pushover_api_token`.
Once you have both the user key and API token, add them to your configuration file using one of the following formats:
**Option 1: Embed directly under your user profile**
```toml
[user.me]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
```
**Option 2: Use a dedicated notification section**
```toml
[notification.pushover]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
[user.me]
notify_with = 'pushover'
```
By default, notifications include the **title**, **price**, **location**, **description**, and **AI-generated comments** (if enabled). To exclude or limit the length of the **listing description**, you can add the `with_description` option to your config.
You can set `with_description` to:
- `True` — to include the **full description**.
- `False` — to exclude the description (default behavior).
- A **number** — to include only the **first N characters** of the description.
For example:
```toml
[user.me]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
with_description = 100
```
This will include up to the first 100 characters of each listing's description in your notifications.
### Setting Up Telegram Notifications
To enable Telegram notifications, you'll need to create a Telegram bot and obtain a chat ID. Follow these steps:
#### Step 1: Create a Telegram Bot
1. **Open Telegram** and search for `@BotFather` (the official bot for creating other bots).
2. **Start a conversation** with BotFather by clicking "Start" or sending `/start`.
3. **Create a new bot** by sending the command `/newbot`.
4. **Choose a bot name** when prompted (e.g., "AI Marketplace Monitor").
5. **Choose a bot username** that ends with "bot" (e.g., "my_marketplace_monitor_bot").
6. **Save your bot token** - BotFather will provide a token that looks like `123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11`. **Keep this token secure and never share it publicly.**
#### Step 2: Get Your Chat ID
You need to find your chat ID to receive messages. Here are two methods:
**Method 1: Using @userinfobot**
1. Search for `@userinfobot` in Telegram and start a conversation.
2. Send any message to the bot.
3. The bot will reply with your user information, including your **Chat ID** (a number like `123456789`).
**Method 2: Using the Telegram Bot API**
1. Start a conversation with your newly created bot (search for its username).
2. Send any message to your bot (e.g., "Hello").
3. Open this URL in your browser, replacing `YOUR_BOT_TOKEN` with your actual token:
```
https://api.telegram.org/botYOUR_BOT_TOKEN/getUpdates
```
4. Look for the `"chat":{"id":` field in the response - this number is your chat ID.
#### Step 3: Configure Your Settings
Add your Telegram credentials to your configuration file using one of these formats:
**Option 1: Direct configuration under user profile**
```toml
[user.me]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '123456789'
```
**Option 2: Using a dedicated notification section**
```toml
[notification.telegram]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '123456789'
[user.me]
notify_with = 'telegram'
```
**Option 3: Using environment variables for security**
```toml
[user.me]
telegram_token = '${TELEGRAM_BOT_TOKEN}'
telegram_chat_id = '${TELEGRAM_CHAT_ID}'
```
Then set the environment variables:
```bash
export TELEGRAM_BOT_TOKEN="123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11"
export TELEGRAM_CHAT_ID="123456789"
```
#### Step 4: Test Your Configuration
You can test your setup by running the monitor. If configured correctly, you should receive Telegram notifications when matching listings are found.
#### Troubleshooting Common Issues
**401 Unauthorized Error**
- **Cause**: Invalid or incorrect bot token
- **Solution**:
1. Verify your bot token is correct (it should look like `123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11`)
2. Make sure there are no extra spaces or characters
3. Create a new bot with @BotFather if the token is lost
**403 Forbidden Error**
- **Cause**: Bot doesn't have permission to send messages to the chat
- **Solution**:
1. Start a conversation with your bot by searching for its username in Telegram
2. Send at least one message to the bot (e.g., "/start" or "Hello")
3. Verify the chat ID is correct
**400 Bad Request Error**
- **Cause**: Invalid chat ID format or the chat doesn't exist
- **Solution**:
1. Double-check your chat ID is a number (positive for users, negative for groups)
2. For group chats, make sure the bot is added to the group
3. Use the getUpdates method to verify your chat ID
**Bot Not Responding**
- **Cause**: Network issues or Telegram API problems
- **Solution**:
1. Check your internet connection
2. Test with a simple curl command:
```bash
curl "https://api.telegram.org/bot<YOUR_BOT_TOKEN>/getMe"
```
3. Wait a few minutes and try again
**Configuration Examples**
Here are complete working configuration examples:
**Basic Telegram setup:**
```toml
[ai.openai]
api_key = 'your-openai-key'
[marketplace.facebook]
search_city = 'houston'
search_city = 'houston' # Replace with your city
[item.gopro]
search_phrases = 'GoPro Hero'
search_phrases = 'Go Pro Hero 11'
min_price = 100
max_price = 300
[user.me]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '123456789'
pushbullet_token = 'your_token_here' # Get from pushbullet.com
```
**Advanced setup with multiple notification methods:**
### Run the Monitor
```bash
ai-marketplace-monitor
```
The program will open a browser, search Facebook Marketplace, and notify you of matching items.
## 💡 Example Usage
**Find GoPro cameras under $300:**
```toml
[notification.telegram]
telegram_token = '${TELEGRAM_BOT_TOKEN}'
telegram_chat_id = '${TELEGRAM_CHAT_ID}'
[notification.email]
smtp_password = 'your-email-password'
[user.me]
email = 'your@email.com'
notify_with = ['telegram', 'email']
[item.gopro]
search_phrases = 'Go Pro Hero'
keywords = "('Go Pro' OR gopro) AND (11 OR 12 OR 13)"
min_price = 100
max_price = 300
```
**Group chat notifications:**
**Search nationwide with shipping:**
```toml
[user.family]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '-987654321' # Note: negative ID for groups
```
**Note on Security**:
- Never commit your bot token to version control
- Use environment variables for sensitive information
- Keep your bot token private - anyone with access can send messages as your bot
### Adjust prompt and notification level
_ai-marketplace-monitor_ asks AI services to evaluate listings against the criteria that you specify with prompts in four parts:
Part 1: buyer intent
```
A user wants to buy a ... with search phrase ... description ..., price range ...,
with keywords .... and exclude ...
```
Part 2: listing details
```
The user found a listing titled ... priced at ..., located ... posted at ...
with description ...
```
Part 3: instruction to AI
```
Evaluate how well this listing matches the user's criteria. Assess the description,
MSRP, model year, condition, and seller's credibility.
```
Part 4: rating instructions
```
Rate from 1 to 5 based on the following:
1 - No match: Missing key details, wrong category/brand, or suspicious activity (e.g., external links).
2 - Potential match: Lacks essential info (e.g., condition, brand, or model); needs clarification.
3 - Poor match: Some mismatches or missing details; acceptable but not ideal.
4 - Good match: Mostly meets criteria with clear, relevant details.
5 - Great deal: Fully matches criteria, with excellent condition or price.
Conclude with:
"Rating [1-5]: [summary]"
where [1-5] is the rating and [summary] is a brief recommendation (max 30 words)."
```
Depending on your specific needs, you can replace part 3 and part 4 of the prompt with options `prompt` and `rating_prompt`, and add an extra prompt before rating prompt with option `extra_prompt`. These options can be specified at the `marketplace` and `item` levels, with the latter overriding the former.
For example, you can add
```toml
[marketplace.facebook]
extra_prompt = """Exclude any listing that recommend visiting an external website \
for purchase."""
```
to describe suspicious listings in a marketplace, and
```toml
[item.ipadpro]
prompt = """Find market value for listing on market places like Ebay \
or Facebook marketplace and compare the price of the listing, considering \
the description, selling price, model year, condition, and seller's \
credibility. Evaluate how well this listing matches the user's criteria.
"""
```
With these settings, the part 3 of the prompt for item `ipadpro` will be replaced
with `prompt` for `item.ipadpro` and the `extra_prompt` from `marketplace.facebook`.
When AI services are used, the program by default notifies you of all listing with a rating of 3 or higher. You can change this behavior by setting for example
```toml
rating = 4
```
to see only listings that match your criteria well. Note that all listings after non-AI-based filtering will be returned if no AI service is specified or non-functional.
### Advanced Keyword-based filters
Options `keywords` and `antikeywords` are used to filter listings according to specified keywords. In the simplest form, these options support a single string. For example,
```toml
keywords = 'drone'
antikeywords = 'Parrot'
```
will select all listings with `drone` in title or description, and `Parrot` not in title or description. You can use multiple keywords and operators `AND`, `OR`, and `NOT` in the parameter. For example
```toml
keywords = 'DJI AND drone'
```
looks for listings with both `DJI` and `drone` in title or description.
If you have multiple keywords specified in a list, they are by default joint by `OR`. That is to say,
```toml
keywords = ['drone', 'DJI', 'Orqa']
antikeywords = ['Parrot', 'Autel']
```
is equivalent to
```toml
keywords = 'drone OR DJI OR Orqa'
antikeywords = 'Parrot OR Autel'
```
which means selecting listings that contains `drone` or `DJI` or `Orga` in title or description, but exclude those listings with `Parrot` or `Autel` in title or description.
These criteria will however, not exclude listings for a `DJI Camera`. If you would like to make sure that `drone` is selected, you can use
```toml
keywords = 'drone AND (DJI OR Orqa)'
antikeywords = 'Parrot OR Autel'
```
If you have special characters and spaces in your keywords, you will need to quote them, such as
```toml
keywords = '("Go Pro" OR gopro) AND HERO'
```
**NOTE**:
1. A list of logical operations are allowed, and they are assumed to be joint by `OR`. For example, `['gopro AND (11 or 12)', 'DJI AND OSMO']` searches for either a gopro version 11 or 12, or a DJI COMO camera.
2. You can construct very complex logical operations using `AND`, `OR` and `NOT`, but it is usually recommended to use simple keyword-based filtering and let AI handle more subtle selection criteria.
### Searching multiple cities and regions
`search_city` is the name, sometimes numbers, used by Facebook marketplace to represent a city. To get the value of `search_city` for your region, visit facebook marketplace, perform a search, and the city should be the name after `marketplace` (e.g. `XXXXX` in a URL like `https://www.facebook.com/marketplace/XXXXX/search?query=YYYY`).
Multiple searches will be performed if multiple cities are provided to option `search_city`. You can also specify `seller_locations` to limit the location of sellers. These locations are names of cities as displayed on the listing pages.
```toml
[item.name]
search_city = ['city1', 'city2']
seller_locations = ['city1', 'city2', 'city3', 'city4']
```
You can also increase the radius of search using
```toml
[item.name]
search_city = ['city1', 'city2']
radius = 50
```
However, if you would like to search for a larger region (e.g. the USA), it is much easier to define `region`s with a list of `search_city` and large `radius`.
_ai-marketplace-monitor_ defines the following regions in its system
[config.toml](https://github.com/BoPeng/ai-marketplace-monitor/blob/main/src/ai_marketplace_monitor/config.toml):
- `usa` for USA (without AK or HI), with currency `USD`
- `usa_full` for USA, with currency `USD`
- `can` for Canada, with currency `CAD`
- `mex` for Mexico, with currency `MXN`
- `bra` for Brazil, with currency `BRL`
- `arg` for Argentina, with currency `ARS`
- `aus` for Australia, with currency `AUD`
- `aus_miles` for Australia using 500 miles radius, with currency `AUD`
- `nzl` for New Zealand, with currency `NZD`
- `ind` for India, with currency `INR`
- `gbr` for United Kingdom, with currency `GBP`
- `fra` for France, with currency `EUR`
- `spa` for Spain, with currency `EUR`
Now, if you would like to search an item across the US, you can
```toml
[item.name]
[item.rare_item]
search_phrases = 'vintage collectible'
search_region = 'usa'
delivery_method = 'shipping'
seller_locations = []
delivery_method = 'shipping'
```
Under the hood, _ai-marketplace-monitor_ will simply replace `search_region` with corresponding pre-defined `search_city`, `radius`, and `currency`. Note that `seller_locations` does not make sense and need to be set to empty for region-based search, and it makes sense to limit the search to listings that offer shipping.
### Searching across regions with different currencies
_AI Marketplace Monitor_ does not enforce any specific currency format for price filters. It assumes that the `min_price` and `max_price` values are provided in the currency commonly used in the specified `search_city`. For example, in the configurations below:
**AI-powered filtering:**
```toml
[item.item1]
min_price = 100
search_city = 'newyork' # for demonstration only, city name for newyork might differ
[ai.openai]
api_key = 'your_openai_key'
[item.camera]
description = '''High-quality DSLR camera in good condition.
Exclude listings with water damage or missing parts.'''
rating = 4 # Only notify for 4+ star AI ratings
```
```toml
[item.item1]
min_price = 100
search_city = 'paris' # for demonstration only, city name for paris might differ
```
The `min_price` is interpreted as 100 `USD` for New York and 100 `EUR` for Paris, based on the typical local currency of each city.
If you perform a search across cities that use different currencies, you can explicitly define the currencies using the `currency` option:
```toml
[item.item1]
min_price = '100 USD'
search_city = ['paris', 'newyork']
currency = ['EUR', 'USD']
```
In this example, the system will perform two searches and convert the `min_price` of `100` `USD` into the equivalent amount in `EUR` when searching `item1` around Paris, using historical exchange rates provided by the [Currency Converter](https://pypi.org/project/CurrencyConverter/) package.
All pre-defined regions has a defined `currency` (see [Searching multiple cities and regions](#searching-multiple-cities-and-regions) for details). If you would like to search across regions with different currencies, you can
```toml
[item.item1]
min_price = '100 EUR'
search_region = ['fra', 'gbr']
```
and _AI Marketplace Monitor_ will automatically convert `100 EUR` to `GBP` when searching United Kingdom.
Note:
1. The following currency codes are supported: `USD`, `JPY`, `BGN`, `CYP`, `EUR`, `CZK`, `DKK`, `EEK`, `GBP`, `HUF`, `LTL`, `LVL`, `MTL`, `PLN`, `ROL`, `RON`, `SEK`, `SIT`, `SKK`, `CHF`, `ISK`, `NOK`, `HRK`, `RUB`, `TRL`, `TRY`, `AUD`, `BRL`, `CAD`, `CNY`, `HKD`, `IDR`, `ILS`, `INR`, `KRW`, `MXN`, `MYR`, `NZD`, `PHP`, `SGD`, `THB`, `ZAR`, and `ARS`.
Note: `ARS` (Argentine Peso) is included for completeness, but conversion support is not currently available.
2. Currency conversion only occurs if:
- `currency` values are explicitly defined.
- The currencies differ between cities or differ from the currency used in `min_price` / `max_price`.
3. Conversion rates are intended for basic filtering and may not reflect real-time market values. In some cases, converted `min_price` and `max_price` values may round down to zero (e.g. converting `100 JPY` to `USD`).
### Support for non-English languages
_AI Marketplace Monitor_ relies on specific keywords from webpages to extract relevant information. For example, it looks for words following `Condition` to determine the condition of an item. If your account is set to another language, _AI Marketplace Monitor_ will be unable to extract the relevant information. That is to say, if you see rampant error messages like
```
Failed to get details of listing https://www.facebook.com/marketplace/item/12121212121212121212
The listing might be missing key information (e.g. seller) or not in English.
Please add option language to your marketplace configuration is the latter is the case.
See https://github.com/BoPeng/ai-marketplace-monitor?tab=readme-ov-file#support-for-non-english-languages for details.
```
you will need to check `Setting -> Language` settings of your facebook account,
and let _AI Marketplace Monitor_ use the same language.
Currently, _AI Marketplace Monitor_ supports the following languages
- `es`: Spanish
- `zh`: Chinese
If your language is not defined, you will need to define your own [`translation` section](docs/README.md#translators) in your configuration file, following a format used by existing translators defined in [config.toml](https://github.com/BoPeng/ai-marketplace-monitor/blob/main/src/ai_marketplace_monitor/config.toml). This can be done by
1. Add a section to your configuration file, by copying one example from the system translators, for example,
```toml
[translator.LAN]
locale = "Your REGION"
"About this vehicle" = "Descripción del vendedor"
"Seller's description" = "Información sobre este vehículo"
"Collection of Marketplace items" = "Colección de artículos de Marketplace"
"Condition" = "Estado"
"Details" = "Detalles"
"Location is approximate" = "La ubicación es aproximada"
"Description" = "Descripción"
```
2. Find example listings (from for example [here](https://github.com/BoPeng/ai-marketplace-monitor/issues/29#issuecomment-2632057196)), locate the relevant words, and update the section. You can switch between different langauges (Facebook -> Settings -> Language) and see the location of the English version.
3. After you have completed the translation, add `language="LAN"` to the `marketplace` section as follows:
```toml
[translation.LAN]
"Condition" = "Condition in your LAN"
"Details" = "Details in your LAN"
...
```
in your configuration file, then add `language="LAN"` to the `marketplace` section as follows:
```toml
[marketplace.facebook]
language = "LAN"
```
It would be very helpful for other users of _AI Marketplace Monitor_ if you could contribute your dictionary to this project by creating a pull request or simply creating a ticket with your translations.
### Check individual listing
If you ever wonder why a listing was excluded, or just want to check a listing against your configuration, you can get the URL (or the item ID) of the listing, and run
```sh
ai-marketplace-monitor --check your-url
```
If you have multiple items specified in your config file, _ai-marketplace-monitor_ will check the product against the configuration of all of them. If you know the _name_ of the item in your config file, you can let the program only check the configuration of this particular item.
```sh
ai-marketplace-monitor --check your-url --for item_name
```
Option `--check` will load the details of the item from the cache if it was previously examined. Otherwise a browser will be started to retrieve the page.
Another way to check individual IDs is to enter interactive mode when the _ai-marketplace-monitor_ is running. If you press `Esc`, then confirm with `c` when prompted, you can enter the `URL` and `item_name` interactively and check the URL. Enter `exit` to exit the interactive session after you are done. However, using this method requires OS to allow the program to monitor your keyboard. It would not work on a terminal accessed through SSH, and you have to allow the terminal that you use to run _ai-marketplace-monitor_ to monitor keyboard from the _Privacy and Security_ settings on MacOS.
### Multiple marketplaces
Although facebook is currently the only supported marketplace, you can create multiple marketplaces such as`marketplace.city1` and `marketplace.city2` with different options such as `search_city`, `search_region`, `seller_locations`, and `notify`. You will need to add options like `marketplace='city1'` in the items section to link these items to the right marketplace.
For example
```toml
[marketplace.facebook]
search_city = 'houston'
seller_locations = ['houston', 'sugarland']
[marketplace.nationwide]
search_region = 'usa'
seller_location = []
delivery_method = 'shipping'
[item.default_item]
search_phrases = 'local item for default market "facebook"'
[item.rare_item1]
marketplace = 'nationwide'
search_phrases = 'rare item1'
[item.rare_item2]
marketplace = 'nationwide'
search_phrases = 'rare item2'
```
If no `marketplace` is defined for an item, it will use the first defined marketplace, which is `houston` in this example.
### First and subsequent searches
A list of two values can be specified for options `rating`, `availability`, `date_listed`, and `delivery_method`, with the first one used for the first search, and second one used for the rest of searches. This allows the use of different search strategies for first and subsequent searches. For example, an initial more lenient search for all listings followed by searches for only new listings can be specified as
```
rating = [2, 4]
availability = ["all", "in"]
date_listed = ["all", "last 24 hours"]
```
### Showing statistics
_ai-marketplace-monitor_ shows statistics such as the number of pages searched, number of listings examined and excluded, number of matching lists found and number of users notified when you exit the program. If you would like to see the statistics during monitoring, press `Esc` and wait till the current search to end.
Counters are persistent across program runs. If you would like to reset the counters, use
```
ai-marketplace-monitor --clear-cache counters
```
### Self-hosted Ollama Model
If you have access to a decent machine and prefer not to pay for AI services from OpenAI or other vendors. You can opt to install Ollama locally and access it using the `provider = "ollama"`. If you have ollama on your local host, you can use
```
[ai.ollama]
base_url = "http://localhost:11434/v1"
model = "deepseek-r1:14b"
timeout = 120
```
Note that
1. Depending on your hardware configuration, you can choose any of the models listed [here](https://ollama.com/search). The default model is `deepseek-r1:14b` becaue it appears to work better than `llama-3.1:8b`.
2. You need to `pull` the model before you can use it.
### Cache Management
_ai-marketplace-monitor_ caches listing details, ai inquiries, and user notifications to avoid repeated queries to marketplaces, AI services, and repeated notification. If for any reason you would like to clear the cache, you can use commands such as
```
ai-marketplace-monitor --clear-cache listing-details
```
to clear the cache. The following cache types are supported
- `listing-details`
- `ai-inquiries`
- `user-notification`
- `counters`
`--clear-cache all` is also possible but not recommended.
### Support for different layouts of facebook listings
Facebook marketplace supports a wide variety of products and use different layouts for them. _ai_marketplace_monitor_ can extract description from common listings such as household items and automobiles, but you may encounter items that this program cannot handle.
Although I certainly do not have the bandwidth to support all possible layouts, I have listed detailed steps on how to debug and resolve the issue on [issue 29](https://github.com/BoPeng/ai-marketplace-monitor/issues/29).
### Searching Anonymously with a Proxy Server
You can search Facebook Marketplace anonymously by disabling login,
- Do not provide a `username` or `password` in the `facebook` section
- (optional) Set `login_wait_time = 0` to stop waiting for login
- (optional) Use the `--headless` command line option to run `ai-marketplace-monitor` without a browser window.
If you would like to use a proxy server, you can
- Sign up for a VPN or proxy service.
- Configure the proxy settings in the `monitor` section of your configuration file as follows
```toml
[monitor]
proxy_server = '${PROXY_SERVER}'
proxy_username = '${PROXY_USERNAME}'
proxy_password = '${PROXY_PASSWORD}'
```
Replace `${PROXY_SERVER}`, `${PROXY_USERNAME}`, and `${PROXY_PASSWORD}` with your proxy service details, or setting the corresponding environment variables.
## Contributing
## 📚 Documentation
For detailed information on setup and advanced features, see the comprehensive documentation:
- **[📖 Full Documentation](https://ai-marketplace-monitor.readthedocs.io/)** - Complete guide and reference
- **[🚀 Quick Start Guide](https://ai-marketplace-monitor.readthedocs.io/en/latest/quickstart.html)** - Get up and running in 10 minutes
- **[⚙️ Configuration](https://ai-marketplace-monitor.readthedocs.io/en/latest/configuration.html)** - Complete configuration reference
- **[🔧 Advanced Features](https://ai-marketplace-monitor.readthedocs.io/en/latest/advanced-features.html)** - Notifications, AI prompts, multi-location search
- **[📱 Usage Guide](https://ai-marketplace-monitor.readthedocs.io/en/latest/usage.html)** - Command-line options and tips
- **[🔍 Features Overview](https://ai-marketplace-monitor.readthedocs.io/en/latest/features.html)** - Complete feature list
### Key Topics Covered in Documentation
**Notification Setup:**
- Email (SMTP), PushBullet, PushOver, Telegram, Ntfy
- Multi-user configurations
- HTML email templates
**AI Integration:**
- OpenAI, DeepSeek, Ollama setup
- Custom prompt configuration
- Rating thresholds and filtering
**Advanced Search:**
- Multi-city and region search
- Currency conversion
- Keyword filtering with Boolean logic
- Proxy/anonymous searching
**Configuration:**
- TOML file structure
- Environment variables
- Multiple marketplace support
- Language/translation support
## 🤝 Contributing
Contributions are welcome! Here are some ways you can contribute:
@@ -901,32 +187,25 @@ Contributions are welcome! Here are some ways you can contribute:
- 🤖 Add support for new AI providers
- 📱 Add new notification methods
Please read our [Contributing Guidelines](CONTRIBUTING.md) before submitting a Pull Request.
Please read our [Contributing Guidelines](https://ai-marketplace-monitor.readthedocs.io/en/latest/contributing.html) before submitting a Pull Request.
## License
## 📜 License
This project is licensed under the **Affero General Public License (AGPL)**. For the full terms and conditions, please refer to the official [GNU AGPL v3](https://www.gnu.org/licenses/agpl-3.0.en.html).
## Support
## 💬 Support
We provide multiple ways to access support and contribute to AI Marketplace Monitor:
- 📖 [Documentation](https://github.com/BoPeng/ai-marketplace-monitor/blob/main/docs/README.md) Explore comprehensive guides and instructions.
- 🤝 [Discussions](https://github.com/BoPeng/ai-marketplace-monitor/discussions): Connect with the community, ask questions, and exchange ideas.
- 🐛 [Issues](https://github.com/BoPeng/ai-marketplace-monitor/issues): Report bugs or suggest new features to help improve the project.
- 💖 [Become a sponsor](https://github.com/sponsors/BoPeng): Support the development and maintenance of this tool. Any contribution, no matter how small, is deeply appreciated.
- 💰 [Donate via PayPal](https://www.paypal.com/donate/?hosted_button_id=3WT5JPQ2793BN): Prefer private support? Consider donating via PayPal.
- 📖 [Documentation](https://ai-marketplace-monitor.readthedocs.io/) - Comprehensive guides and instructions
- 🤝 [Discussions](https://github.com/BoPeng/ai-marketplace-monitor/discussions) - Community support and ideas
- 🐛 [Issues](https://github.com/BoPeng/ai-marketplace-monitor/issues) - Bug reports and feature requests
- 💖 [Become a sponsor](https://github.com/sponsors/BoPeng) - Support development
- 💰 [Donate via PayPal](https://www.paypal.com/donate/?hosted_button_id=3WT5JPQ2793BN) - Alternative donation method
**Important Note:**
**Important Note:** Due to time constraints, priority support is provided to sponsors and donors. For general questions, please use the GitHub Discussions or Issues.
Due to time constraints, answering individual inquiries about _AI Marketplace Monitor_ on a one-on-one basis is not scalable. While I enjoy engaging on platforms like Reddit, GitHub, and email, I am generally unable to respond to personal emails or direct messages unless:
- You are a sponsor or donor.
- Your inquiry is related to business opportunities.
I greatly appreciate your understanding. To help expedite responses, please remember to mention your **sponsor or donation status** when contacting me.
## Credits
## 🙏 Credits
- Some of the code was copied from [facebook-marketplace-scraper](https://github.com/passivebot/facebook-marketplace-scraper).
- Region definitions were copied from [facebook-marketplace-nationwide](https://github.com/gmoz22/facebook-marketplace-nationwide/), which is released under an MIT license as of Jan 2025.

489
docs/advanced-features.rst Normal file
View File

@@ -0,0 +1,489 @@
================
Advanced Features
================
This section covers advanced configuration options and setup procedures for AI Marketplace Monitor.
Setting Up Email Notifications
==============================
To send email notifications, you need to specify recipient email addresses in the `email` of a `user` or a notification setting. You can configure multiple users with individual or multiple email addresses like this:
.. code-block:: toml
[user.user1]
email = 'user1@gmail.com'
[user.user2]
email = ['user2@gmail.com', 'user2@outlook.com']
SMTP Configuration
------------------
An SMTP server is required for sending emails, for which you will need to know `smtp_server`, `smtp_port`, `smtp_username` and `smtp_password`. Generally speaking, you will need to create a notification section with the information obtained from your email service provider.
.. code-block:: toml
[notification.myprovider]
smtp_username = 'username@EMAIL.COM' # default to email
smtp_server = 'smtp.EMAIL.COM' # default to smtp.EMAIL.COM
smtp_port = 587 # default for most providers
smtp_password = 'mypassword'
`ai-marketplace-monitor` will try to use `email` if `smtp_username` is unspecified, and determine `smtp_username` and `smtp_server` automatically from the sender email address. For example, your Gmail setup could be as simple as:
.. code-block:: toml
[notification.gmail]
smtp_password = 'abcdefghijklmnop'
You can specify `smtp_password` directly in the `user` section if you are not sharing the `notification` setting with other users.
.. code-block:: toml
[user.me]
email = 'myemail@gmail.com'
smtp_password = 'abcdefghijklmnop'
.. note::
- **Gmail Users**: You will need to create a separate app password for your Google account as `smtp_password`.
- **Commercial Users**: If you are a subscriber to our Pro or Business Plans, detailed instructions on configuring the SMTP service we provide will be sent to you via email.
Setting Up PushOver Notifications
=================================
To enable PushOver notifications, follow these steps:
1. **Install the PushOver app** on your mobile device.
2. **Create a PushOver account** at `pushover.net <https://pushover.net>`_. After registration, you will find your **User Key** labeled as `Your User Key` — this is your `pushover_user_key`.
3. **Create a new application** (you can name it `AIMarketplaceMonitor`). After creation, you will receive an **API Token/Key**, referred to as `pushover_api_token`.
Configuration Options
--------------------
Once you have both the user key and API token, add them to your configuration file using one of the following formats:
**Option 1: Embed directly under your user profile**
.. code-block:: toml
[user.me]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
**Option 2: Use a dedicated notification section**
.. code-block:: toml
[notification.pushover]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
[user.me]
notify_with = 'pushover'
Description Settings
-------------------
By default, notifications include the **title**, **price**, **location**, **description**, and **AI-generated comments** (if enabled). To exclude or limit the length of the **listing description**, you can add the `with_description` option to your config.
You can set `with_description` to:
- `True` — to include the **full description**.
- `False` — to exclude the description (default behavior).
- A **number** — to include only the **first N characters** of the description.
For example:
.. code-block:: toml
[user.me]
pushover_user_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
pushover_api_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
with_description = 100
This will include up to the first 100 characters of each listing's description in your notifications.
Setting Up Telegram Notifications
=================================
To enable Telegram notifications, you'll need to create a Telegram bot and obtain a chat ID.
Step 1: Create a Telegram Bot
-----------------------------
1. **Open Telegram** and search for `@BotFather` (the official bot for creating other bots).
2. **Start a conversation** with BotFather by clicking "Start" or sending `/start`.
3. **Create a new bot** by sending the command `/newbot`.
4. **Choose a bot name** when prompted (e.g., "AI Marketplace Monitor").
5. **Choose a bot username** that ends with "bot" (e.g., "my_marketplace_monitor_bot").
6. **Save your bot token** - BotFather will provide a token that looks like `123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11`. **Keep this token secure and never share it publicly.**
Step 2: Get Your Chat ID
------------------------
You need to find your chat ID to receive messages. Here are two methods:
**Method 1: Using @userinfobot**
1. Search for `@userinfobot` in Telegram and start a conversation.
2. Send any message to the bot.
3. The bot will reply with your user information, including your **Chat ID** (a number like `123456789`).
**Method 2: Using the Telegram Bot API**
1. Start a conversation with your newly created bot (search for its username).
2. Send any message to your bot (e.g., "Hello").
3. Open this URL in your browser, replacing `YOUR_BOT_TOKEN` with your actual token::
https://api.telegram.org/botYOUR_BOT_TOKEN/getUpdates
4. Look for the `"chat":{"id":` field in the response - this number is your chat ID.
Step 3: Configure Your Settings
-------------------------------
Add your Telegram credentials to your configuration file using one of these formats:
**Option 1: Direct configuration under user profile**
.. code-block:: toml
[user.me]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '123456789'
**Option 2: Using a dedicated notification section**
.. code-block:: toml
[notification.telegram]
telegram_token = '123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11'
telegram_chat_id = '123456789'
[user.me]
notify_with = 'telegram'
**Option 3: Using environment variables for security**
.. code-block:: toml
[user.me]
telegram_token = '${TELEGRAM_BOT_TOKEN}'
telegram_chat_id = '${TELEGRAM_CHAT_ID}'
Then set the environment variables:
.. code-block:: bash
export TELEGRAM_BOT_TOKEN="123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11"
export TELEGRAM_CHAT_ID="123456789"
Telegram Troubleshooting
------------------------
**401 Unauthorized Error**
- **Cause**: Invalid or incorrect bot token
- **Solution**:
1. Verify your bot token is correct (it should look like `123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11`)
2. Make sure there are no extra spaces or characters
3. Create a new bot with @BotFather if the token is lost
**403 Forbidden Error**
- **Cause**: Bot doesn't have permission to send messages to the chat
- **Solution**:
1. Start a conversation with your bot by searching for its username in Telegram
2. Send at least one message to the bot (e.g., "/start" or "Hello")
3. Verify the chat ID is correct
**400 Bad Request Error**
- **Cause**: Invalid chat ID format or the chat doesn't exist
- **Solution**:
1. Double-check your chat ID is a number (positive for users, negative for groups)
2. For group chats, make sure the bot is added to the group
3. Use the getUpdates method to verify your chat ID
AI Prompt Customization
=======================
_ai-marketplace-monitor_ asks AI services to evaluate listings against the criteria that you specify with prompts in four parts:
**Part 1: Buyer Intent**
.. code-block:: text
A user wants to buy a ... with search phrase ... description ..., price range ...,
with keywords .... and exclude ...
**Part 2: Listing Details**
.. code-block:: text
The user found a listing titled ... priced at ..., located ... posted at ...
with description ...
**Part 3: Instruction to AI**
.. code-block:: text
Evaluate how well this listing matches the user's criteria. Assess the description,
MSRP, model year, condition, and seller's credibility.
**Part 4: Rating Instructions**
.. code-block:: text
Rate from 1 to 5 based on the following:
1 - No match: Missing key details, wrong category/brand, or suspicious activity (e.g., external links).
2 - Potential match: Lacks essential info (e.g., condition, brand, or model); needs clarification.
3 - Poor match: Some mismatches or missing details; acceptable but not ideal.
4 - Good match: Mostly meets criteria with clear, relevant details.
5 - Great deal: Fully matches criteria, with excellent condition or price.
Conclude with:
"Rating [1-5]: [summary]"
where [1-5] is the rating and [summary] is a brief recommendation (max 30 words)."
Custom Prompts
--------------
Depending on your specific needs, you can replace part 3 and part 4 of the prompt with options `prompt` and `rating_prompt`, and add an extra prompt before rating prompt with option `extra_prompt`. These options can be specified at the `marketplace` and `item` levels, with the latter overriding the former.
For example, you can add:
.. code-block:: toml
[marketplace.facebook]
extra_prompt = """Exclude any listing that recommend visiting an external website \
for purchase."""
to describe suspicious listings in a marketplace, and:
.. code-block:: toml
[item.ipadpro]
prompt = """Find market value for listing on market places like Ebay \
or Facebook marketplace and compare the price of the listing, considering \
the description, selling price, model year, condition, and seller's \
credibility. Evaluate how well this listing matches the user's criteria.
"""
Rating Thresholds
----------------
When AI services are used, the program by default notifies you of all listing with a rating of 3 or higher. You can change this behavior by setting for example:
.. code-block:: toml
rating = 4
to see only listings that match your criteria well. Note that all listings after non-AI-based filtering will be returned if no AI service is specified or non-functional.
Advanced Keyword-Based Filters
==============================
Options `keywords` and `antikeywords` are used to filter listings according to specified keywords. In the simplest form, these options support a single string. For example:
.. code-block:: toml
keywords = 'drone'
antikeywords = 'Parrot'
will select all listings with `drone` in title or description, and `Parrot` not in title or description.
Boolean Operators
----------------
You can use multiple keywords and operators `AND`, `OR`, and `NOT` in the parameter. For example:
.. code-block:: toml
keywords = 'DJI AND drone'
looks for listings with both `DJI` and `drone` in title or description.
If you have multiple keywords specified in a list, they are by default joint by `OR`. That is to say:
.. code-block:: toml
keywords = ['drone', 'DJI', 'Orqa']
antikeywords = ['Parrot', 'Autel']
is equivalent to:
.. code-block:: toml
keywords = 'drone OR DJI OR Orqa'
antikeywords = 'Parrot OR Autel'
which means selecting listings that contains `drone` or `DJI` or `Orga` in title or description, but exclude those listings with `Parrot` or `Autel` in title or description.
Complex Expressions
-------------------
These criteria will however, not exclude listings for a `DJI Camera`. If you would like to make sure that `drone` is selected, you can use:
.. code-block:: toml
keywords = 'drone AND (DJI OR Orqa)'
antikeywords = 'Parrot OR Autel'
If you have special characters and spaces in your keywords, you will need to quote them, such as:
.. code-block:: toml
keywords = '("Go Pro" OR gopro) AND HERO'
.. note::
1. A list of logical operations are allowed, and they are assumed to be joint by `OR`. For example, `['gopro AND (11 or 12)', 'DJI AND OSMO']` searches for either a gopro version 11 or 12, or a DJI OSMO camera.
2. You can construct very complex logical operations using `AND`, `OR` and `NOT`, but it is usually recommended to use simple keyword-based filtering and let AI handle more subtle selection criteria.
Multi-Location and Region Search
================================
`search_city` is the name, sometimes numbers, used by Facebook marketplace to represent a city. To get the value of `search_city` for your region, visit facebook marketplace, perform a search, and the city should be the name after `marketplace` (e.g. `XXXXX` in a URL like `https://www.facebook.com/marketplace/XXXXX/search?query=YYYY`).
Multiple Cities
---------------
Multiple searches will be performed if multiple cities are provided to option `search_city`. You can also specify `seller_locations` to limit the location of sellers. These locations are names of cities as displayed on the listing pages.
.. code-block:: toml
[item.name]
search_city = ['city1', 'city2']
seller_locations = ['city1', 'city2', 'city3', 'city4']
You can also increase the radius of search using:
.. code-block:: toml
[item.name]
search_city = ['city1', 'city2']
radius = 50
Pre-defined Regions
------------------
However, if you would like to search for a larger region (e.g. the USA), it is much easier to define `region`s with a list of `search_city` and large `radius`.
_ai-marketplace-monitor_ defines the following regions in its system:
- `usa` for USA (without AK or HI), with currency `USD`
- `usa_full` for USA, with currency `USD`
- `can` for Canada, with currency `CAD`
- `mex` for Mexico, with currency `MXN`
- `bra` for Brazil, with currency `BRL`
- `arg` for Argentina, with currency `ARS`
- `aus` for Australia, with currency `AUD`
- `aus_miles` for Australia using 500 miles radius, with currency `AUD`
- `nzl` for New Zealand, with currency `NZD`
- `ind` for India, with currency `INR`
- `gbr` for United Kingdom, with currency `GBP`
- `fra` for France, with currency `EUR`
- `spa` for Spain, with currency `EUR`
Now, if you would like to search an item across the US, you can:
.. code-block:: toml
[item.name]
search_region = 'usa'
seller_locations = []
delivery_method = 'shipping'
Under the hood, _ai-marketplace-monitor_ will simply replace `search_region` with corresponding pre-defined `search_city`, `radius`, and `currency`. Note that `seller_locations` does not make sense and need to be set to empty for region-based search, and it makes sense to limit the search to listings that offer shipping.
Multi-Currency Support
======================
_AI Marketplace Monitor_ does not enforce any specific currency format for price filters. It assumes that the `min_price` and `max_price` values are provided in the currency commonly used in the specified `search_city`. For example, in the configurations below:
.. code-block:: toml
[item.item1]
min_price = 100
search_city = 'newyork' # for demonstration only, city name for newyork might differ
.. code-block:: toml
[item.item1]
min_price = 100
search_city = 'paris' # for demonstration only, city name for paris might differ
The `min_price` is interpreted as 100 `USD` for New York and 100 `EUR` for Paris, based on the typical local currency of each city.
Explicit Currency Configuration
------------------------------
If you perform a search across cities that use different currencies, you can explicitly define the currencies using the `currency` option:
.. code-block:: toml
[item.item1]
min_price = '100 USD'
search_city = ['paris', 'newyork']
currency = ['EUR', 'USD']
In this example, the system will perform two searches and convert the `min_price` of `100` `USD` into the equivalent amount in `EUR` when searching `item1` around Paris, using historical exchange rates provided by the Currency Converter package.
All pre-defined regions has a defined `currency`. If you would like to search across regions with different currencies, you can:
.. code-block:: toml
[item.item1]
min_price = '100 EUR'
search_region = ['fra', 'gbr']
and _AI Marketplace Monitor_ will automatically convert `100 EUR` to `GBP` when searching United Kingdom.
.. note::
1. The following currency codes are supported: `USD`, `JPY`, `BGN`, `CYP`, `EUR`, `CZK`, `DKK`, `EEK`, `GBP`, `HUF`, `LTL`, `LVL`, `MTL`, `PLN`, `ROL`, `RON`, `SEK`, `SIT`, `SKK`, `CHF`, `ISK`, `NOK`, `HRK`, `RUB`, `TRL`, `TRY`, `AUD`, `BRL`, `CAD`, `CNY`, `HKD`, `IDR`, `ILS`, `INR`, `KRW`, `MXN`, `MYR`, `NZD`, `PHP`, `SGD`, `THB`, `ZAR`, and `ARS`.
2. Currency conversion only occurs if currencies are explicitly defined and differ between cities or from the currency used in `min_price`/`max_price`.
3. Conversion rates are intended for basic filtering and may not reflect real-time market values.
Self-hosted AI with Ollama
==========================
If you have access to a decent machine and prefer not to pay for AI services from OpenAI or other vendors, you can opt to install Ollama locally and access it using the `provider = "ollama"`. If you have ollama on your local host, you can use:
.. code-block:: toml
[ai.ollama]
base_url = "http://localhost:11434/v1"
model = "deepseek-r1:14b"
timeout = 120
.. note::
1. Depending on your hardware configuration, you can choose any of the models listed at `ollama.com/library <https://ollama.com/library>`_. The default model is `deepseek-r1:14b` because it appears to work better than `llama-3.1:8b`.
2. You need to `pull` the model before you can use it.
Anonymous Search with Proxy
===========================
You can search Facebook Marketplace anonymously by disabling login:
- Do not provide a `username` or `password` in the `facebook` section
- (optional) Set `login_wait_time = 0` to stop waiting for login
- (optional) Use the `--headless` command line option to run `ai-marketplace-monitor` without a browser window.
Proxy Configuration
-------------------
If you would like to use a proxy server, you can:
- Sign up for a VPN or proxy service.
- Configure the proxy settings in the `monitor` section of your configuration file as follows:
.. code-block:: toml
[monitor]
proxy_server = '${PROXY_SERVER}'
proxy_username = '${PROXY_USERNAME}'
proxy_password = '${PROXY_PASSWORD}'
Replace `${PROXY_SERVER}`, `${PROXY_USERNAME}`, and `${PROXY_PASSWORD}` with your proxy service details, or setting the corresponding environment variables.

141
docs/ai_marketplace_monitor.rst Normal file
View File

@@ -0,0 +1,141 @@
ai\_marketplace\_monitor package
================================
Submodules
----------
ai\_marketplace\_monitor.ai module
----------------------------------
.. automodule:: ai_marketplace_monitor.ai
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.cli module
-----------------------------------
.. automodule:: ai_marketplace_monitor.cli
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.config module
--------------------------------------
.. automodule:: ai_marketplace_monitor.config
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.email\_notify module
---------------------------------------------
.. automodule:: ai_marketplace_monitor.email_notify
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.facebook module
----------------------------------------
.. automodule:: ai_marketplace_monitor.facebook
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.listing module
---------------------------------------
.. automodule:: ai_marketplace_monitor.listing
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.marketplace module
-------------------------------------------
.. automodule:: ai_marketplace_monitor.marketplace
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.monitor module
---------------------------------------
.. automodule:: ai_marketplace_monitor.monitor
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.notification module
--------------------------------------------
.. automodule:: ai_marketplace_monitor.notification
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.ntfy module
------------------------------------
.. automodule:: ai_marketplace_monitor.ntfy
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.pushbullet module
------------------------------------------
.. automodule:: ai_marketplace_monitor.pushbullet
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.pushover module
----------------------------------------
.. automodule:: ai_marketplace_monitor.pushover
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.region module
--------------------------------------
.. automodule:: ai_marketplace_monitor.region
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.telegram module
----------------------------------------
.. automodule:: ai_marketplace_monitor.telegram
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.user module
------------------------------------
.. automodule:: ai_marketplace_monitor.user
:members:
:show-inheritance:
:undoc-members:
ai\_marketplace\_monitor.utils module
-------------------------------------
.. automodule:: ai_marketplace_monitor.utils
:members:
:show-inheritance:
:undoc-members:
Module contents
---------------
.. automodule:: ai_marketplace_monitor
:members:
:show-inheritance:
:undoc-members:

41
docs/conf.py
View File

@@ -37,7 +37,12 @@ extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.viewcode",
"sphinx.ext.napoleon",
"recommonmark",
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
"sphinx.ext.coverage",
"sphinx.ext.imgmath",
"sphinx.ext.ifconfig",
"myst_parser",
]
# Add any paths that contain templates here, relative to this directory.
@@ -54,17 +59,43 @@ exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "alabaster"
html_theme = "furo"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
"source_repository": "https://github.com/BoPeng/ai-marketplace-monitor/",
"source_branch": "main",
"source_directory": "docs/",
"edit_page": True,
}
# Intersphinx configuration
intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
"playwright": ("https://playwright.dev/python/", None),
}
# MyST parser configuration
myst_enable_extensions = [
"colon_fence",
"deflist",
"html_admonition",
"html_image",
"linkify",
"replacements",
"smartquotes",
"substitution",
"tasklist",
]
# Additional HTML context
html_context = {
"display_github": True,
"github_user": "BoPeng",
"github_repo": "ai-marketplace-monitor",
"github_banner": True,
"show_related": False,
"fixed_sidebar": True,
"github_version": "main/docs/",
}
# Add any paths that contain custom static files (such as style sheets) here,

2
docs/configuration.rst Normal file
View File

@@ -0,0 +1,2 @@
.. include:: README.md
:parser: myst_parser.sphinx_

296
docs/contributing.rst Normal file
View File

@@ -0,0 +1,296 @@
============
Contributing
============
We welcome contributions to AI Marketplace Monitor! Here's how you can help make this project better.
Ways to Contribute
==================
🐛 **Report Bugs and Issues**
- Search existing issues first
- Provide detailed reproduction steps
- Include system information and error messages
💡 **Suggest New Features**
- Open a feature request issue
- Explain the use case and expected behavior
- Discuss implementation approaches
🔧 **Submit Pull Requests**
- Fix bugs or implement new features
- Follow our coding standards
- Add tests for new functionality
📚 **Improve Documentation**
- Fix typos and unclear sections
- Add examples and tutorials
- Translate documentation
🏪 **Add Marketplace Support**
- Implement new marketplace backends
- Extend the marketplace abstraction
- Add region and currency support
🌍 **Internationalization**
- Add support for new languages
- Create translation dictionaries
- Test non-English Facebook pages
🤖 **AI Provider Integration**
- Add new AI service providers
- Improve prompt engineering
- Test different AI models
📱 **Notification Methods**
- Implement new notification backends
- Improve existing notification features
- Add notification customization
Development Setup
=================
Prerequisites
-------------
- Python 3.10 or higher
- Git
- `uv` package manager (recommended) or pip
Clone and Setup
---------------
.. code-block:: console
$ git clone https://github.com/BoPeng/ai-marketplace-monitor.git
$ cd ai-marketplace-monitor
$ uv sync --extra dev
$ playwright install
$ uv run invoke install-hooks
This installs:
- All dependencies including development tools
- Playwright browsers for testing
- Pre-commit hooks for code quality
Development Workflow
===================
Code Quality
-----------
Run all quality checks:
.. code-block:: console
$ uv run invoke lint # Run linting and formatting checks
$ uv run invoke mypy # Type checking
$ uv run invoke security # Security vulnerability scan
Auto-format code:
.. code-block:: console
$ uv run invoke format # Format with black and isort
Testing
-------
Run the test suite:
.. code-block:: console
$ uv run invoke tests # Run pytest with coverage
$ uv run pytest tests/ # Run specific tests
$ nox # Test across Python versions
Add tests for new features:
- Unit tests in ``tests/test_*.py``
- Mock external services (Facebook, AI APIs)
- Use fixtures for common test data
Documentation
------------
Build documentation locally:
.. code-block:: console
$ uv run invoke docs # Build Sphinx documentation
$ uv run invoke docs --serve --open-browser # Live preview
Documentation files:
- ``docs/*.rst`` - Main documentation pages
- ``README.md`` - Project overview
- ``CHANGELOG.md`` - Version history
- Docstrings in code for API documentation
Coding Standards
===============
Style Guidelines
---------------
- **Line length**: 99 characters maximum
- **Formatting**: Use black and isort (automated by pre-commit)
- **Linting**: Follow ruff recommendations
- **Type hints**: Required for all public functions
Code Organization
----------------
- ``src/ai_marketplace_monitor/`` - Main package
- ``tests/`` - Test files matching ``test_*.py``
- ``docs/`` - Sphinx documentation
- ``noxfile.py`` - Multi-environment testing
- ``tasks.py`` - Development tasks (invoke)
Architecture Patterns
---------------------
- **Abstract base classes** for marketplaces and AI backends
- **Configuration inheritance** from marketplace to item level
- **Plugin architecture** for extensible components
- **Caching strategy** to minimize external API calls
Commit Guidelines
================
Commit Message Format
--------------------
.. code-block:: text
type(scope): brief description
Longer explanation of the change if needed.
Closes #123
Types:
- ``feat``: New feature
- ``fix``: Bug fix
- ``docs``: Documentation changes
- ``style``: Code formatting
- ``refactor``: Code restructuring
- ``test``: Adding tests
- ``chore``: Maintenance tasks
Examples:
.. code-block:: text
feat(telegram): add support for group chat notifications
fix(facebook): handle new marketplace layout changes
docs: add troubleshooting section for AI service errors
Pull Request Process
===================
Before Submitting
----------------
1. **Create an issue** first to discuss major changes
2. **Fork the repository** and create a feature branch
3. **Write tests** for new functionality
4. **Update documentation** as needed
5. **Run all checks** locally
.. code-block:: console
$ uv run invoke lint
$ uv run invoke tests
$ uv run invoke mypy
Pull Request Checklist
---------------------
- [ ] Tests pass locally
- [ ] Code follows style guidelines
- [ ] Documentation updated
- [ ] CHANGELOG.md updated (for significant changes)
- [ ] Commit messages are clear
- [ ] PR description explains the change
Review Process
-------------
1. **Automated checks** must pass (CI/CD)
2. **Code review** by maintainers
3. **Testing** on different environments
4. **Documentation review** if applicable
5. **Merge** when approved
Community Guidelines
===================
Code of Conduct
---------------
This project follows the `Contributor Covenant Code of Conduct <https://www.contributor-covenant.org/version/2/1/code_of_conduct/>`_.
Key principles:
- Be respectful and inclusive
- Focus on what's best for the community
- Accept constructive criticism gracefully
- Help newcomers get involved
Communication Channels
---------------------
- **GitHub Issues**: Bug reports and feature requests
- **GitHub Discussions**: Questions and community chat
- **Pull Requests**: Code changes and reviews
Support for Contributors
========================
Getting Help
-----------
- **Documentation**: Start with this guide and the main docs
- **Issues**: Search existing issues for similar problems
- **Discussions**: Ask questions in GitHub Discussions
- **Code**: Read existing code for patterns and examples
Recognition
----------
Contributors are recognized in:
- ``CONTRIBUTORS.md`` file
- GitHub contributor graphs
- Release notes for significant contributions
- Special thanks in documentation
Getting Started with Your First Contribution
===========================================
Good First Issues
----------------
Look for issues labeled:
- ``good first issue`` - Simple changes perfect for newcomers
- ``documentation`` - Documentation improvements
- ``help wanted`` - Community input needed
Simple Contributions
-------------------
1. **Fix typos** in documentation
2. **Add examples** to configuration guide
3. **Improve error messages** for better user experience
4. **Add tests** for existing functionality
5. **Translate** interface messages
Example First Contribution
--------------------------
1. Find a typo in documentation
2. Fork the repository
3. Fix the typo in your fork
4. Submit a pull request
5. Celebrate your contribution! 🎉
Thank you for contributing to AI Marketplace Monitor!

74
docs/features.rst Normal file
View File

@@ -0,0 +1,74 @@
========
Features
========
🔍 **Smart Search**
-------------------
- **Multi-Product Search**: Search for multiple products using customizable keywords
- **Advanced Filtering**: Filter by price range, location, and seller criteria
- **Spam Detection**: Automatically exclude irrelevant results and known spammers
- **Layout Support**: Compatible with different Facebook Marketplace layouts
🤖 **AI-Powered Analysis**
--------------------------
- **Intelligent Evaluation**: AI services analyze each listing against your specific criteria
- **Smart Recommendations**: Get AI-generated insights on deal quality and condition assessment
- **Multiple AI Providers**: Support for OpenAI, DeepSeek, Ollama, and other OpenAI-compatible APIs
- **Self-Hosted Options**: Run your own AI models locally using Ollama
- **Customizable Prompts**: Tailor AI evaluation criteria to your specific needs
📱 **Comprehensive Notifications**
----------------------------------
- **Multiple Channels**: PushBullet, PushOver, Telegram, Ntfy, and HTML email notifications
- **Rich Content**: Notifications include images, descriptions, and AI-generated comments
- **Customizable Levels**: Set notification thresholds based on AI rating scores
- **Repeated Notifications**: Optional reminders for listings that remain available
- **Multi-User Support**: Configure different notification preferences for different users
🌎 **Global Location Support**
------------------------------
- **Multi-City Search**: Search across multiple cities simultaneously
- **Pre-Defined Regions**: Built-in support for USA, Canada, Australia, UK, and more
- **Custom Regions**: Define your own search areas with customizable radius
- **Seller Location Filtering**: Filter results by seller proximity
- **Multi-Currency**: Automatic currency conversion for international searches
⚙️ **Advanced Configuration**
-----------------------------
- **TOML Configuration**: Human-readable configuration files with environment variable support
- **Hot Reloading**: Automatic restart when configuration changes are detected
- **Keyword Logic**: Complex boolean search expressions with AND, OR, NOT operators
- **Multi-Language**: Support for Spanish, Chinese, and extensible translation system
- **Proxy Support**: Anonymous searching with proxy server integration
🔒 **Security & Privacy**
-------------------------
- **Environment Variables**: Keep sensitive data out of config files
- **Anonymous Mode**: Search without logging into Facebook
- **Proxy Integration**: Route traffic through VPN/proxy services
- **Local AI**: Use self-hosted Ollama models to keep data private
- **Cache Management**: Control what data is stored locally
📊 **Monitoring & Analytics**
-----------------------------
- **Search Statistics**: Track pages searched, listings examined, and matches found
- **Performance Metrics**: Monitor search frequency and success rates
- **Interactive Mode**: Real-time statistics and manual listing checks
- **Persistent Counters**: Statistics maintained across program restarts
- **Individual Listing Analysis**: Debug why specific listings were included or excluded
🔧 **Developer Features**
----------------------
- **Plugin Architecture**: Extensible marketplace and AI backend support
- **Caching System**: Efficient caching prevents redundant API calls
- **Browser Automation**: Robust Playwright integration with login handling
- **Configuration Validation**: Comprehensive error checking and helpful messages
- **Extensive Testing**: Full test suite with CI/CD integration

50
docs/index.rst
View File

@@ -1,22 +1,54 @@
Welcome to AI Marketplace Monitor's documentation!
===========================================================
AI Marketplace Monitor Documentation
====================================
An intelligent tool that monitors Facebook Marketplace listings using AI to help you find the best deals. Get instant notifications when items matching your criteria are posted, with AI-powered analysis of each listing.
.. image:: AIMM_neutral.png
:alt: AI Marketplace Monitor Logo
:align: center
Getting Started
===============
.. toctree::
:maxdepth: 2
readme
quickstart
installation
usage
features
Configuration & Setup
====================
.. toctree::
:maxdepth: 2
configuration
advanced-features
troubleshooting
Development & Reference
======================
.. toctree::
:maxdepth: 2
modules
changelog
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
contributing
.. toctree::
:hidden:
:caption: Project Links
GitHub Repository <https://github.com/BoPeng/ai-marketplace-monitor>
PyPI Package <https://pypi.org/project/ai-marketplace-monitor/>
License <license>
Indices and Tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

38
docs/installation.rst
View File

@@ -4,20 +4,42 @@
Installation
============
Prerequisites
-------------
Stable release
--------------
- Python 3.10 or higher
- Internet connection
To install ai-marketplace-monitor, run this command in your terminal:
Quick Installation
------------------
Install the program:
.. code-block:: console
$ pip install ai-marketplace-monitor
This is the preferred method to install ai-marketplace-monitor, as it will always install the most recent stable release.
Install a browser for Playwright:
If you don't have `pip`_ installed, this `Python installation guide`_ can guide
you through the process.
.. code-block:: console
.. _pip: https://pip.pypa.io
.. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/
$ playwright install
For more detailed instructions, please refer to:
- `Linux (pipx) installation guide <https://github.com/BoPeng/ai-marketplace-monitor/blob/main/INSTALL.linux.md>`_
- `Community installation instructions #234 <https://github.com/BoPeng/ai-marketplace-monitor/issues/234>`_
Development Installation
------------------------
If you want to contribute to the project:
.. code-block:: console
$ git clone https://github.com/BoPeng/ai-marketplace-monitor.git
$ cd ai-marketplace-monitor
$ uv sync --extra dev
$ playwright install
This will install the project with development dependencies using `uv`.

1
docs/license.rst
View File

@@ -1 +0,0 @@
../LICENSE.rst

6
docs/license.rst Normal file
View File

@@ -0,0 +1,6 @@
=======
License
=======
.. include:: ../LICENSE
:literal:

7
docs/modules.rst Normal file
View File

@@ -0,0 +1,7 @@
ai_marketplace_monitor
======================
.. toctree::
:maxdepth: 4
ai_marketplace_monitor

141
docs/quickstart.rst Normal file
View File

@@ -0,0 +1,141 @@
==========
Quickstart
==========
This guide will get you up and running with AI Marketplace Monitor in under 10 minutes.
.. warning::
**Important Legal Notice**: Facebook's EULA prohibits automated data collection without authorization. This tool was developed for personal, hobbyist use only. You are solely responsible for ensuring compliance with platform terms and applicable laws. For commercial use, obtain explicit permission from Meta first.
Step 1: Installation
--------------------
Install AI Marketplace Monitor:
.. code-block:: console
$ pip install ai-marketplace-monitor
Install Playwright browser:
.. code-block:: console
$ playwright install
Step 2: Set Up Notifications (Optional)
---------------------------------------
Choose one notification method:
**PushBullet** (Recommended for beginners):
1. Sign up at `pushbullet.com <https://www.pushbullet.com/>`_
2. Install the app on your phone
3. Get your API token from the website
**Email**:
1. Use your existing email account
2. Get SMTP settings from your email provider
3. For Gmail, create an app password
Step 3: Create Configuration
---------------------------
Create the configuration directory:
.. code-block:: console
$ mkdir -p ~/.ai-marketplace-monitor
Create a minimal configuration file at ``~/.ai-marketplace-monitor/config.toml``:
.. code-block:: toml
[marketplace.facebook]
search_city = 'houston' # Replace with your city
[item.gopro]
search_phrases = 'Go Pro Hero 11'
min_price = 100
max_price = 300
[user.me]
pushbullet_token = 'your_pushbullet_token_here'
.. note::
Replace ``'houston'`` with your city name and ``'your_pushbullet_token_here'`` with your actual PushBullet token.
Step 4: Add AI Service (Optional but Recommended)
------------------------------------------------
Sign up for an AI service like `OpenAI <https://openai.com/>`_ or `DeepSeek <https://www.deepseek.com/>`_ and add to your config:
.. code-block:: toml
[ai.openai]
api_key = 'your_openai_api_key'
# ... rest of your config
Step 5: Run the Monitor
----------------------
Start monitoring:
.. code-block:: console
$ ai-marketplace-monitor
What happens next:
1. **Browser Opens**: A browser window will appear
2. **Login Prompt**: Enter Facebook credentials if prompted
3. **CAPTCHA**: Complete any CAPTCHA challenges
4. **Monitoring Starts**: The program begins searching automatically
5. **Notifications**: You'll receive notifications when matches are found
Step 6: Test Your Setup
-----------------------
To verify everything works, check a specific listing:
.. code-block:: console
$ ai-marketplace-monitor --check https://facebook.com/marketplace/item/123456789
Example Output
-------------
When the monitor finds a matching item, you'll see console output like:
.. code-block:: text
[2025-01-08 10:30:15] Found 1 new gopro from facebook
[Great deal (5)] Go Pro Hero 12
$250, Houston, TX
https://facebook.com/marketplace/item/1234567890
AI: Excellent condition camera with original accessories - great value!
And receive a notification on your phone via PushBullet.
Next Steps
----------
- :doc:`configuration` - Learn about advanced configuration options
- :doc:`features` - Explore all available features
- :doc:`usage` - Master command-line options and interactive mode
- `GitHub Issues <https://github.com/BoPeng/ai-marketplace-monitor/issues>`_ - Get help or report problems
Common Issues
-------------
**"Config file not found"**
Make sure the file is at ``~/.ai-marketplace-monitor/config.toml``
**"Cannot login to Facebook"**
The monitor will still work but with limited results. Try providing username/password in config.
**"No notifications received"**
Check your PushBullet token and ensure the app is installed on your phone.
**Browser doesn't open**
Try running without ``--headless`` flag to see the browser window.

2
docs/readme.rst Normal file
View File

@@ -0,0 +1,2 @@
.. include:: ../README.md
:parser: myst_parser.sphinx_

8
docs/requirements.txt
View File

@@ -1,2 +1,6 @@
sphinx==8.2.3
recommonmark==0.7.1
sphinx>=8.1.3
furo>=2024.1.29
myst-parser>=3.0.0
sphinx-copybutton>=0.5.2
sphinxext-opengraph>=0.9.0
linkify-it-py>=2.0.0

280
docs/troubleshooting.rst Normal file
View File

@@ -0,0 +1,280 @@
===============
Troubleshooting
===============
Common Issues and Solutions
===========================
Configuration Problems
----------------------
**Config file not found**
.. code-block:: text
Error: Config file ~/.ai-marketplace-monitor/config.toml not found
*Solution:*
- Ensure the config file exists at ``~/.ai-marketplace-monitor/config.toml``
- Check file permissions (should be readable by your user)
- Use ``--config`` flag to specify a different location
**Invalid TOML syntax**
.. code-block:: text
Error: Invalid TOML configuration
*Solution:*
- Validate your TOML syntax at `toml-lint.com <https://www.toml-lint.com/>`_
- Check for missing quotes around strings
- Ensure proper section headers like ``[marketplace.facebook]``
Login and Authentication Issues
------------------------------
**Facebook login failure**
.. code-block:: text
Warning: Failed to login to Facebook
*Solution:*
- Provide username/password in config file
- Complete CAPTCHA challenges manually
- The monitor will continue with limited results if login fails
**Two-factor authentication required**
*Solution:*
- Complete 2FA manually in the browser window
- Consider using app passwords where supported
- Monitor will remember login state for future runs
Browser and Playwright Issues
-----------------------------
**Playwright browser not installed**
.. code-block:: text
Error: Browser executable not found
*Solution:*
.. code-block:: console
$ playwright install
**Browser crashes or hangs**
*Solution:*
- Restart the monitor
- Try headless mode: ``ai-marketplace-monitor --headless``
- Check system resources (RAM, CPU)
Notification Problems
--------------------
**PushBullet notifications not received**
*Solutions:*
- Verify API token is correct
- Check PushBullet app is installed and logged in
- Test token at `pushbullet.com <https://www.pushbullet.com/>`_
**Email notifications failing**
*Solutions:*
- Check SMTP settings (server, port, username, password)
- For Gmail, use app passwords instead of account password
- Verify firewall/antivirus isn't blocking SMTP
**Telegram bot not responding**
*Solutions:*
- Verify bot token format: ``123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11``
- Ensure you've started a conversation with the bot
- Check chat ID is correct (positive for users, negative for groups)
AI Service Issues
----------------
**OpenAI API errors**
.. code-block:: text
Error: OpenAI API request failed
*Solutions:*
- Check API key is valid and has sufficient credits
- Verify network connectivity
- Review OpenAI status page for service issues
**AI responses seem incorrect**
*Solutions:*
- Adjust AI prompts in configuration
- Try different AI models (gpt-4, gpt-3.5-turbo, etc.)
- Check if AI service has usage limits
Search and Filtering Problems
----------------------------
**No listings found**
*Solutions:*
- Verify search city name is correct
- Check price ranges aren't too restrictive
- Review keyword filters (``keywords`` and ``antikeywords``)
- Test without AI filtering to see raw results
**Too many irrelevant results**
*Solutions:*
- Add more specific keywords
- Use ``antikeywords`` to exclude unwanted terms
- Adjust AI rating threshold higher
- Refine item descriptions for better AI filtering
**Currency conversion issues**
*Solutions:*
- Check currency codes are valid (USD, EUR, GBP, etc.)
- Ensure currency is specified for multi-region searches
- Verify exchange rate data is available
Performance Issues
-----------------
**Monitor running slowly**
*Solutions:*
- Reduce search frequency in configuration
- Clear cache: ``ai-marketplace-monitor --clear-cache all``
- Check system resources
- Consider using fewer simultaneous searches
**High CPU/memory usage**
*Solutions:*
- Use headless mode: ``--headless``
- Reduce number of concurrent browser tabs
- Clear browser cache and data
- Consider running on a more powerful machine
Cache-Related Issues
-------------------
**Stale data or wrong notifications**
*Solution:*
.. code-block:: console
$ ai-marketplace-monitor --clear-cache all
**Cache corruption**
.. code-block:: text
Error: Cannot read cache data
*Solution:*
- Clear specific cache types:
.. code-block:: console
$ ai-marketplace-monitor --clear-cache listing-details
$ ai-marketplace-monitor --clear-cache ai-inquiries
$ ai-marketplace-monitor --clear-cache user-notification
Language and Localization Issues
--------------------------------
**Non-English Facebook pages**
.. code-block:: text
Failed to get details of listing. The listing might not be in English.
*Solutions:*
- Change Facebook language settings to English
- Add ``language`` option to marketplace configuration
- Define custom translation dictionary in config
Debug Mode and Logging
----------------------
**Enable verbose logging**
Add to your configuration:
.. code-block:: toml
[monitor]
log_level = "DEBUG"
**Check log files**
Logs are typically saved to:
- Console output (default)
- Log files if configured in your system
**Interactive debugging**
While monitor is running:
- Press ``Esc`` to view statistics
- Use ``--check URL`` to test individual listings
- Enter interactive mode to debug specific issues
Getting Help
============
If you're still having issues:
1. **Check GitHub Issues**: Search existing issues at `github.com/BoPeng/ai-marketplace-monitor/issues <https://github.com/BoPeng/ai-marketplace-monitor/issues>`_
2. **Community Support**: Join discussions at `github.com/BoPeng/ai-marketplace-monitor/discussions <https://github.com/BoPeng/ai-marketplace-monitor/discussions>`_
3. **Create New Issue**: If you find a bug, create a detailed issue report including:
- Your operating system
- Python version
- Complete error messages
- Configuration file (remove sensitive data)
- Steps to reproduce the problem
4. **Sponsor Support**: Sponsors and donors receive priority email support - mention your sponsor status when contacting.
Reporting Bugs
==============
When reporting bugs, please include:
.. code-block:: text
**Environment:**
- OS: [e.g., Ubuntu 20.04, macOS 12.0, Windows 10]
- Python: [e.g., 3.10.2]
- ai-marketplace-monitor: [e.g., 0.9.6]
**Configuration:**
```toml
# Your config file with sensitive data removed
```
**Error Message:**
```
# Complete error message/traceback
```
**Steps to Reproduce:**
1. Step one
2. Step two
3. Step three
**Expected Behavior:**
What should have happened
**Actual Behavior:**
What actually happened

92
docs/usage.rst
View File

@@ -1,7 +1,89 @@
=====
Usage
=====
===========
Usage Guide
===========
To use ai-marketplace-monitor in a project::
Basic Usage
-----------
import ai_marketplace_monitor
Run the monitor with default configuration:
.. code-block:: console
$ ai-marketplace-monitor
Run with a custom configuration file:
.. code-block:: console
$ ai-marketplace-monitor --config /path/to/your/config.toml
Run in headless mode (without browser window):
.. code-block:: console
$ ai-marketplace-monitor --headless
Check Individual Listings
-------------------------
You can check why a listing was excluded or test a listing against your configuration:
.. code-block:: console
$ ai-marketplace-monitor --check https://facebook.com/marketplace/item/123456789
For specific item configurations:
.. code-block:: console
$ ai-marketplace-monitor --check https://facebook.com/marketplace/item/123456789 --for item_name
Cache Management
---------------
Clear different types of cache:
.. code-block:: console
$ ai-marketplace-monitor --clear-cache listing-details
$ ai-marketplace-monitor --clear-cache ai-inquiries
$ ai-marketplace-monitor --clear-cache user-notification
$ ai-marketplace-monitor --clear-cache counters
$ ai-marketplace-monitor --clear-cache all
Important Notes
--------------
1. **Keep Terminal Running**: You need to keep the terminal running to allow the program to monitor continuously.
2. **Browser Interaction**: You will see a browser window open. You may need to manually:
- Enter username/password if not specified in config
- Complete CAPTCHA challenges
- Click "OK" to save passwords
3. **Login Requirements**: If login fails, the monitor continues but Facebook may show limited results.
4. **Configuration Updates**: The program automatically detects config file changes and restarts searches.
Interactive Mode
---------------
While the monitor is running, you can:
- Press ``Esc`` to view current statistics
- Enter interactive mode to check individual URLs
- Type ``exit`` to leave interactive mode
Cost Considerations
------------------
**Free Components:**
- The software itself (AGPL license)
**Usage-Based Costs:**
- Notification services (PushBullet, SMTP, etc.)
- AI platforms (OpenAI, DeepSeek, etc.)
**Infrastructure:**
- 24/7 operation requires a PC, server, or cloud hosting
- Example: AWS t3.micro (~$10/month for continuous operation)

9
pyproject.toml
View File

@@ -67,7 +67,14 @@ test = [
linters = ["isort>=5.13.2,<7.0.0", "black>=24.10,<26.0", "ruff>=0.9.2,<0.13.0"]
security = ["safety>=3.2.11"]
typing = ["mypy>=1.13.0"]
docs = ["sphinx>=8.1.3", "recommonmark>=0.7.1"]
docs = [
"sphinx>=8.1.3",
"furo>=2024.1.29",
"myst-parser>=3.0.0",
"sphinx-copybutton>=0.5.2",
"sphinxext-opengraph>=0.9.0",
"linkify-it-py>=2.0.0"
]
[tool.coverage.paths]
source = ["src", "*/site-packages"]

10
tasks.py
View File

@@ -90,6 +90,12 @@ def hooks(c: Context) -> None:
@task(name="format", help={"check": "Checks if source is formatted without applying changes"})
def format_(c: Context, check: bool = False) -> None:
"""Format code."""
# Fix whitespace and end-of-file issues using the same tools as pre-commit
if not check:
_run(c, "uv run pre-commit run trailing-whitespace --all-files")
_run(c, "uv run pre-commit run end-of-file-fixer --all-files")
# Run isort and black
isort_options = ["--check-only", "--diff"] if check else []
_run(c, f"uv run isort {' '.join(isort_options)} {PYTHON_TARGETS_STR}")
black_options = ["--diff", "--check"] if check else ["--quiet"]
@@ -161,8 +167,8 @@ def coverage(c: Context, fmt: str = "report", open_browser: bool = False) -> Non
)
def docs(c: Context, serve: bool = False, open_browser: bool = False) -> None:
"""Build documentation."""
_run(c, f"sphinx-apidoc -o {DOCS_DIR} {SOURCE_DIR}")
build_docs = f"sphinx-build -b html {DOCS_DIR} {DOCS_BUILD_DIR}"
_run(c, f"uv run sphinx-apidoc -f -o {DOCS_DIR} {SOURCE_DIR}")
build_docs = f"uv run sphinx-build -b html {DOCS_DIR} {DOCS_BUILD_DIR}"
_run(c, build_docs)
if open_browser:
webbrowser.open(DOCS_INDEX.absolute().as_uri())

2451
uv.lock generated
View File

File diff suppressed because it is too large Load Diff