From 38b796fd1257bcb4030c333327e56efa70d74ddf Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 14 Dec 2025 16:40:46 +1100 Subject: [PATCH] GEMINI.md for antigravity --- CLAUDE.md | 14 +- GEMINI.md | 409 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 416 insertions(+), 7 deletions(-) create mode 100644 GEMINI.md diff --git a/CLAUDE.md b/CLAUDE.md index 8ec938d..fb2e7e2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -60,7 +60,6 @@ docker-compose up -d ## Architecture -**⚠️ IMPORTANT: The codebase has been refactored into a modular structure.** The application now uses a clean `src/` package structure with clear separation of concerns. @@ -81,7 +80,7 @@ src/ ├── core/ # Core client (Twitch client) ├── exceptions.py # Custom exceptions ├── version.py # Version string -└── __main__.py # Entry point (replaces old main.py) +└── __main__.py # Entry point lang/ # Translation JSON files (19 languages) ├── English.json # Default/fallback translations @@ -122,7 +121,6 @@ lang/ # Translation JSON files (19 languages) - `MaintenanceService`: Periodic maintenance tasks - `MessageHandlerService`: Websocket message routing and handling -Note: The services layer is complete and handles all business logic that was previously in the monolithic client file. **src/models/channel.py** - Channel and Stream: @@ -168,9 +166,10 @@ Note: The services layer is complete and handles all business logic that was pre - Games to watch list (auto-populated from available campaigns if empty) - Connection quality multiplier - Language selection -- Proxy support +- Proxy support (including verification) - Logging and dump flags from command-line arguments - Persistence to JSON file (`settings.json`) in DATA_DIR +- Inventory filters (Status, Benefit Type, Game Search) ### State Machine Flow @@ -310,6 +309,7 @@ source env/bin/activate && python -m pytest tests/ - `tests/test_proxy_settings.py` - Tests for proxy settings configuration - `tests/test_verify_proxy.py` - Tests for proxy verification functionality + ### Manual Testing 1. Run with `-vvv` for maximum verbosity (levels: -v, -vv, -vvv, -vvvv) @@ -341,7 +341,7 @@ The application uses a web-based interface accessible via browser: **web/** - Frontend assets: - `index.html` - Single-page application layout with tabs -- `static/app.js` - Socket.IO client, real-time UI updates, API calls +- `static/app.js` - Socket.IO client, real-time UI updates, API calls, Inventory Filtering logic - `static/styles.css` - Responsive design with dark mode support ### Communication Protocol @@ -397,7 +397,7 @@ The application uses a web-based interface accessible via browser: **Supported:** -- ✅ Web GUI - browser-based interface +- ✅ Web GUI - browser-based interface with advanced filtering - ✅ Docker deployment - containerized for any platform - ✅ Remote access - access from any device on network - ✅ Headless operation - no display server required @@ -407,4 +407,4 @@ The application uses a web-based interface accessible via browser: - Multi-account support - Channel points mining - Mining for unlinked campaigns -- Desktop GUI (removed in favor of web-only) +- Desktop GUI diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 0000000..9df9dfa --- /dev/null +++ b/GEMINI.md @@ -0,0 +1,409 @@ +# GEMINI.md + +This file provides guidance to Gemini when working with code in this repository. + +## Project Overview + +Twitch Drops Miner is a Python application that automatically mines timed Twitch drops without downloading stream data. It uses Twitch's GraphQL API and websocket connections to simulate watching streams while tracking drop progress. + +**Key Characteristics:** + +- Python 3.10+ required +- Web-based GUI using FastAPI and Socket.IO +- Async/await architecture with asyncio +- Session persistence via cookies +- No stream video/audio download (bandwidth-efficient) +- Docker-ready for easy deployment + +## Development Commands + +**IMPORTANT: Always activate the virtual environment first!** + +The project uses a virtual environment located at `env/`. All Python commands must be run within this environment: + +```bash +# Activate the virtual environment (required before any Python commands) +source env/bin/activate +``` + +### Running the Application + +```bash +# Run from source (remember to activate venv first!) +source env/bin/activate && python main.py + +# With verbose logging (stackable: -vv, -vvv) +source env/bin/activate && python main.py -v + +# Create data dump for debugging +source env/bin/activate && python main.py --dump + +# Access the web interface at http://localhost:8080 +``` + +### Development Setup + +The application requires: + +- Python 3.10+ +- Virtual environment at `env/` (must be activated before running commands) +- Dependencies from `pyproject.toml` (includes FastAPI, uvicorn, Socket.IO) + +Docker deployment: + +```bash +# Build and run with docker-compose +docker-compose up -d + +# Access at http://localhost:8080 +``` + +## Architecture + + +The application now uses a clean `src/` package structure with clear separation of concerns. + +### Project Structure + +```text +src/ +├── models/ # Domain models (Game, Channel, Campaign, Drop, Benefit) +├── config/ # Configuration (constants, paths, operations, settings, client_info) +├── utils/ # Pure utilities (string, JSON, async helpers, rate_limiter, backoff) +├── i18n/ # Translation system (Translator class, TypedDict schemas) +├── auth/ # Authentication (auth_state for OAuth and token management) +├── api/ # External API (HTTP client, GraphQL client) +├── websocket/ # Real-time updates (websocket connection, pool) +├── web/ # Web GUI (app, gui_manager, api/) +│ └── managers/ # Individual UI managers (status, console, channels, campaigns, inventory, login, settings, cache, broadcaster) +├── services/ # Business logic services (channel, inventory, watch, maintenance, message_handlers) +├── core/ # Core client (Twitch client) +├── exceptions.py # Custom exceptions +├── version.py # Version string +└── __main__.py # Entry point + +lang/ # Translation JSON files (19 languages) +├── English.json # Default/fallback translations +├── Español.json +├── Français.json +├── Deutsch.json +└── ... # 15 more languages +``` + +### Core Components + +**main.py** - Simple launcher: + +- Runs the `src` package as a module using `runpy.run_module("src")` +- All application logic is now in `src/__main__.py` + +**src/__main__.py** - Entry point: + +- Parses command-line arguments +- Initializes Settings, Twitch client, and WebGUIManager +- Starts the FastAPI web server (uvicorn on port 8080) +- Runs the main asyncio event loop +- Handles signals (SIGINT, SIGTERM on Linux) and exit codes + +**src/core/client.py** - Central client (`Twitch` class): + +- State machine: IDLE, INVENTORY_FETCH, GAMES_UPDATE, CHANNELS_CLEANUP, CHANNELS_FETCH, CHANNEL_SWITCH, EXIT +- Composes `_AuthState`, `HTTPClient`, and `GQLClient` +- Delegates to service layer for business logic +- Drop progress monitoring via periodic "watch" payloads +- Manages WebsocketPool and maintenance tasks + +**src/services/** - Business logic layer (fully implemented): + +- `ChannelService`: Channel management and selection logic +- `InventoryService`: Campaign and drop inventory operations +- `WatchService`: Drop mining watch payload logic +- `MaintenanceService`: Periodic maintenance tasks +- `MessageHandlerService`: Websocket message routing and handling + + +**src/models/channel.py** - Channel and Stream: + +- `Channel` class: Twitch channel with online/offline status +- `Stream` class: Active stream with game, viewers, drop status +- Stream URL fetching and validation +- ACL-based vs directory channels + +**src/models/campaign.py** - Drop campaigns: + +- `DropsCampaign`: Campaign with game, timeframe, allowed channels +- Time-based eligibility and progress tracking + +**src/models/drop.py** - Drop types: + +- `TimedDrop`: Drops with minute requirements and progress +- `BaseDrop`: Base class with claim logic +- Precondition chains for sequential drops + +**src/web/gui_manager.py** - Web GUI: + +- `WebGUIManager`: Main GUI coordinator +- Composes individual managers for different UI concerns (status, console, channels, campaigns, inventory, login, settings, cache) +- Uses `WebSocketBroadcaster` for real-time Socket.IO updates +- Pure asyncio, no tkinter dependency + +**src/web/app.py** - FastAPI application: + +- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close` +- Socket.IO server for real-time bi-directional communication +- Serves static web frontend from `web/` directory +- Integrates with WebGUIManager via `set_managers()` + +**src/websocket/pool.py** - WebSocket management: + +- Sharded connections (up to 50 topics per socket, max 199 channels) +- Topics: User.Drops, User.Notifications, Channel.StreamState, Channel.StreamUpdate +- Automatic reconnection with exponential backoff +- Message routing to registered callbacks + +**src/config/settings.py** - Application settings: + +- Games to watch list (auto-populated from available campaigns if empty) +- Connection quality multiplier +- Language selection +- Proxy support (including verification) +- Logging and dump flags from command-line arguments +- Persistence to JSON file (`settings.json`) in DATA_DIR +- Inventory filters (Status, Benefit Type, Game Search) + +### State Machine Flow + +1. **IDLE** - Waiting for campaigns or user action +2. **INVENTORY_FETCH** - Fetch campaigns from GraphQL, claim completed drops +3. **GAMES_UPDATE** - Determine wanted games based on priority/exclude lists +4. **CHANNELS_CLEANUP** - Remove channels not streaming wanted games +5. **CHANNELS_FETCH** - Discover channels via ACL lists or game directories +6. **CHANNEL_SWITCH** - Select best channel to watch based on priority/ACL +7. Loop between CHANNEL_SWITCH and periodic INVENTORY_FETCH (hourly) + +### Authentication + +- Uses OAuth device code flow (user enters code at twitch.tv/activate) +- Managed by `src/auth/auth_state.py` (`_AuthState` class) +- Access tokens stored in `cookies.jar` in DATA_DIR +- Device ID from Twitch's `unique_id` cookie +- Session ID generated per run +- Client info defined in `src/config/client_info.py` (presents as Android app with Client-Id and User-Agent spoofing) + +### Drop Mining Mechanism + +The application sends periodic "watch" payloads to a spade URL every ~20 seconds: + +- Payload contains minute-watched events with channel/broadcast IDs +- Twitch reports progress via websocket (User.Drops topic) +- If websocket updates stop, fallback to GQL CurrentDrop query +- Extrapolation via "bump minutes" when no updates received + +### GraphQL Operations + +Defined in `src/config/operations.py` as `GQL_OPERATIONS`: + +- **Inventory** - Fetch in-progress campaigns and claimed benefits +- **Campaigns** - List available active/upcoming campaigns +- **CampaignDetails** - Detailed drop info for a campaign +- **GameDirectory** - Find live streams for a game with drops enabled +- **GetStreamInfo** - Check if channel is online and get stream details +- **CurrentDrop** - Query currently mined drop progress +- **ClaimDrop** - Claim a completed drop +- **AvailableDrops** - Check which campaigns a channel qualifies for (badge validation) +- **NotificationsDelete** - Delete Twitch notifications + +### Channel Selection Priority + +1. Selected channel (if user clicked one) +2. ACL-based channels over directory channels +3. Game priority order (from settings) +4. Viewer count (descending) +5. Maximum 199 channels tracked simultaneously + +### Maintenance Task + +Runs in background to trigger: + +- Channel cleanup when drops start/end (based on time_triggers) +- Inventory reload every ~60 minutes + +### Translation System + +**Architecture:** + +- All translations stored as JSON files in `lang/` directory (19 languages supported) +- English (`lang/English.json`) is the single source of truth and fallback language +- Strongly typed with TypedDict schema defined in `src/i18n/translator.py` +- Translator class (`src/i18n/translator.py`) handles language loading and fallback +- Singleton instance `_` available via `from src.i18n import _` + +**Supported Languages:** + +- English, Dansk (Danish), Deutsch (German), Español (Spanish), Français (French) +- Indonesian, Italiano (Italian), Nederlandse (Dutch), Polski (Polish), Português (Portuguese) +- Română (Romanian), Türkçe (Turkish), Čeština (Czech) +- Русский (Russian), Українська (Ukrainian), العربية (Arabic) +- 日本語 (Japanese), 简体中文 (Simplified Chinese), 繁體中文 (Traditional Chinese) + +**Translation Structure:** + +```python +Translation = { + "language_name": str, # Display name of language + "english_name": str, # English name of language + "status": StatusMessages, # Console status messages + "login": LoginMessages, # Login-related messages + "error": ErrorMessages, # Error messages + "gui": GUIMessages # All web GUI text (tabs, settings, help, etc.) +} +``` + +**Usage:** + +```python +from src.i18n import _ + +# Access translations +status_text = _.t["gui"]["status"]["idle"] # Returns "Idle" +login_text = _.t["login"]["status"]["logged_in"] # Returns "Logged in" +``` + +**Language Persistence:** + +- Language selection persisted in `settings.json` (DATA_DIR) +- Dynamic language switching supported in web GUI +- Changes take effect immediately without restart + +## Key Files + +- **src/config/constants.py** - Core enums (State, WebsocketTopic), logging config, type aliases +- **src/config/operations.py** - GraphQL operation definitions (GQL_OPERATIONS) +- **src/config/paths.py** - Path management and Docker environment detection +- **src/config/client_info.py** - Twitch client info (Client-Id, User-Agent) +- **src/config/settings.py** - Application settings with JSON persistence +- **src/exceptions.py** - Custom exceptions (MinerException, ExitRequest, RequestException, RequestInvalid, WebsocketClosed, LoginException, CaptchaRequired, GQLException) +- **src/utils/** - Helper utilities (string_utils, json_utils, async_helpers, rate_limiter, backoff) +- **src/i18n/** - Internationalization package with TypedDict schema and Translator class + - **translator.py** - Translator class with typed translation schema (Translation TypedDict) + - **__init__.py** - Exports translation types and `_` (Translator instance) +- **lang/** - Translation JSON files for 19 languages (English.json is the single source of truth) +- **src/version.py** - Version string +- **src/web/app.py** - FastAPI application with REST API and Socket.IO +- **src/web/managers/cache.py** - ImageCache for campaign artwork caching +- **web/** - Frontend assets (index.html, static/app.js, static/styles.css) + +## Testing + +### Automated Tests + +The project includes a test suite in the `tests/` directory: + +```bash +# Activate virtual environment and run tests +source env/bin/activate && python -m pytest tests/ +``` + +**Test Files:** + +- `tests/test_proxy_settings.py` - Tests for proxy settings configuration +- `tests/test_verify_proxy.py` - Tests for proxy verification functionality + +### Manual Testing + +1. Run with `-vvv` for maximum verbosity (levels: -v, -vv, -vvv, -vvvv) +2. Use `--dump` to generate debug data dumps +3. Check log files in `./logs/` directory +4. Use `--debug-ws` for websocket debug logging +5. Use `--debug-gql` for GraphQL debug logging +6. Monitor web GUI console output and browser developer tools + +## Web GUI Architecture + +The application uses a web-based interface accessible via browser: + +### Web GUI Components + +**src/web/gui_manager.py** - WebGUIManager class: + +- Managers: StatusManager, ConsoleOutputManager, ChannelListManager, CampaignProgressManager, InventoryManager, LoginFormManager, SettingsManager, CacheManager +- Uses WebSocketBroadcaster to push real-time updates to connected clients via Socket.IO +- Pure async/await implementation + +**src/web/app.py** - FastAPI application: + +- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/console`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close` +- Socket.IO events for real-time bi-directional communication +- Serves static web frontend from `web/` directory +- Integrates with WebGUIManager via `set_managers(gui, twitch)` + +**web/** - Frontend assets: + +- `index.html` - Single-page application layout with tabs +- `static/app.js` - Socket.IO client, real-time UI updates, API calls, Inventory Filtering logic +- `static/styles.css` - Responsive design with dark mode support + +### Communication Protocol + +**Server → Client (Socket.IO events):** + +- `initial_state` - Full state on connect +- `status_update` - Status bar changes +- `console_output` - New log lines +- `channel_add/update/remove` - Channel list changes +- `drop_progress` - Drop mining progress +- `campaign_add` - New campaign added +- `login_required` - Prompt for credentials +- `settings_updated` - Settings changed + +**Client → Server:** + +- REST API for actions (login, settings, channel selection) +- Socket.IO for connection management + +### Docker Integration + +**src/config/paths.py:** + +- Detects Docker environment via `DOCKER_ENV` env var or `/.dockerenv` file +- Docker: Uses `/app` for code, `/app/data` for persistent storage +- Development: Uses `/data` for persistent storage +- All user data (cookies, settings, cache, logs) stored in DATA_DIR +- Provides `_resource_path()` helper for locating bundled resources + +**Dockerfile:** + +- Based on `python:3` +- Installs dependencies from `pyproject.toml` +- Exposes port 8080 +- Health check on `/api/status` + +**docker-compose.yml:** + +- Volume mounts `./data:/app/data` for persistence +- Port mapping `8080:8080` +- Auto-restart policy +- Timezone configuration + +### Key Design Decisions + +- **WebSocket for real-time** - Socket.IO chosen for reliability (fallback to polling) +- **Single-page app** - Simpler than full framework (React/Vue), fast load times +- **Direct Docker support** - Environment detection, proper path handling +- **OAuth device code flow** - Works great for web-based deployment + +## Project Scope + +**Supported:** + +- ✅ Web GUI - browser-based interface with advanced filtering +- ✅ Docker deployment - containerized for any platform +- ✅ Remote access - access from any device on network +- ✅ Headless operation - no display server required + +**NOT supported:** + +- Multi-account support +- Channel points mining +- Mining for unlinked campaigns +- Desktop GUI