TwitchDropsMiner/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Twitch Drops Miner is a Python application that automatically mines timed Twitch drops without downloading stream data. It uses Twitch's GraphQL API and websocket connections to simulate watching streams while tracking drop progress.

**Key Characteristics:**
- Python 3.10+ required
- Web-based GUI using FastAPI and Socket.IO
- Async/await architecture with asyncio
- Session persistence via cookies
- No stream video/audio download (bandwidth-efficient)
- Docker-ready for easy deployment

## Development Commands

**IMPORTANT: Always activate the virtual environment first!**

The project uses a virtual environment located at `env/`. All Python commands must be run within this environment:

```bash
# Activate the virtual environment (required before any Python commands)
source env/bin/activate
```

### Running the Application

```bash
# Run from source (remember to activate venv first!)
source env/bin/activate && python main.py

# With verbose logging (stackable: -vv, -vvv)
source env/bin/activate && python main.py -v

# Enable legacy log file (logs always go to ./logs/TDM.TIMESTAMP.log)
source env/bin/activate && python main.py --log

# Create data dump for debugging
source env/bin/activate && python main.py --dump

# Access the web interface at http://localhost:8080
```

### Development Setup

The application requires:
- Python 3.10+
- Virtual environment at `env/` (must be activated before running commands)
- Dependencies from `requirements.txt` (includes FastAPI, uvicorn, Socket.IO)

Docker deployment:
```bash
# Build and run with docker-compose
docker-compose up -d

# Access at http://localhost:8080
```

## Architecture

**⚠️ IMPORTANT: The codebase has been refactored into a modular structure.**

The application now uses a clean `src/` package structure with clear separation of concerns. See [REFACTORING_SUMMARY.md](REFACTORING_SUMMARY.md) for complete details.

### Project Structure

```
src/
├── models/          # Domain models (Game, Channel, Campaign, Drop, Benefit)
├── config/          # Configuration (constants, paths, operations, settings, client_info)
├── utils/           # Pure utilities (time, string, JSON, async helpers, cache, rate_limiter, backoff, registry)
├── i18n/            # Translation system (translator)
├── auth/            # Authentication (auth_state for OAuth and token management)
├── api/             # External API (HTTP client, GraphQL client)
├── websocket/       # Real-time updates (websocket connection, pool)
├── web/             # Web GUI (app, gui_manager, api/)
│   └── managers/    # Individual UI managers (status, console, channels, campaigns, inventory, login, settings, cache, tray, broadcaster)
├── services/        # Business logic services (channel, inventory, watch, maintenance, message_handlers)
├── core/            # Core client (Twitch client)
├── exceptions.py    # Custom exceptions
├── version.py       # Version string
└── __main__.py      # Entry point (replaces old main.py)
```

### Core Components

**main.py** - Simple launcher:
- Runs the `src` package as a module using `runpy.run_module("src")`
- All application logic is now in `src/__main__.py`

**src/__main__.py** - Entry point:
- Parses command-line arguments
- Initializes Settings, Twitch client, and WebGUIManager
- Starts the FastAPI web server (uvicorn on port 8080)
- Runs the main asyncio event loop
- Handles signals (SIGINT, SIGTERM on Linux) and exit codes

**src/core/client.py** - Central client (`Twitch` class):
- State machine: IDLE, INVENTORY_FETCH, GAMES_UPDATE, CHANNELS_CLEANUP, CHANNELS_FETCH, CHANNEL_SWITCH, EXIT
- Composes `_AuthState`, `HTTPClient`, and `GQLClient`
- Delegates to service layer for business logic
- Drop progress monitoring via periodic "watch" payloads
- Manages WebsocketPool and maintenance tasks

**src/services/** - Business logic layer (fully implemented):
- `ChannelService`: Channel management and selection logic
- `InventoryService`: Campaign and drop inventory operations
- `WatchService`: Drop mining watch payload logic
- `MaintenanceService`: Periodic maintenance tasks
- `MessageHandlerService`: Websocket message routing and handling

Note: The services layer is complete and handles all business logic that was previously in the monolithic client file.

**src/models/channel.py** - Channel and Stream:
- `Channel` class: Twitch channel with online/offline status
- `Stream` class: Active stream with game, viewers, drop status
- Stream URL fetching and validation
- ACL-based vs directory channels

**src/models/campaign.py** - Drop campaigns:
- `DropsCampaign`: Campaign with game, timeframe, allowed channels
- Time-based eligibility and progress tracking

**src/models/drop.py** - Drop types:
- `TimedDrop`: Drops with minute requirements and progress
- `BaseDrop`: Base class with claim logic
- Precondition chains for sequential drops

**src/web/gui_manager.py** - Web GUI:
- `WebGUIManager`: Main GUI coordinator
- Composes individual managers for different UI concerns (status, console, channels, campaigns, inventory, login, settings, cache, tray)
- Uses `WebSocketBroadcaster` for real-time Socket.IO updates
- Pure asyncio, no tkinter dependency

**src/web/app.py** - FastAPI application:
- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close`
- Socket.IO server for real-time bi-directional communication
- Serves static web frontend from `web/` directory
- Integrates with WebGUIManager via `set_managers()`

**src/websocket/pool.py** - WebSocket management:
- Sharded connections (up to 50 topics per socket, max 199 channels)
- Topics: User.Drops, User.Notifications, Channel.StreamState, Channel.StreamUpdate
- Automatic reconnection with exponential backoff
- Message routing to registered callbacks

**src/config/settings.py** - Application settings:
- Games to watch list (auto-populated from available campaigns if empty)
- Connection quality multiplier
- Language selection
- Proxy support
- Logging and dump flags from command-line arguments
- Persistence to JSON file (`settings.json`) in DATA_DIR

### State Machine Flow

1. **IDLE** - Waiting for campaigns or user action
2. **INVENTORY_FETCH** - Fetch campaigns from GraphQL, claim completed drops
3. **GAMES_UPDATE** - Determine wanted games based on priority/exclude lists
4. **CHANNELS_CLEANUP** - Remove channels not streaming wanted games
5. **CHANNELS_FETCH** - Discover channels via ACL lists or game directories
6. **CHANNEL_SWITCH** - Select best channel to watch based on priority/ACL
7. Loop between CHANNEL_SWITCH and periodic INVENTORY_FETCH (hourly)

### Authentication

- Uses OAuth device code flow (user enters code at twitch.tv/activate)
- Managed by `src/auth/auth_state.py` (`_AuthState` class)
- Access tokens stored in `cookies.jar` in DATA_DIR
- Device ID from Twitch's `unique_id` cookie
- Session ID generated per run
- Client info defined in `src/config/client_info.py` (presents as Android app with Client-Id and User-Agent spoofing)

### Drop Mining Mechanism

The application sends periodic "watch" payloads to a spade URL every ~20 seconds:
- Payload contains minute-watched events with channel/broadcast IDs
- Twitch reports progress via websocket (User.Drops topic)
- If websocket updates stop, fallback to GQL CurrentDrop query
- Extrapolation via "bump minutes" when no updates received

### GraphQL Operations

Defined in `src/config/operations.py` as `GQL_OPERATIONS`:
- **Inventory** - Fetch in-progress campaigns and claimed benefits
- **Campaigns** - List available active/upcoming campaigns
- **CampaignDetails** - Detailed drop info for a campaign
- **GameDirectory** - Find live streams for a game with drops enabled
- **GetStreamInfo** - Check if channel is online and get stream details
- **CurrentDrop** - Query currently mined drop progress
- **ClaimDrop** - Claim a completed drop
- **AvailableDrops** - Check which campaigns a channel qualifies for (badge validation)
- **NotificationsDelete** - Delete Twitch notifications

### Channel Selection Priority

1. Selected channel (if user clicked one)
2. ACL-based channels over directory channels
3. Game priority order (from settings)
4. Viewer count (descending)
5. Maximum 199 channels tracked simultaneously

### Maintenance Task

Runs in background to trigger:
- Channel cleanup when drops start/end (based on time_triggers)
- Inventory reload every ~60 minutes

## Key Files

- **src/config/constants.py** - Core enums (State, WebsocketTopic), logging config, type aliases
- **src/config/operations.py** - GraphQL operation definitions (GQL_OPERATIONS)
- **src/config/paths.py** - Path management and Docker environment detection
- **src/config/client_info.py** - Twitch client info (Client-Id, User-Agent)
- **src/config/settings.py** - Application settings with JSON persistence
- **src/exceptions.py** - Custom exceptions (LoginException, CaptchaRequired, ExitRequest, ReloadRequest, etc.)
- **src/utils/** - Helper utilities (time, string, JSON, async helpers, cache, rate limiter, backoff, registry)
- **src/i18n/translator.py** - i18n support with JSON translation files
- **src/version.py** - Version string
- **src/web/app.py** - FastAPI application with REST API and Socket.IO
- **web/** - Frontend assets (index.html, static/app.js, static/styles.css)

## Testing

The project does not include a test suite. Manual testing workflow:
1. Run with `-vvv` for maximum verbosity (levels: -v, -vv, -vvv, -vvvv)
2. Use `--dump` to generate debug data dumps
3. Check timestamped log files in `./logs/` directory (always created as `TDM.TIMESTAMP.log`)
4. Use `--log` flag to also create legacy `log.txt` file
5. Use `--debug-ws` for websocket debug logging
6. Use `--debug-gql` for GraphQL debug logging
7. Monitor web GUI console output and browser developer tools

## Web GUI Architecture

The application uses a web-based interface accessible via browser:

### Web GUI Components

**src/web/gui_manager.py** - WebGUIManager class:
- Managers: StatusManager, ConsoleOutputManager, ChannelListManager, CampaignProgressManager, InventoryManager, LoginFormManager, SettingsManager, CacheManager, TrayManager
- Uses WebSocketBroadcaster to push real-time updates to connected clients via Socket.IO
- Pure async/await implementation

**src/web/app.py** - FastAPI application:
- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/console`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close`
- Socket.IO events for real-time bi-directional communication
- Serves static web frontend from `web/` directory
- Integrates with WebGUIManager via `set_managers(gui, twitch)`

**web/** - Frontend assets:
- `index.html` - Single-page application layout with tabs
- `static/app.js` - Socket.IO client, real-time UI updates, API calls
- `static/styles.css` - Responsive design with dark mode support

### Communication Protocol

**Server → Client (Socket.IO events):**
- `initial_state` - Full state on connect
- `status_update` - Status bar changes
- `console_output` - New log lines
- `channel_add/update/remove` - Channel list changes
- `drop_progress` - Drop mining progress
- `campaign_add` - New campaign added
- `login_required` - Prompt for credentials
- `settings_updated` - Settings changed

**Client → Server:**
- REST API for actions (login, settings, channel selection)
- Socket.IO for connection management

### Docker Integration

**src/config/paths.py:**
- Detects Docker environment via `DOCKER_ENV` env var or `/.dockerenv` file
- Uses `/app` for code, `/app/data` for persistent storage
- All user data (cookies, settings, cache, logs) stored in DATA_DIR
- Provides `_resource_path()` helper for locating resources

**Dockerfile:**
- Based on `python:3.11-slim`
- Installs dependencies from `requirements.txt`
- Exposes port 8080
- Health check on `/api/status`

**docker-compose.yml:**
- Volume mounts `./data:/app/data` for persistence
- Port mapping `8080:8080`
- Auto-restart policy
- Timezone configuration

### Key Design Decisions

- **WebSocket for real-time** - Socket.IO chosen for reliability (fallback to polling)
- **Single-page app** - Simpler than full framework (React/Vue), fast load times
- **Direct Docker support** - Environment detection, proper path handling
- **OAuth device code flow** - Works great for web-based deployment

## Project Scope

**Supported:**
- ✅ Web GUI - browser-based interface
- ✅ Docker deployment - containerized for any platform
- ✅ Remote access - access from any device on network
- ✅ Headless operation - no display server required

**NOT supported:**
- Multi-account support
- Channel points mining
- Mining for unlinked campaigns
- Desktop GUI (removed in favor of web-only)