TwitchDropsMiner/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Twitch Drops Miner is a Python application that automatically mines timed Twitch drops without downloading stream data. It uses Twitch's GraphQL API and websocket connections to simulate watching streams while tracking drop progress.

Key Characteristics:

• Python 3.10+ required
• Web-based GUI using FastAPI and Socket.IO
• Async/await architecture with asyncio
• Session persistence via cookies
• No stream video/audio download (bandwidth-efficient)
• Docker-ready for easy deployment

Development Commands

IMPORTANT: Always activate the virtual environment first!

The project uses a virtual environment located at env/. All Python commands must be run within this environment:

# Activate the virtual environment (required before any Python commands)
source env/bin/activate

Running the Application

# Run from source (remember to activate venv first!)
source env/bin/activate && python main.py

# With verbose logging (stackable: -vv, -vvv)
source env/bin/activate && python main.py -v

# Create data dump for debugging
source env/bin/activate && python main.py --dump

# Access the web interface at http://localhost:8080

Development Setup

The application requires:

• Python 3.10+
• Virtual environment at env/ (must be activated before running commands)
• Dependencies from requirements.txt (includes FastAPI, uvicorn, Socket.IO)

Docker deployment:

# Build and run with docker-compose
docker-compose up -d

# Access at http://localhost:8080

Architecture

⚠️ IMPORTANT: The codebase has been refactored into a modular structure.

The application now uses a clean src/ package structure with clear separation of concerns. See REFACTORING_SUMMARY.md for complete details.

Project Structure

src/
├── models/          # Domain models (Game, Channel, Campaign, Drop, Benefit)
├── config/          # Configuration (constants, paths, operations, settings, client_info)
├── utils/           # Pure utilities (time, string, JSON, async helpers, cache, rate_limiter, backoff, registry)
├── i18n/            # Translation system (translator)
├── auth/            # Authentication (auth_state for OAuth and token management)
├── api/             # External API (HTTP client, GraphQL client)
├── websocket/       # Real-time updates (websocket connection, pool)
├── web/             # Web GUI (app, gui_manager, api/)
│   └── managers/    # Individual UI managers (status, console, channels, campaigns, inventory, login, settings, cache, tray, broadcaster)
├── services/        # Business logic services (channel, inventory, watch, maintenance, message_handlers)
├── core/            # Core client (Twitch client)
├── exceptions.py    # Custom exceptions
├── version.py       # Version string
└── __main__.py      # Entry point (replaces old main.py)

Core Components

main.py - Simple launcher:

• Runs the src package as a module using runpy.run_module("src")
• All application logic is now in src/__main__.py

src/main.py - Entry point:

• Parses command-line arguments
• Initializes Settings, Twitch client, and WebGUIManager
• Starts the FastAPI web server (uvicorn on port 8080)
• Runs the main asyncio event loop
• Handles signals (SIGINT, SIGTERM on Linux) and exit codes

src/core/client.py - Central client (Twitch class):

• State machine: IDLE, INVENTORY_FETCH, GAMES_UPDATE, CHANNELS_CLEANUP, CHANNELS_FETCH, CHANNEL_SWITCH, EXIT
• Composes _AuthState, HTTPClient, and GQLClient
• Delegates to service layer for business logic
• Drop progress monitoring via periodic "watch" payloads
• Manages WebsocketPool and maintenance tasks

src/services/ - Business logic layer (fully implemented):

• ChannelService: Channel management and selection logic
• InventoryService: Campaign and drop inventory operations
• WatchService: Drop mining watch payload logic
• MaintenanceService: Periodic maintenance tasks
• MessageHandlerService: Websocket message routing and handling

Note: The services layer is complete and handles all business logic that was previously in the monolithic client file.

src/models/channel.py - Channel and Stream:

• Channel class: Twitch channel with online/offline status
• Stream class: Active stream with game, viewers, drop status
• Stream URL fetching and validation
• ACL-based vs directory channels

src/models/campaign.py - Drop campaigns:

• DropsCampaign: Campaign with game, timeframe, allowed channels
• Time-based eligibility and progress tracking

src/models/drop.py - Drop types:

• TimedDrop: Drops with minute requirements and progress
• BaseDrop: Base class with claim logic
• Precondition chains for sequential drops

src/web/gui_manager.py - Web GUI:

• WebGUIManager: Main GUI coordinator
• Composes individual managers for different UI concerns (status, console, channels, campaigns, inventory, login, settings, cache, tray)
• Uses WebSocketBroadcaster for real-time Socket.IO updates
• Pure asyncio, no tkinter dependency

src/web/app.py - FastAPI application:

• REST API endpoints: /api/status, /api/channels, /api/campaigns, /api/settings, /api/login, /api/oauth/confirm, /api/reload, /api/close
• Socket.IO server for real-time bi-directional communication
• Serves static web frontend from web/ directory
• Integrates with WebGUIManager via set_managers()

src/websocket/pool.py - WebSocket management:

• Sharded connections (up to 50 topics per socket, max 199 channels)
• Topics: User.Drops, User.Notifications, Channel.StreamState, Channel.StreamUpdate
• Automatic reconnection with exponential backoff
• Message routing to registered callbacks

src/config/settings.py - Application settings:

• Games to watch list (auto-populated from available campaigns if empty)
• Connection quality multiplier
• Language selection
• Proxy support
• Logging and dump flags from command-line arguments
• Persistence to JSON file (settings.json) in DATA_DIR

State Machine Flow

1. IDLE - Waiting for campaigns or user action
2. INVENTORY_FETCH - Fetch campaigns from GraphQL, claim completed drops
3. GAMES_UPDATE - Determine wanted games based on priority/exclude lists
4. CHANNELS_CLEANUP - Remove channels not streaming wanted games
5. CHANNELS_FETCH - Discover channels via ACL lists or game directories
6. CHANNEL_SWITCH - Select best channel to watch based on priority/ACL
7. Loop between CHANNEL_SWITCH and periodic INVENTORY_FETCH (hourly)

Authentication

• Uses OAuth device code flow (user enters code at twitch.tv/activate)
• Managed by src/auth/auth_state.py (_AuthState class)
• Access tokens stored in cookies.jar in DATA_DIR
• Device ID from Twitch's unique_id cookie
• Session ID generated per run
• Client info defined in src/config/client_info.py (presents as Android app with Client-Id and User-Agent spoofing)

Drop Mining Mechanism

The application sends periodic "watch" payloads to a spade URL every ~20 seconds:

• Payload contains minute-watched events with channel/broadcast IDs
• Twitch reports progress via websocket (User.Drops topic)
• If websocket updates stop, fallback to GQL CurrentDrop query
• Extrapolation via "bump minutes" when no updates received

GraphQL Operations

Defined in src/config/operations.py as GQL_OPERATIONS:

• Inventory - Fetch in-progress campaigns and claimed benefits
• Campaigns - List available active/upcoming campaigns
• CampaignDetails - Detailed drop info for a campaign
• GameDirectory - Find live streams for a game with drops enabled
• GetStreamInfo - Check if channel is online and get stream details
• CurrentDrop - Query currently mined drop progress
• ClaimDrop - Claim a completed drop
• AvailableDrops - Check which campaigns a channel qualifies for (badge validation)
• NotificationsDelete - Delete Twitch notifications

Channel Selection Priority

1. Selected channel (if user clicked one)
2. ACL-based channels over directory channels
3. Game priority order (from settings)
4. Viewer count (descending)
5. Maximum 199 channels tracked simultaneously

Maintenance Task

Runs in background to trigger:

• Channel cleanup when drops start/end (based on time_triggers)
• Inventory reload every ~60 minutes

Key Files

• src/config/constants.py - Core enums (State, WebsocketTopic), logging config, type aliases
• src/config/operations.py - GraphQL operation definitions (GQL_OPERATIONS)
• src/config/paths.py - Path management and Docker environment detection
• src/config/client_info.py - Twitch client info (Client-Id, User-Agent)
• src/config/settings.py - Application settings with JSON persistence
• src/exceptions.py - Custom exceptions (LoginException, CaptchaRequired, ExitRequest, ReloadRequest, etc.)
• src/utils/ - Helper utilities (time, string, JSON, async helpers, cache, rate limiter, backoff, registry)
• src/i18n/translator.py - i18n support with JSON translation files
• src/version.py - Version string
• src/web/app.py - FastAPI application with REST API and Socket.IO
• web/ - Frontend assets (index.html, static/app.js, static/styles.css)

Testing

The project does not include a test suite. Manual testing workflow:

1. Run with -vvv for maximum verbosity (levels: -v, -vv, -vvv, -vvvv)
2. Use --dump to generate debug data dumps
3. Check timestamped log files in ./logs/ directory (always created as TDM.TIMESTAMP.log)
4. Use --debug-ws for websocket debug logging
5. Use --debug-gql for GraphQL debug logging
6. Monitor web GUI console output and browser developer tools

Web GUI Architecture

The application uses a web-based interface accessible via browser:

Web GUI Components

src/web/gui_manager.py - WebGUIManager class:

• Managers: StatusManager, ConsoleOutputManager, ChannelListManager, CampaignProgressManager, InventoryManager, LoginFormManager, SettingsManager, CacheManager, TrayManager
• Uses WebSocketBroadcaster to push real-time updates to connected clients via Socket.IO
• Pure async/await implementation

src/web/app.py - FastAPI application:

• REST API endpoints: /api/status, /api/channels, /api/campaigns, /api/console, /api/settings, /api/login, /api/oauth/confirm, /api/reload, /api/close
• Socket.IO events for real-time bi-directional communication
• Serves static web frontend from web/ directory
• Integrates with WebGUIManager via set_managers(gui, twitch)

web/ - Frontend assets:

• index.html - Single-page application layout with tabs
• static/app.js - Socket.IO client, real-time UI updates, API calls
• static/styles.css - Responsive design with dark mode support

Communication Protocol

Server → Client (Socket.IO events):

• initial_state - Full state on connect
• status_update - Status bar changes
• console_output - New log lines
• channel_add/update/remove - Channel list changes
• drop_progress - Drop mining progress
• campaign_add - New campaign added
• login_required - Prompt for credentials
• settings_updated - Settings changed

Client → Server:

• REST API for actions (login, settings, channel selection)
• Socket.IO for connection management

Docker Integration

src/config/paths.py:

• Detects Docker environment via DOCKER_ENV env var or /.dockerenv file
• Uses /app for code, /app/data for persistent storage
• All user data (cookies, settings, cache, logs) stored in DATA_DIR
• Provides _resource_path() helper for locating resources

Dockerfile:

• Based on python:3.11-slim
• Installs dependencies from requirements.txt
• Exposes port 8080
• Health check on /api/status

docker-compose.yml:

• Volume mounts ./data:/app/data for persistence
• Port mapping 8080:8080
• Auto-restart policy
• Timezone configuration

Key Design Decisions

• WebSocket for real-time - Socket.IO chosen for reliability (fallback to polling)
• Single-page app - Simpler than full framework (React/Vue), fast load times
• Direct Docker support - Environment detection, proper path handling
• OAuth device code flow - Works great for web-based deployment

Project Scope

Supported:

• ✅ Web GUI - browser-based interface
• ✅ Docker deployment - containerized for any platform
• ✅ Remote access - access from any device on network
• ✅ Headless operation - no display server required

NOT supported:

• Multi-account support
• Channel points mining
• Mining for unlinked campaigns
• Desktop GUI (removed in favor of web-only)