GitHub/TwitchDropsMiner
1
0
Fork 0
mirror of https://github.com/rangermix/TwitchDropsMiner.git synced 2026-05-26 07:08:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Migrate to Docker-ready web UI and remove legacy desktop GUI

Browse Source 
Replace the legacy desktop/Tkinter client and packaging artifacts with a Docker-first, web-hosted approach.

- Add Docker quickstart and run-from-source instructions to the README to simplify deployment.
- Simplify launcher to invoke the new web backend module instead of the old desktop entrypoint.
- Update dependencies for a web UI stack (FastAPI, Uvicorn, python-socketio, Jinja2, etc.) and remove desktop/tray-specific packages.
- Remove legacy GUI, packaging and platform-specific helper code, along with obsolete build/pack scripts and AppImage assets to declutter the repo.
- Tidy project ignore rules to add runtime logs and editor metadata.

Rationale: streamline deployment, favor a browser-accessible interface, and reduce maintenance overhead from multiple platform-specific GUI/packaging implementations.
This commit is contained in:
Fengqing Liu
2025-10-16 21:54:43 +11:00
parent 2388757ebe
commit 5b736e3bb1
77 changed files with 7852 additions and 5401 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
.dockerignore Normal file
View File

@@ -0,0 +1,62 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
*.egg-info/
dist/
build/
*.egg
# Virtual Environment
env/
venv/
ENV/
# IDEs
.vscode/
.idea/
*.swp
*.swo
*~
# Git
.git/
.gitignore
.gitattributes
# GitHub
.github/
# CI/CD
.gitlab-ci.yml
# Documentation
*.md
!README.md
# OS
.DS_Store
Thumbs.db
# Application data (will be mounted as volumes)
data/
cache/
cookies.jar
settings.json
log.txt
dump.dat
lock.file
logs/
.claude/
# Build artifacts
*.exe
*.AppImage
*.deb
*.rpm
# Other
.env
.env.local

9
.gitignore vendored
View File

@@ -17,9 +17,6 @@ log.txt
/lock.file /lock.file
settings.json settings.json
/lang/English.json /lang/English.json
# AppImage
/AppDir logs/
/appimage-builder .claude/
/appimage-build
/*.AppImage
/*.AppImage.zsync

313
CLAUDE.md Normal file
View File

@@ -0,0 +1,313 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Twitch Drops Miner is a Python application that automatically mines timed Twitch drops without downloading stream data. It uses Twitch's GraphQL API and websocket connections to simulate watching streams while tracking drop progress.
**Key Characteristics:**
- Python 3.10+ required
- Web-based GUI using FastAPI and Socket.IO
- Async/await architecture with asyncio
- Session persistence via cookies
- No stream video/audio download (bandwidth-efficient)
- Docker-ready for easy deployment
## Development Commands
**IMPORTANT: Always activate the virtual environment first!**
The project uses a virtual environment located at `env/`. All Python commands must be run within this environment:
```bash
# Activate the virtual environment (required before any Python commands)
source env/bin/activate
```
### Running the Application
```bash
# Run from source (remember to activate venv first!)
source env/bin/activate && python main.py
# With verbose logging (stackable: -vv, -vvv)
source env/bin/activate && python main.py -v
# Enable legacy log file (logs always go to ./logs/TDM.TIMESTAMP.log)
source env/bin/activate && python main.py --log
# Create data dump for debugging
source env/bin/activate && python main.py --dump
# Access the web interface at http://localhost:8080
```
### Development Setup
The application requires:
- Python 3.10+
- Virtual environment at `env/` (must be activated before running commands)
- Dependencies from `requirements.txt` (includes FastAPI, uvicorn, Socket.IO)
Docker deployment:
```bash
# Build and run with docker-compose
docker-compose up -d
# Access at http://localhost:8080
```
## Architecture
**⚠️ IMPORTANT: The codebase has been refactored into a modular structure.**
The application now uses a clean `src/` package structure with clear separation of concerns. See [REFACTORING_SUMMARY.md](REFACTORING_SUMMARY.md) for complete details.
### Project Structure
```
src/
├── models/ # Domain models (Game, Channel, Campaign, Drop, Benefit)
├── config/ # Configuration (constants, paths, operations, settings, client_info)
├── utils/ # Pure utilities (time, string, JSON, async helpers, cache, rate_limiter, backoff, registry)
├── i18n/ # Translation system (translator)
├── auth/ # Authentication (auth_state for OAuth and token management)
├── api/ # External API (HTTP client, GraphQL client)
├── websocket/ # Real-time updates (websocket connection, pool)
├── web/ # Web GUI (app, gui_manager, api/)
│ └── managers/ # Individual UI managers (status, console, channels, campaigns, inventory, login, settings, cache, tray, broadcaster)
├── services/ # Business logic services (channel, inventory, watch, maintenance, message_handlers)
├── core/ # Core client (Twitch client)
├── exceptions.py # Custom exceptions
├── version.py # Version string
└── __main__.py # Entry point (replaces old main.py)
```
### Core Components
**main.py** - Simple launcher:
- Runs the `src` package as a module using `runpy.run_module("src")`
- All application logic is now in `src/__main__.py`
**src/__main__.py** - Entry point:
- Parses command-line arguments
- Initializes Settings, Twitch client, and WebGUIManager
- Starts the FastAPI web server (uvicorn on port 8080)
- Runs the main asyncio event loop
- Handles signals (SIGINT, SIGTERM on Linux) and exit codes
**src/core/client.py** - Central client (`Twitch` class):
- State machine: IDLE, INVENTORY_FETCH, GAMES_UPDATE, CHANNELS_CLEANUP, CHANNELS_FETCH, CHANNEL_SWITCH, EXIT
- Composes `_AuthState`, `HTTPClient`, and `GQLClient`
- Delegates to service layer for business logic
- Drop progress monitoring via periodic "watch" payloads
- Manages WebsocketPool and maintenance tasks
**src/services/** - Business logic layer (fully implemented):
- `ChannelService`: Channel management and selection logic
- `InventoryService`: Campaign and drop inventory operations
- `WatchService`: Drop mining watch payload logic
- `MaintenanceService`: Periodic maintenance tasks
- `MessageHandlerService`: Websocket message routing and handling
Note: The services layer is complete and handles all business logic that was previously in the monolithic client file.
**src/models/channel.py** - Channel and Stream:
- `Channel` class: Twitch channel with online/offline status
- `Stream` class: Active stream with game, viewers, drop status
- Stream URL fetching and validation
- ACL-based vs directory channels
**src/models/campaign.py** - Drop campaigns:
- `DropsCampaign`: Campaign with game, timeframe, allowed channels
- Time-based eligibility and progress tracking
**src/models/drop.py** - Drop types:
- `TimedDrop`: Drops with minute requirements and progress
- `BaseDrop`: Base class with claim logic
- Precondition chains for sequential drops
**src/web/gui_manager.py** - Web GUI:
- `WebGUIManager`: Main GUI coordinator
- Composes individual managers for different UI concerns (status, console, channels, campaigns, inventory, login, settings, cache, tray)
- Uses `WebSocketBroadcaster` for real-time Socket.IO updates
- Pure asyncio, no tkinter dependency
**src/web/app.py** - FastAPI application:
- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close`
- Socket.IO server for real-time bi-directional communication
- Serves static web frontend from `web/` directory
- Integrates with WebGUIManager via `set_managers()`
**src/websocket/pool.py** - WebSocket management:
- Sharded connections (up to 50 topics per socket, max 199 channels)
- Topics: User.Drops, User.Notifications, Channel.StreamState, Channel.StreamUpdate
- Automatic reconnection with exponential backoff
- Message routing to registered callbacks
**src/config/settings.py** - Application settings:
- Games to watch list (auto-populated from available campaigns if empty)
- Connection quality multiplier
- Language selection
- Proxy support
- Logging and dump flags from command-line arguments
- Persistence to JSON file (`settings.json`) in DATA_DIR
### State Machine Flow
1. **IDLE** - Waiting for campaigns or user action
2. **INVENTORY_FETCH** - Fetch campaigns from GraphQL, claim completed drops
3. **GAMES_UPDATE** - Determine wanted games based on priority/exclude lists
4. **CHANNELS_CLEANUP** - Remove channels not streaming wanted games
5. **CHANNELS_FETCH** - Discover channels via ACL lists or game directories
6. **CHANNEL_SWITCH** - Select best channel to watch based on priority/ACL
7. Loop between CHANNEL_SWITCH and periodic INVENTORY_FETCH (hourly)
### Authentication
- Uses OAuth device code flow (user enters code at twitch.tv/activate)
- Managed by `src/auth/auth_state.py` (`_AuthState` class)
- Access tokens stored in `cookies.jar` in DATA_DIR
- Device ID from Twitch's `unique_id` cookie
- Session ID generated per run
- Client info defined in `src/config/client_info.py` (presents as Android app with Client-Id and User-Agent spoofing)
### Drop Mining Mechanism
The application sends periodic "watch" payloads to a spade URL every ~20 seconds:
- Payload contains minute-watched events with channel/broadcast IDs
- Twitch reports progress via websocket (User.Drops topic)
- If websocket updates stop, fallback to GQL CurrentDrop query
- Extrapolation via "bump minutes" when no updates received
### GraphQL Operations
Defined in `src/config/operations.py` as `GQL_OPERATIONS`:
- **Inventory** - Fetch in-progress campaigns and claimed benefits
- **Campaigns** - List available active/upcoming campaigns
- **CampaignDetails** - Detailed drop info for a campaign
- **GameDirectory** - Find live streams for a game with drops enabled
- **GetStreamInfo** - Check if channel is online and get stream details
- **CurrentDrop** - Query currently mined drop progress
- **ClaimDrop** - Claim a completed drop
- **AvailableDrops** - Check which campaigns a channel qualifies for (badge validation)
- **NotificationsDelete** - Delete Twitch notifications
### Channel Selection Priority
1. Selected channel (if user clicked one)
2. ACL-based channels over directory channels
3. Game priority order (from settings)
4. Viewer count (descending)
5. Maximum 199 channels tracked simultaneously
### Maintenance Task
Runs in background to trigger:
- Channel cleanup when drops start/end (based on time_triggers)
- Inventory reload every ~60 minutes
## Key Files
- **src/config/constants.py** - Core enums (State, WebsocketTopic), logging config, type aliases
- **src/config/operations.py** - GraphQL operation definitions (GQL_OPERATIONS)
- **src/config/paths.py** - Path management and Docker environment detection
- **src/config/client_info.py** - Twitch client info (Client-Id, User-Agent)
- **src/config/settings.py** - Application settings with JSON persistence
- **src/exceptions.py** - Custom exceptions (LoginException, CaptchaRequired, ExitRequest, ReloadRequest, etc.)
- **src/utils/** - Helper utilities (time, string, JSON, async helpers, cache, rate limiter, backoff, registry)
- **src/i18n/translator.py** - i18n support with JSON translation files
- **src/version.py** - Version string
- **src/web/app.py** - FastAPI application with REST API and Socket.IO
- **web/** - Frontend assets (index.html, static/app.js, static/styles.css)
## Testing
The project does not include a test suite. Manual testing workflow:
1. Run with `-vvv` for maximum verbosity (levels: -v, -vv, -vvv, -vvvv)
2. Use `--dump` to generate debug data dumps
3. Check timestamped log files in `./logs/` directory (always created as `TDM.TIMESTAMP.log`)
4. Use `--log` flag to also create legacy `log.txt` file
5. Use `--debug-ws` for websocket debug logging
6. Use `--debug-gql` for GraphQL debug logging
7. Monitor web GUI console output and browser developer tools
## Web GUI Architecture
The application uses a web-based interface accessible via browser:
### Web GUI Components
**src/web/gui_manager.py** - WebGUIManager class:
- Managers: StatusManager, ConsoleOutputManager, ChannelListManager, CampaignProgressManager, InventoryManager, LoginFormManager, SettingsManager, CacheManager, TrayManager
- Uses WebSocketBroadcaster to push real-time updates to connected clients via Socket.IO
- Pure async/await implementation
**src/web/app.py** - FastAPI application:
- REST API endpoints: `/api/status`, `/api/channels`, `/api/campaigns`, `/api/console`, `/api/settings`, `/api/login`, `/api/oauth/confirm`, `/api/reload`, `/api/close`
- Socket.IO events for real-time bi-directional communication
- Serves static web frontend from `web/` directory
- Integrates with WebGUIManager via `set_managers(gui, twitch)`
**web/** - Frontend assets:
- `index.html` - Single-page application layout with tabs
- `static/app.js` - Socket.IO client, real-time UI updates, API calls
- `static/styles.css` - Responsive design with dark mode support
### Communication Protocol
**Server → Client (Socket.IO events):**
- `initial_state` - Full state on connect
- `status_update` - Status bar changes
- `console_output` - New log lines
- `channel_add/update/remove` - Channel list changes
- `drop_progress` - Drop mining progress
- `campaign_add` - New campaign added
- `login_required` - Prompt for credentials
- `settings_updated` - Settings changed
**Client → Server:**
- REST API for actions (login, settings, channel selection)
- Socket.IO for connection management
### Docker Integration
**src/config/paths.py:**
- Detects Docker environment via `DOCKER_ENV` env var or `/.dockerenv` file
- Uses `/app` for code, `/app/data` for persistent storage
- All user data (cookies, settings, cache, logs) stored in DATA_DIR
- Provides `_resource_path()` helper for locating resources
**Dockerfile:**
- Based on `python:3.11-slim`
- Installs dependencies from `requirements.txt`
- Exposes port 8080
- Health check on `/api/status`
**docker-compose.yml:**
- Volume mounts `./data:/app/data` for persistence
- Port mapping `8080:8080`
- Auto-restart policy
- Timezone configuration
### Key Design Decisions
- **WebSocket for real-time** - Socket.IO chosen for reliability (fallback to polling)
- **Single-page app** - Simpler than full framework (React/Vue), fast load times
- **Direct Docker support** - Environment detection, proper path handling
- **OAuth device code flow** - Works great for web-based deployment
## Project Scope
**Supported:**
- ✅ Web GUI - browser-based interface
- ✅ Docker deployment - containerized for any platform
- ✅ Remote access - access from any device on network
- ✅ Headless operation - no display server required
**NOT supported:**
- Multi-account support
- Channel points mining
- Mining for unlinked campaigns
- Desktop GUI (removed in favor of web-only)

40
Dockerfile Normal file
View File

@@ -0,0 +1,40 @@
FROM python:3.11-slim
# Set environment variables
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
DOCKER_ENV=1 \
PORT=8080
# Set working directory
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements file
COPY requirements-web.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements-web.txt
# Copy application code
COPY *.py ./
COPY lang/ ./lang/
COPY icons/ ./icons/
COPY web/ ./web/
# Create data directory for persistent storage
RUN mkdir -p /app/data && chmod 777 /app/data
# Expose web port
EXPOSE 8080
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/api/status')" || exit 1
# Run with web GUI
CMD ["python", "main.py", "--web", "-v"]

135
README.md
View File

@@ -19,17 +19,44 @@ Every several seconds, the application pretends to watch a particular stream by
### Usage: ### Usage:
- Download and unzip [the latest release](https://github.com/DevilXD/TwitchDropsMiner/releases) - it's recommended to keep it in the folder it comes in. **Quick Start with Docker (Recommended):**
- Run it and login/connect the miner to your Twitch account by using the in-app login form.
- After a successful login, the app should fetch a list of all available campaigns and games you can mine drops for - you can then select and add games of choice to the Priority List available on the Settings tab, and then press on the `Reload` button to start processing. It will fetch a list of all applicable streams it can watch, and start mining right away. You can also manually switch to a different channel as needed.
- If you wish to keep the miner occupied with mining anything it can, beyond what you've selected via the Priority List, you can use the Priority Mode setting to specify the mining order for the rest of the games.
- Make sure to link your Twitch account to game accounts on the [campaigns page](https://www.twitch.tv/drops/campaigns), to enable more games to be mined.
### Pictures: ```bash
# Clone the repository
git clone https://github.com/DevilXD/TwitchDropsMiner.git
cd TwitchDropsMiner
![Main](https://user-images.githubusercontent.com/4180725/164298155-c0880ad7-6423-4419-8d73-f3c053730a1b.png) # Start with docker-compose
![Inventory](https://user-images.githubusercontent.com/4180725/164298315-81cae0d2-24a4-4822-a056-154fd763c284.png) docker-compose up -d
![Settings](https://user-images.githubusercontent.com/4180725/164298391-b13ad40d-3881-436c-8d4c-34e2bbe33a78.png)
# Access the web interface at http://localhost:8080
```
**Running from Source:**
```bash
# Install Python 3.10+ and dependencies
pip install -r requirements.txt
# Run the application
python main.py
# Access the web interface at http://localhost:8080
```
**Using the Application:**
- Open the web interface in your browser at `http://localhost:8080`
- Login/connect the miner to your Twitch account using the OAuth device code flow
- After a successful login, the app will fetch all available campaigns and games you can mine drops for
- Select and add games to the Priority List on the Settings tab, then press `Reload` to start processing
- The miner will fetch applicable streams and start mining automatically
- You can manually switch to a different channel as needed
- Make sure to link your Twitch account to game accounts on the [campaigns page](https://www.twitch.tv/drops/campaigns)
### Screenshots:
The application features a modern web-based interface accessible from any browser on your network.
### Notes: ### Notes:
@@ -50,24 +77,42 @@ Every several seconds, the application pretends to watch a particular stream by
> [!NOTE] > [!NOTE]
> The source code requires Python 3.10 or higher to run. > The source code requires Python 3.10 or higher to run.
### Notes about the Windows build: ### Docker Deployment:
- To achieve a portable-executable format, the application is packaged with PyInstaller into an `EXE`. Some antivirus engines (including Windows Defender) might report the packaged executable as a trojan, because PyInstaller has been used by others to package malicious Python code in the past. These reports can be safely ignored. If you absolutely do not trust the executable, you'll have to install Python yourself and run everything from source. The application is designed for Docker deployment, making it easy to run on any platform:
- The executable uses the `%TEMP%` directory for temporary runtime storage of files, that don't need to be exposed to the user (like compiled code and translation files). For persistent storage, the directory the executable resides in is used instead.
- The autostart feature is implemented as a registry entry to the current user's (`HKCU`) autostart key. It is only altered when toggling the respective option. If you relocate the app to a different directory, the autostart feature will stop working, until you toggle the option off and back on again
### Notes about the Linux build: ```bash
# Build and run with docker-compose
docker-compose up -d
- The Linux app is built and distributed using two distinct portable-executable formats: [AppImage](https://appimage.org/) and [PyInstaller](https://pyinstaller.org/). # Or build and run manually
- There are no major differences between the two formats, but if you're looking for a recommendation, use the AppImage. docker build -t twitch-drops-miner .
- The Linux app should work out of the box on any modern distribution, as long as it has `glibc>=2.35`, plus a working display server. docker run -d -p 8080:8080 -v $(pwd)/data:/app/data twitch-drops-miner
- Every feature of the app is expected to work on Linux just as well as it does on Windows. If you find something that's broken, please [open a new issue](https://github.com/DevilXD/TwitchDropsMiner/issues/new). ```
- The size of the Linux app is significantly larger than the Windows app due to the inclusion of the `gtk3` library (and its dependencies), which is required for proper system tray/notifications support.
- As an alternative to the native Linux app, you can run the Windows app via [Wine](https://www.winehq.org/) instead. It works really well!
### Advanced Usage: **Important Docker notes:**
- All persistent data (cookies, settings, logs) is stored in the `data/` directory
- Login uses OAuth device code flow - you'll be given a code to enter at twitch.tv/activate
- Browser notifications supported (requires permission)
- Health checks and automatic restart included in docker-compose
- Configure timezone with `TZ` environment variable
If you'd be interested in running the latest master from source or building your own executable, see the wiki page explaining how to do so: https://github.com/DevilXD/TwitchDropsMiner/wiki/Setting-up-the-environment,-building-and-running ### Running from Source:
For development or customization:
```bash
# Install Python 3.10+
# Create virtual environment (recommended)
python -m venv env
source env/bin/activate # On Windows: env\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Run the application
python main.py
```
### Support ### Support
@@ -84,37 +129,29 @@ If you'd be interested in running the latest master from source or building your
### Project goals: ### Project goals:
Twitch Drops Miner (TDM for short) has been designed with a couple of simple goals in mind. These are, specifically: Twitch Drops Miner (TDM for short) has been designed with a couple of simple goals in mind:
- Twitch Drops oriented - it's in the name. That's what I made it for. **What TDM is:**
- Easy to use for an average person. Includes a nice looking GUI and is packaged as a ready-to-go executable, without requiring an existing Python installation to work. - **Twitch Drops focused** - Automatic mining of timed Twitch drops
- Intended as a helper tool that starts together with your PC, runs in the background through out the day, and then closes together with your PC shutting down at the end of the day. If it can run continuously for 24 hours at minimum, and not run into any errors, I'd call that good enough already. - **Easy to use** - Simple web interface accessible from any browser
- Requiring a minimum amount of attention during operation - check it once or twice through out the day to see if everything's fine with it. - **Reliable** - Designed to run continuously with minimal attention needed
- Underlying service friendly - the amount of interactions done with the Twitch site is kept to the minimum required for reliable operation, at a level achievable by a diligent site user. - **Efficient** - Minimal interactions with Twitch, respecting their service
- **Docker-ready** - Easy deployment on any platform or server
TDM is not intended for/as: **What TDM is NOT:**
- Mining channel points
- Mining other platforms besides Twitch
- Multi-account support
- Mining unlinked campaigns
- 100% guaranteed uptime (expect occasional errors)
- Mining channel points - again, it's about the drops: only. **Limitations:**
- Mining anything else besides Twitch drops - no, I won't be adding support for a random 3rd party site that also happens to rely on watching Twitch streams. - Single account per instance
- Unattended operation: worst case scenario, it'll stop working and you'll hopefully notice that at some point. Hopefully. - No automatic error recovery (monitor periodically)
- 100% uptime application, due to the underlying nature of it, expect fatal errors to happen every so often. - No additional notification systems (email, webhook, etc.)
- Being hosted on a remote server as a 24/7 miner. - Campaigns must be linked to your Twitch account
- Being used with more than one managed account.
- Mining campaigns the managed account isn't linked to.
This means that features such as: This is a web-only application designed for Docker deployment and headless operation.
- It being possible to run it without a GUI, or with only a console attached.
- Any form of automatic restart when an error happens.
- Docker or any other form of remote deployment.
- Using it with more than one managed account.
- Making it possible to mine campaigns that the managed account isn't linked to.
- Anything that increases the site processing load caused by the application.
- Any form of additional notifications system (email, webhook, etc.), beyond what's already implemented.
..., are most likely not going to be a feature, ever. You're welcome to search through the existing issues to comment on your point of view on the relevant matters, where applicable. Otherwise, most of the new issues that go against these goals will be closed and the user will be pointed to this paragraph.
For more context about these goals, please check out these issues: [#161](https://github.com/DevilXD/TwitchDropsMiner/issues/161), [#105](https://github.com/DevilXD/TwitchDropsMiner/issues/105), [#84](https://github.com/DevilXD/TwitchDropsMiner/issues/84)
### Credits: ### Credits:

164
appimage/AppImageBuilder.yml
View File

@@ -1,164 +0,0 @@
# Useful links:
# https://appimage-builder.readthedocs.io/en/latest/reference/recipe.html
version: 1
script:
# Clean up any previous builds.
- rm -rf "$BUILD_DIR" "$TARGET_APPDIR" || true
- mkdir -p "$BUILD_DIR" "$TARGET_APPDIR"
- cd "$BUILD_DIR"
# Build a recent version of libXft first. This fixes an issue where the app won't start with libXft 2.3.3 if an emoji font is installed on the system.
- curl -fL https://xorg.freedesktop.org/releases/individual/lib/libXft-2.3.9.tar.xz -o libXft.tar.xz
- sha256sum libXft.tar.xz
- tar xvf libXft.tar.xz
- cd libXft-*
- ./configure --prefix="$TARGET_APPDIR/usr" --disable-static
- make -j$(nproc)
- make install-strip
# Package the app.
- mkdir -p "$TARGET_APPDIR"/usr/{src,share/icons/hicolor/128x128/apps}
- cp -r "$SOURCE_DIR/../lang" "$SOURCE_DIR/../icons" "$SOURCE_DIR"/../*.py "$TARGET_APPDIR/usr/src"
- cp "$SOURCE_DIR/pickaxe.png" "$TARGET_APPDIR/usr/share/icons/hicolor/128x128/apps/io.github.devilxd.twitchdropsminer.png"
# Create a virtual environment and install up-to-date versions of meson and ninja.
- python3 -m venv env && ./env/bin/python3 -m pip install meson ninja
# Install requirements.
- PATH="$PWD/env/bin:$PATH" python3 -m pip install --prefix=/usr --root="$TARGET_APPDIR" -r "$SOURCE_DIR/../requirements.txt"
# Generate byte-code files and package them for a slightly faster app startup.
- python3 -m compileall "$TARGET_APPDIR/usr/src/"*.py
# Recursively remove ELF debug symbols, trimming roughly 10 MB off the AppDir.
- find "$TARGET_APPDIR" -type f -print0 | xargs -0 file | grep -F 'not stripped' | cut -f1 -d':' | sort -V | xargs -r strip -s -v
AppDir:
app_info:
id: io.github.devilxd.twitchdropsminer
name: Twitch Drops Miner
icon: io.github.devilxd.twitchdropsminer
version: '{{APP_VERSION}}'
exec: usr/bin/python3
exec_args: '${APPDIR}/usr/src/main.py $@'
apt:
arch:
- '{{ARCH_APT}}'
sources:
- sourceline: deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ jammy main
- sourceline: deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ jammy-updates main
- sourceline: deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ jammy-security main
- sourceline: deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ jammy main
- sourceline: deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ jammy-updates main
- sourceline: deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ jammy-security main
key_url: http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x871920D1991BC93C
include:
- gir1.2-ayatanaappindicator3-0.1
- python3-tk
exclude:
- adwaita-icon-theme
- dconf-service
- glib-networking-services
- gsettings-desktop-schemas
- hicolor-icon-theme
- humanity-icon-theme
- libavahi-client3
- libavahi-common3
- libbrotli1
- libcolord2
- libcups2
- libdb5.3
- libdconf1
- libgmp10
- libgnutls30
- libgssapi-krb5-2
- libhogweed5
- libicu66
- libjson-glib-1.0-0
- libk5crypto3
- libkrb5-3
- libkrb5support0
- liblcms2-2
- libncursesw6
- libnettle7
- libp11-kit0
- libpangoxft-1.0-0
- libpsl5
- librest-0.7-0
- libsoup2.4-1
- libsoup-gnome2.4-1
- libsqlite3-0
- libtasn1-6
- libtiff5
- libtinfo6
- libunistring2
- libwebp6
- libxft2 # We'll ship our own, updated version of this library.
- libxml2
- mime-support
- readline-common
- tzdata
- xkb-data
files:
exclude:
- etc
- usr/bin/normalizer
- usr/bin/pdb3*
- usr/bin/py3*
- usr/bin/pydoc*
- usr/bin/pygettext3*
- usr/include
- usr/lib/binfmt.d
- usr/lib/blt2.5
- usr/lib/libBLTlite.2.5*
# The next 2 files come from our own build of libXft, and they can be removed.
- usr/lib/libXft.la
- usr/lib/pkgconfig
- usr/lib/python3
- usr/lib/python3.11
- usr/lib/python3*/pydoc_data
- usr/lib/python3*/test
- usr/lib/python3*/unittest
- usr/lib/valgrind
- usr/lib/*-linux-gnu/audit
- usr/lib/*-linux-gnu/engines-3
- usr/lib/*-linux-gnu/gconv
- usr/lib/*-linux-gnu/glib-2.0
- usr/lib/*-linux-gnu/gtk-3.0
- usr/lib/*-linux-gnu/libgtk-3-0
- usr/lib/*-linux-gnu/ossl-modules
- usr/sbin
- usr/share/applications
- usr/share/binfmts
- usr/share/bug
- usr/share/doc
- usr/share/doc-base
- usr/share/gcc
- usr/share/gdb
- usr/share/glib-2.0
- usr/share/lintian
- usr/share/man
- usr/share/pixmaps
- usr/share/python3
- usr/share/themes
runtime:
env:
PATH: '${APPDIR}/usr/bin:${PATH}'
PYTHONHOME: '${APPDIR}/usr'
PYTHONPATH: '${APPDIR}/usr/lib/python3.10/tkinter:${APPDIR}/usr/lib/python3.10/site-packages'
APPDIR_LIBRARY_PATH: '${APPDIR}/usr/lib:${APPDIR}/usr/lib/{{ARCH}}-linux-gnu:${APPDIR}/lib/{{ARCH}}-linux-gnu'
TCL_LIBRARY: '${APPDIR}/usr/share/tcltk/tcl8.6'
TK_LIBRARY: '${APPDIR}/usr/lib/tcltk/{{ARCH}}-linux-gnu/tk8.6'
TKPATH: '${APPDIR}/usr/lib/tcltk/{{ARCH}}-linux-gnu/tk8.6'
AppImage:
arch: '{{ARCH}}'
file_name: Twitch.Drops.Miner-{{ARCH}}.AppImage
comp: zstd
sign-key: None
update-information: guess

BIN
appimage/pickaxe.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.2 KiB

495
constants.py
View File

@@ -1,495 +0,0 @@
from __future__ import annotations
import os
import sys
import random
import logging
from pathlib import Path
from copy import deepcopy
from enum import Enum, auto
from datetime import timedelta
from typing import Any, Dict, Literal, NewType, TYPE_CHECKING
from yarl import URL
from version import __version__
if TYPE_CHECKING:
from collections import abc # noqa
from typing_extensions import TypeAlias
# True if we're running from a built EXE (or a Linux AppImage), False inside a dev build
IS_APPIMAGE = "APPIMAGE" in os.environ and os.path.exists(os.environ["APPIMAGE"])
IS_PACKAGED = hasattr(sys, "_MEIPASS") or IS_APPIMAGE
# logging special levels
CALL: int = logging.INFO - 1
logging.addLevelName(CALL, "CALL")
# site-packages venv path changes depending on the system platform
if sys.platform == "win32":
SYS_SITE_PACKAGES = "Lib/site-packages"
else:
# On Linux, the site-packages path includes a versioned 'pythonX.Y' folder part
# The Lib folder is also spelled in lowercase: 'lib'
version_info = sys.version_info
SYS_SITE_PACKAGES = f"lib/python{version_info.major}.{version_info.minor}/site-packages"
# scripts venv path changes depending on the system platform
if sys.platform == "win32":
SYS_SCRIPTS = "Scripts"
else:
SYS_SCRIPTS = "bin"
def _resource_path(relative_path: Path | str) -> Path:
"""
Get an absolute path to a bundled resource.
Works for dev and for PyInstaller.
"""
if IS_APPIMAGE:
base_path = Path(sys.argv[0]).resolve().parent
elif IS_PACKAGED:
# PyInstaller's folder where the one-file app is unpacked
meipass: str = getattr(sys, "_MEIPASS")
base_path = Path(meipass)
else:
base_path = WORKING_DIR
return base_path.joinpath(relative_path)
def _merge_vars(base_vars: JsonType, vars: JsonType) -> None:
# NOTE: This modifies base in place
for k, v in vars.items():
if k not in base_vars:
base_vars[k] = v
elif isinstance(v, dict):
if isinstance(base_vars[k], dict):
_merge_vars(base_vars[k], v)
elif base_vars[k] is Ellipsis:
# unspecified base, use the passed in var
base_vars[k] = v
else:
raise RuntimeError(f"Var is a dict, base is not: '{k}'")
elif isinstance(base_vars[k], dict):
raise RuntimeError(f"Base is a dict, var is not: '{k}'")
else:
# simple overwrite
base_vars[k] = v
# ensure none of the vars are ellipsis (unset value)
for k, v in base_vars.items():
if v is Ellipsis:
raise RuntimeError(f"Unspecified variable: '{k}'")
# Base Paths
if IS_APPIMAGE:
SELF_PATH = Path(os.environ["APPIMAGE"]).resolve()
else:
# NOTE: pyinstaller will set sys.argv[0] to its own executable when building
# NOTE: sys.argv[0] will point to gui.py when running the gui.py directly for GUI debug
# detect these and use __file__ and main.py redirection instead
SELF_PATH = Path(sys.argv[0]).resolve()
if SELF_PATH.stem == "pyinstaller" or SELF_PATH.name == "gui.py":
SELF_PATH = Path(__file__).with_name("main.py").resolve()
WORKING_DIR = SELF_PATH.parent
# Development paths
VENV_PATH = Path(WORKING_DIR, "env")
SITE_PACKAGES_PATH = Path(VENV_PATH, SYS_SITE_PACKAGES)
SCRIPTS_PATH = Path(VENV_PATH, SYS_SCRIPTS)
# Translations path
# NOTE: These don't have to be available to the end-user, so the path points to the internal dir
LANG_PATH = _resource_path("lang")
# Other Paths
LOG_PATH = Path(WORKING_DIR, "log.txt")
DUMP_PATH = Path(WORKING_DIR, "dump.dat")
LOCK_PATH = Path(WORKING_DIR, "lock.file")
CACHE_PATH = Path(WORKING_DIR, "cache")
CACHE_DB = Path(CACHE_PATH, "mapping.json")
COOKIES_PATH = Path(WORKING_DIR, "cookies.jar")
SETTINGS_PATH = Path(WORKING_DIR, "settings.json")
# Typing
JsonType = Dict[str, Any]
URLType = NewType("URLType", str)
TopicProcess: TypeAlias = "abc.Callable[[int, JsonType], Any]"
# Values
MAX_INT = sys.maxsize
MAX_EXTRA_MINUTES = 15
BASE_TOPICS = 2
MAX_WEBSOCKETS = 8
WS_TOPICS_LIMIT = 50
TOPICS_PER_CHANNEL = 2
MAX_TOPICS = (MAX_WEBSOCKETS * WS_TOPICS_LIMIT) - BASE_TOPICS
MAX_CHANNELS = MAX_TOPICS // TOPICS_PER_CHANNEL
# Misc
DEFAULT_LANG = "English"
# Intervals and Delays
PING_INTERVAL = timedelta(minutes=3)
PING_TIMEOUT = timedelta(seconds=10)
ONLINE_DELAY = timedelta(seconds=120)
WATCH_INTERVAL = timedelta(seconds=59)
# Strings
WINDOW_TITLE = f"Twitch Drops Miner v{__version__} (by DevilXD)"
# Logging
LOGGING_LEVELS = {
0: logging.ERROR,
1: logging.WARNING,
2: logging.INFO,
3: CALL,
4: logging.DEBUG,
}
FILE_FORMATTER = logging.Formatter(
"{asctime}.{msecs:03.0f}:\t{levelname:>7}:\t{message}",
style='{',
datefmt="%Y-%m-%d %H:%M:%S",
)
OUTPUT_FORMATTER = logging.Formatter("{levelname}: {message}", style='{', datefmt="%H:%M:%S")
class ClientInfo:
def __init__(self, client_url: URL, client_id: str, user_agents: str | list[str]) -> None:
self.CLIENT_URL: URL = client_url
self.CLIENT_ID: str = client_id
self.USER_AGENT: str
if isinstance(user_agents, list):
self.USER_AGENT = random.choice(user_agents)
else:
self.USER_AGENT = user_agents
def __iter__(self):
return iter((self.CLIENT_URL, self.CLIENT_ID, self.USER_AGENT))
class ClientType:
WEB = ClientInfo(
URL("https://www.twitch.tv"),
"kimne78kx3ncx6brgo4mv6wki5h1ko",
(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
),
)
MOBILE_WEB = ClientInfo(
URL("https://m.twitch.tv"),
"r8s4dac0uhzifbpu9sjdiwzctle17ff",
[
# Chrome versioning is done fully on android only,
# other platforms only use the major version
(
"Mozilla/5.0 (Linux; Android 16) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-A205U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-A102U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-G960U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-N960U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; LM-Q720) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; LM-X420) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
]
)
ANDROID_APP = ClientInfo(
URL("https://www.twitch.tv"),
"kd1unb4b3q4t58fwlpcbzcbnm76a8fp",
[
(
"Dalvik/2.1.0 (Linux; U; Android 16; SM-S911B Build/TP1A.220624.014) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 16; SM-S938B Build/BP2A.250605.031) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; Android 16; SM-X716N Build/UP1A.231005.007) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-G990B Build/AP3A.240905.015.A2) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-G970F Build/AP3A.241105.008) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-A566E Build/AP3A.240905.015.A2) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 14; SM-X306B Build/UP1A.231005.007) "
"tv.twitch.android.app/25.3.0/2503006"
),
]
)
SMARTBOX = ClientInfo(
URL("https://android.tv.twitch.tv"),
"ue6666qo983tsx6so1t0vnawi233wa",
(
"Mozilla/5.0 (Linux; Android 7.1; Smart Box C1) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
),
)
class State(Enum):
IDLE = auto()
INVENTORY_FETCH = auto()
GAMES_UPDATE = auto()
CHANNELS_FETCH = auto()
CHANNELS_CLEANUP = auto()
CHANNEL_SWITCH = auto()
EXIT = auto()
class PriorityMode(Enum):
PRIORITY_ONLY = 0
ENDING_SOONEST = 1
LOW_AVBL_FIRST = 2
class GQLOperation(JsonType):
def __init__(self, name: str, sha256: str, *, variables: JsonType | None = None):
super().__init__(
operationName=name,
extensions={
"persistedQuery": {
"version": 1,
"sha256Hash": sha256,
}
}
)
if variables is not None:
self.__setitem__("variables", variables)
def with_variables(self, variables: JsonType) -> GQLOperation:
modified = deepcopy(self)
if "variables" in self:
existing_variables: JsonType = modified["variables"]
_merge_vars(existing_variables, variables)
else:
modified["variables"] = variables
return modified
GQL_OPERATIONS: dict[str, GQLOperation] = {
# returns stream information for a particular channel
"GetStreamInfo": GQLOperation(
"VideoPlayerStreamInfoOverlayChannel",
"198492e0857f6aedead9665c81c5a06d67b25b58034649687124083ff288597d",
variables={
"channel": ..., # channel login
},
),
# can be used to claim channel points
"ClaimCommunityPoints": GQLOperation(
"ClaimCommunityPoints",
"46aaeebe02c99afdf4fc97c7c0cba964124bf6b0af229395f1f6d1feed05b3d0",
variables={
"input": {
"claimID": ..., # points claim_id
"channelID": ..., # channel ID as a str
},
},
),
# can be used to claim a drop
"ClaimDrop": GQLOperation(
"DropsPage_ClaimDropRewards",
"a455deea71bdc9015b78eb49f4acfbce8baa7ccbedd28e549bb025bd0f751930",
variables={
"input": {
"dropInstanceID": ..., # drop claim_id
},
},
),
# returns current state of points (balance, claim available) for a particular channel
"ChannelPointsContext": GQLOperation(
"ChannelPointsContext",
"374314de591e69925fce3ddc2bcf085796f56ebb8cad67a0daa3165c03adc345",
variables={
"channelLogin": ..., # channel login
},
),
# returns all in-progress campaigns
"Inventory": GQLOperation(
"Inventory",
"d86775d0ef16a63a33ad52e80eaff963b2d5b72fada7c991504a57496e1d8e4b",
variables={
"fetchRewardCampaigns": False,
}
),
# returns current state of drops (current drop progress)
"CurrentDrop": GQLOperation(
"DropCurrentSessionContext",
"4d06b702d25d652afb9ef835d2a550031f1cf762b193523a92166f40ea3d142b",
variables={
"channelID": ..., # watched channel ID as a str
"channelLogin": "", # always empty string
},
),
# returns all available campaigns
"Campaigns": GQLOperation(
"ViewerDropsDashboard",
"5a4da2ab3d5b47c9f9ce864e727b2cb346af1e3ea8b897fe8f704a97ff017619",
variables={
"fetchRewardCampaigns": False,
}
),
# returns extended information about a particular campaign
"CampaignDetails": GQLOperation(
"DropCampaignDetails",
"039277bf98f3130929262cc7c6efd9c141ca3749cb6dca442fc8ead9a53f77c1",
variables={
"channelLogin": ..., # user login
"dropID": ..., # campaign ID
},
),
# returns drops available for a particular channel
"AvailableDrops": GQLOperation(
"DropsHighlightService_AvailableDrops",
"9a62a09bce5b53e26e64a671e530bc599cb6aab1e5ba3cbd5d85966d3940716f",
variables={
"channelID": ..., # channel ID as a str
},
),
# retuns stream playback access token
"PlaybackAccessToken": GQLOperation(
"PlaybackAccessToken",
"ed230aa1e33e07eebb8928504583da78a5173989fadfb1ac94be06a04f3cdbe9",
variables={
"isLive": True,
"isVod": False,
"login": ..., # channel login
"platform": "web",
"playerType": "site",
"vodID": "",
},
),
# returns live channels for a particular game
"GameDirectory": GQLOperation(
"DirectoryPage_Game",
"98a996c3c3ebb1ba4fd65d6671c6028d7ee8d615cb540b0731b3db2a911d3649",
variables={
"limit": 30, # limit of channels returned
"slug": ..., # game slug
"imageWidth": 50,
"includeCostreaming": False,
"options": {
"broadcasterLanguages": [],
"freeformTags": None,
"includeRestricted": ["SUB_ONLY_LIVE"],
"recommendationsContext": {"platform": "web"},
"sort": "RELEVANCE", # also accepted: "VIEWER_COUNT"
"systemFilters": [],
"tags": [],
"requestID": "JIRA-VXP-2397",
},
"sortTypeIsRecency": False,
},
),
"SlugRedirect": GQLOperation( # can be used to turn game name -> game slug
"DirectoryGameRedirect",
"1f0300090caceec51f33c5e20647aceff9017f740f223c3c532ba6fa59f6b6cc",
variables={
"name": ..., # game name
},
),
"NotificationsView": GQLOperation( # unused, triggers notifications "update-summary"
"OnsiteNotifications_View",
"e8e06193f8df73d04a1260df318585d1bd7a7bb447afa058e52095513f2bfa4f",
variables={
"input": {},
},
),
"NotificationsList": GQLOperation( # unused
"OnsiteNotifications_ListNotifications",
"11cdb54a2706c2c0b2969769907675680f02a6e77d8afe79a749180ad16bfea6",
variables={
"cursor": "",
"displayType": "VIEWER",
"language": "en",
"limit": 10,
"shouldLoadLastBroadcast": False,
},
),
"NotificationsDelete": GQLOperation(
"OnsiteNotifications_DeleteNotification",
"13d463c831f28ffe17dccf55b3148ed8b3edbbd0ebadd56352f1ff0160616816",
variables={
"input": {
"id": "", # ID of the notification to delete
}
},
),
}
class WebsocketTopic:
def __init__(
self,
category: Literal["User", "Channel"],
topic_name: str,
target_id: int,
process: TopicProcess,
):
assert isinstance(target_id, int)
self._id: str = self.as_str(category, topic_name, target_id)
self._target_id = target_id
self._process: TopicProcess = process
@classmethod
def as_str(
cls, category: Literal["User", "Channel"], topic_name: str, target_id: int
) -> str:
return f"{WEBSOCKET_TOPICS[category][topic_name]}.{target_id}"
def __call__(self, message: JsonType):
return self._process(self._target_id, message)
def __str__(self) -> str:
return self._id
def __repr__(self) -> str:
return f"Topic({self._id})"
def __eq__(self, other) -> bool:
if isinstance(other, WebsocketTopic):
return self._id == other._id
elif isinstance(other, str):
return self._id == other
return NotImplemented
def __hash__(self) -> int:
return hash((self.__class__.__name__, self._id))
WEBSOCKET_TOPICS: dict[str, dict[str, str]] = {
"User": { # Using user_id
"Presence": "presence", # unused
"Drops": "user-drop-events",
"Notifications": "onsite-notifications",
"CommunityPoints": "community-points-user-v1",
},
"Channel": { # Using channel_id
"Drops": "channel-drop-events", # unused
"StreamState": "video-playback-by-id",
"StreamUpdate": "broadcast-settings-update",
"CommunityPoints": "community-points-channel-v1", # unused
},
}

33
docker-compose.yml Normal file
View File

@@ -0,0 +1,33 @@
version: '3.8'
services:
twitch-drops-miner:
build: .
container_name: twitch-drops-miner
image: twitch-drops-miner:latest
ports:
- "8080:8080"
volumes:
# Mount data directory for persistent storage
- ./data:/app/data
environment:
# Set timezone (optional, defaults to UTC)
- TZ=UTC
# Docker environment flag
- DOCKER_ENV=1
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/api/status')"]
interval: 30s
timeout: 3s
retries: 3
start_period: 10s
# Optional: Limit resources
# deploy:
# resources:
# limits:
# cpus: '1'
# memory: 512M
# reservations:
# cpus: '0.5'
# memory: 256M

2759
gui.py
View File

File diff suppressed because it is too large Load Diff

207
main.py
View File

@@ -1,202 +1,13 @@
from __future__ import annotations #!/usr/bin/env python3
"""
# import an additional thing for proper PyInstaller freeze support TwitchDropsMiner - Main entry point
from multiprocessing import freeze_support
This is a simple launcher that runs the src package as a module.
All application code is in the src/ directory.
"""
if __name__ == "__main__": if __name__ == "__main__":
freeze_support() import runpy
import io
import sys
import signal
import asyncio
import logging
import argparse
import warnings
import traceback
import tkinter as tk
from tkinter import messagebox
from typing import NoReturn, TYPE_CHECKING
import truststore # Run the src package as a module
truststore.inject_into_ssl() runpy.run_module("src", run_name="__main__")
from translate import _
from twitch import Twitch
from settings import Settings
from version import __version__
from exceptions import CaptchaRequired
from utils import lock_file, resource_path, set_root_icon
from constants import LOGGING_LEVELS, SELF_PATH, FILE_FORMATTER, LOG_PATH, LOCK_PATH
if TYPE_CHECKING:
from _typeshed import SupportsWrite
warnings.simplefilter("default", ResourceWarning)
# import tracemalloc
# tracemalloc.start(3)
if sys.version_info < (3, 10):
raise RuntimeError("Python 3.10 or higher is required")
class Parser(argparse.ArgumentParser):
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
self._message: io.StringIO = io.StringIO()
def _print_message(self, message: str, file: SupportsWrite[str] | None = None) -> None:
self._message.write(message)
# print(message, file=self._message)
def exit(self, status: int = 0, message: str | None = None) -> NoReturn:
try:
super().exit(status, message) # sys.exit(2)
finally:
messagebox.showerror("Argument Parser Error", self._message.getvalue())
class ParsedArgs(argparse.Namespace):
_verbose: int
_debug_ws: bool
_debug_gql: bool
log: bool
tray: bool
dump: bool
# TODO: replace int with union of literal values once typeshed updates
@property
def logging_level(self) -> int:
return LOGGING_LEVELS[min(self._verbose, 4)]
@property
def debug_ws(self) -> int:
"""
If the debug flag is True, return DEBUG.
If the main logging level is DEBUG, return INFO to avoid seeing raw messages.
Otherwise, return NOTSET to inherit the global logging level.
"""
if self._debug_ws:
return logging.DEBUG
elif self._verbose >= 4:
return logging.INFO
return logging.NOTSET
@property
def debug_gql(self) -> int:
if self._debug_gql:
return logging.DEBUG
elif self._verbose >= 4:
return logging.INFO
return logging.NOTSET
# handle input parameters
# NOTE: parser output is shown via message box
# we also need a dummy invisible window for the parser
root = tk.Tk()
root.overrideredirect(True)
root.withdraw()
set_root_icon(root, resource_path("icons/pickaxe.ico"))
root.update()
parser = Parser(
SELF_PATH.name,
description="A program that allows you to mine timed drops on Twitch.",
)
parser.add_argument("--version", action="version", version=f"v{__version__}")
parser.add_argument("-v", dest="_verbose", action="count", default=0)
parser.add_argument("--tray", action="store_true")
parser.add_argument("--log", action="store_true")
parser.add_argument("--dump", action="store_true")
# undocumented debug args
parser.add_argument(
"--debug-ws", dest="_debug_ws", action="store_true", help=argparse.SUPPRESS
)
parser.add_argument(
"--debug-gql", dest="_debug_gql", action="store_true", help=argparse.SUPPRESS
)
args = parser.parse_args(namespace=ParsedArgs())
# load settings
try:
settings = Settings(args)
except Exception:
messagebox.showerror(
"Settings error",
f"There was an error while loading the settings file:\n\n{traceback.format_exc()}"
)
sys.exit(4)
# dummy window isn't needed anymore
root.destroy()
# get rid of unneeded objects
del root, parser
# client run
async def main():
# set language
try:
_.set_language(settings.language)
except ValueError:
# this language doesn't exist - stick to English
pass
# handle logging stuff
if settings.logging_level > logging.DEBUG:
# redirect the root logger into a NullHandler, effectively ignoring all logging calls
# that aren't ours. This always runs, unless the main logging level is DEBUG or lower.
logging.getLogger().addHandler(logging.NullHandler())
logger = logging.getLogger("TwitchDrops")
logger.setLevel(settings.logging_level)
if settings.log:
handler = logging.FileHandler(LOG_PATH)
handler.setFormatter(FILE_FORMATTER)
logger.addHandler(handler)
logging.getLogger("TwitchDrops.gql").setLevel(settings.debug_gql)
logging.getLogger("TwitchDrops.websocket").setLevel(settings.debug_ws)
exit_status = 0
client = Twitch(settings)
loop = asyncio.get_running_loop()
if sys.platform == "linux":
loop.add_signal_handler(signal.SIGINT, lambda *_: client.gui.close())
loop.add_signal_handler(signal.SIGTERM, lambda *_: client.gui.close())
try:
await client.run()
except CaptchaRequired:
exit_status = 1
client.prevent_close()
client.print(_("error", "captcha"))
except Exception:
exit_status = 1
client.prevent_close()
client.print("Fatal error encountered:\n")
client.print(traceback.format_exc())
finally:
if sys.platform == "linux":
loop.remove_signal_handler(signal.SIGINT)
loop.remove_signal_handler(signal.SIGTERM)
client.print(_("gui", "status", "exiting"))
await client.shutdown()
if not client.gui.close_requested:
# user didn't request the closure
client.gui.tray.change_icon("error")
client.print(_("status", "terminated"))
client.gui.status.update(_("gui", "status", "terminated"))
# notify the user about the closure
client.gui.grab_attention(sound=True)
await client.gui.wait_until_closed()
# save the application state
# NOTE: we have to do it after wait_until_closed,
# because the user can alter some settings between app termination and closing the window
client.save(force=True)
client.gui.stop()
client.gui.close_window()
sys.exit(exit_status)
try:
# use lock_file to check if we're not already running
success, file = lock_file(LOCK_PATH)
if not success:
# already running - exit
sys.exit(3)
asyncio.run(main())
finally:
file.close()

31
manual.txt
View File

@@ -1,31 +0,0 @@
###################################
# Twitch Drops Miner (by DevilXD) #
###################################
Available command line arguments:
• --tray
Start the application as minimised into tray.
• -v
Increase verbosity level. Can be stacked up several times (-vv, -vvv, etc.) to show
increasingly more information during application runtime.
• --log
Enables logging of runtime information into a 'log.txt' file. Verbosity level of this logging
matches the level set by `-v`.
• --dump
Start the application in a data-dump mode, where a 'dump.dat' file is created.
The file contains anonymous raw Twitch API data, studying of which can help troubleshoot
issues with the application. The application automatically closes shortly after launching,
once dumping is finished.
• --version
Show application version information.
Note: Additional settings are available within the application GUI.
Exit codes:
• 0: Application exited successfully
• 1: Exit caused by the CAPTCHA or a Fatal Exception
• 2: Incorrect command line arguments
• 3: Application already running
• 4: Loading of the settings file failed

24
pack.bat
View File

@@ -1,24 +0,0 @@
@echo off
IF NOT EXIST 7z.exe GOTO NO7Z
IF NOT EXIST "Twitch Drops Miner" mkdir "Twitch Drops Miner"
rem Prepare files
copy /y /v dist\*.exe "Twitch Drops Miner"
copy /y /v manual.txt "Twitch Drops Miner"
IF EXIST "Twitch Drops Miner.zip" (
rem Add action
set action=a
) ELSE (
rem Update action
set action=u
)
rem Pack and test
7z %action% "Twitch Drops Miner.zip" "Twitch Drops Miner/" -r
7z t "Twitch Drops Miner.zip" * -r
rem Cleanup
IF EXIST "Twitch Drops Miner" rmdir /s /q "Twitch Drops Miner"
GOTO EXIT
:NO7Z
echo No 7z.exe detected, skipping packaging!
GOTO EXIT
:EXIT
exit %errorlevel%

112
registry.py
View File

@@ -1,112 +0,0 @@
from __future__ import annotations
import winreg as reg
from typing import Any
from enum import Enum, Flag
from collections.abc import Generator
class RegistryError(Exception):
pass
class ValueNotFound(RegistryError):
pass
class Access(Flag):
KEY_READ = reg.KEY_READ
KEY_WRITE = reg.KEY_WRITE
KEY_NOTIFY = reg.KEY_NOTIFY
KEY_EXECUTE = reg.KEY_EXECUTE
KEY_SET_VALUE = reg.KEY_SET_VALUE
KEY_ALL_ACCESS = reg.KEY_ALL_ACCESS
KEY_CREATE_LINK = reg.KEY_CREATE_LINK
KEY_QUERY_VALUE = reg.KEY_QUERY_VALUE
KEY_CREATE_SUB_KEY = reg.KEY_CREATE_SUB_KEY
KEY_ENUMERATE_SUB_KEYS = reg.KEY_ENUMERATE_SUB_KEYS
class MainKey(Enum):
HKU = reg.HKEY_USERS
HKCR = reg.HKEY_CLASSES_ROOT
HKCU = reg.HKEY_CURRENT_USER
HKLM = reg.HKEY_LOCAL_MACHINE
HKEY_USERS = reg.HKEY_USERS
HKEY_CLASSES_ROOT = reg.HKEY_CLASSES_ROOT
HKEY_CURRENT_USER = reg.HKEY_CURRENT_USER
HKEY_LOCAL_MACHINE = reg.HKEY_LOCAL_MACHINE
HKEY_CURRENT_CONFIG = reg.HKEY_CURRENT_CONFIG
HKEY_PERFORMANCE_DATA = reg.HKEY_PERFORMANCE_DATA
class ValueType(Enum):
REG_SZ = reg.REG_SZ
REG_NONE = reg.REG_NONE
REG_LINK = reg.REG_LINK
REG_DWORD = reg.REG_DWORD
REG_QWORD = reg.REG_QWORD
REG_BINARY = reg.REG_BINARY
REG_MULTI_SZ = reg.REG_MULTI_SZ
REG_EXPAND_SZ = reg.REG_EXPAND_SZ
REG_RESOURCE_LIST = reg.REG_RESOURCE_LIST
REG_DWORD_BIG_ENDIAN = reg.REG_DWORD_BIG_ENDIAN
REG_DWORD_LITTLE_ENDIAN = reg.REG_DWORD_LITTLE_ENDIAN
REG_QWORD_LITTLE_ENDIAN = reg.REG_QWORD_LITTLE_ENDIAN
REG_FULL_RESOURCE_DESCRIPTOR = reg.REG_FULL_RESOURCE_DESCRIPTOR
REG_RESOURCE_REQUIREMENTS_LIST = reg.REG_RESOURCE_REQUIREMENTS_LIST
class RegistryKey:
def __init__(self, path: str, *, read_only: bool = False):
main_key, _, path = path.replace('/', '\\').partition('\\')
self.main_key = MainKey[main_key]
self.path = path
access_flags = Access.KEY_QUERY_VALUE
if not read_only:
access_flags |= Access.KEY_SET_VALUE
self._handle = reg.OpenKey(self.main_key.value, path, access=access_flags.value)
def __enter__(self) -> RegistryKey:
return self
def __exit__(self, exc_type, exc, tb):
self._handle.Close()
def get(self, name: str) -> tuple[ValueType, Any]:
try:
value, value_type = reg.QueryValueEx(self._handle, name)
except FileNotFoundError:
# TODO: consider returning None for missing values
raise ValueNotFound(name)
return (ValueType(value_type), value)
def set(self, name: str, value_type: ValueType, value: Any) -> bool:
reg.SetValueEx(self._handle, name, 0, value_type.value, value)
return True # TODO: return False if the set operation fails
def delete(self, name: str, *, silent: bool = False) -> bool:
try:
reg.DeleteValue(self._handle, name)
except FileNotFoundError:
if not silent:
raise ValueNotFound(name)
return False
return True
def values(self) -> Generator[tuple[str, ValueType, Any], None, None]:
len_keys, len_values, last_modified = reg.QueryInfoKey(self._handle)
for i in range(len_values):
try:
name, value, value_type = reg.EnumValue(self._handle, i)
yield name, ValueType(value_type), value
except OSError:
return
if __name__ == "__main__":
with RegistryKey("HKCU/Software/Microsoft/Windows/CurrentVersion/Run") as key:
# key.get("test")
# key.set("test", ValueType.REG_SZ, "test\\path")
for name, value_type, value in key.values():
print(name, value_type, value)

13
requirements.txt
View File

@@ -1,8 +1,11 @@
# Core dependencies
aiohttp>=3.9,<4.0 aiohttp>=3.9,<4.0
Pillow Pillow
pystray
PyGObject<3.51; sys_platform == "linux" # required for better system tray support on Linux
# environment-dependent dependencies
pywin32; sys_platform == "win32"
truststore truststore
python-dateutil
# Web GUI dependencies (required)
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
python-socketio>=5.10.0
jinja2>=3.1.2

11
run_dev.bat
View File

@@ -1,11 +0,0 @@
@echo off
cls
set dirpath=%~dp0
if "%dirpath:~-1%" == "\" set dirpath=%dirpath:~0,-1%
set /p "choice=Start with a console? (y/n) "
if "%choice%"=="y" (
set "exepath=%dirpath%\env\scripts\python"
) else (
set "exepath=%dirpath%\env\scripts\pythonw"
)
start "TwitchDropsMiner" "%exepath%" "%dirpath%\main.py"

48
setup_env.bat
View File

@@ -1,48 +0,0 @@
@echo off
REM Get the directory path of the script
set "dirpath=%~dp0"
if "%dirpath:~-1%" == "\" set "dirpath=%dirpath:~0,-1%"
REM Check if git is installed
git --version > nul 2>&1
if %errorlevel% NEQ 0 (
echo:
echo No git executable found in PATH!
echo:
pause
exit /b 1
)
REM Create the virtual environment if it doesn't exist
if not exist "%dirpath%\env" (
echo:
echo Creating the env folder...
python -m venv "%dirpath%\env"
if %errorlevel% NEQ 0 (
echo:
echo No python executable found in PATH or failed to create virtual environment!
echo:
pause
exit /b 1
)
)
REM Activate the virtual environment and install requirements
echo:
echo Installing requirements.txt...
"%dirpath%\env\scripts\python" -m pip install -U pip
"%dirpath%\env\scripts\pip" install wheel
"%dirpath%\env\scripts\pip" install -r "%dirpath%\requirements.txt"
if %errorlevel% NEQ 0 (
echo:
echo Failed to install requirements.
echo:
pause
exit /b 1
)
echo:
echo Environment setup completed successfully.
echo:
pause

4
src/__init__.py Normal file
View File

@@ -0,0 +1,4 @@
"""TwitchDropsMiner - Modular source package."""
__version__ = "1.0.0"

263
src/__main__.py Normal file
View File

@@ -0,0 +1,263 @@
from __future__ import annotations
# import an additional thing for proper PyInstaller freeze support
from multiprocessing import freeze_support
if __name__ == "__main__":
freeze_support()
import sys
import signal
import asyncio
import logging
import argparse
import warnings
import traceback
from typing import NoReturn
import truststore
truststore.inject_into_ssl()
from src.i18n import _
from src.core.client import Twitch
from src.config.settings import Settings
from src.version import __version__
from src.exceptions import CaptchaRequired
from src.config.paths import _resource_path as resource_path
from src.config import LOGGING_LEVELS, SELF_PATH, FILE_FORMATTER, LOG_PATH, LOCK_PATH
logger = logging.getLogger("TwitchDrops")
# Force INFO level logging by default for better visibility
logger.setLevel(logging.DEBUG)
# Always add console handler
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setFormatter(FILE_FORMATTER)
logger.addHandler(console_handler)
logger.info("Logger initialized")
warnings.simplefilter("default", ResourceWarning)
# import tracemalloc
# tracemalloc.start(3)
if sys.version_info < (3, 10):
raise RuntimeError("Python 3.10 or higher is required")
class ParsedArgs(argparse.Namespace):
_verbose: int
_debug_ws: bool
_debug_gql: bool
log: bool
dump: bool
# TODO: replace int with union of literal values once typeshed updates
@property
def logging_level(self) -> int:
return LOGGING_LEVELS[min(self._verbose, 4)]
@property
def debug_ws(self) -> int:
"""
If the debug flag is True, return DEBUG.
If the main logging level is DEBUG, return INFO to avoid seeing raw messages.
Otherwise, return NOTSET to inherit the global logging level.
"""
if self._debug_ws:
return logging.DEBUG
elif self._verbose >= 4:
return logging.INFO
return logging.NOTSET
@property
def debug_gql(self) -> int:
if self._debug_gql:
return logging.DEBUG
elif self._verbose >= 4:
return logging.INFO
return logging.NOTSET
# handle input parameters
logger.debug("Parsing command line arguments")
parser = argparse.ArgumentParser(
SELF_PATH.name,
description="A program that allows you to mine timed drops on Twitch.",
)
parser.add_argument("--version", action="version", version=f"v{__version__}")
parser.add_argument("-v", dest="_verbose", action="count", default=0)
parser.add_argument("--log", action="store_true")
parser.add_argument("--dump", action="store_true")
# undocumented debug args
parser.add_argument(
"--debug-ws", dest="_debug_ws", action="store_true", help=argparse.SUPPRESS
)
parser.add_argument(
"--debug-gql", dest="_debug_gql", action="store_true", help=argparse.SUPPRESS
)
logger.debug("Parsing arguments into ParsedArgs namespace")
args = parser.parse_args(namespace=ParsedArgs())
# load settings
logger.debug("Loading settings")
try:
settings = Settings(args)
except Exception:
logger.exception("Error while loading settings")
print(f"Settings error: {traceback.format_exc()}", file=sys.stderr)
sys.exit(4)
# client run
logger.debug("Defining main async function")
async def main():
# set language
try:
_.set_language(settings.language)
except ValueError:
# this language doesn't exist - stick to English
pass
# Always log to file with timestamped filename in ./logs/ directory
from datetime import datetime
from pathlib import Path
# Create logs directory if it doesn't exist
logs_dir = Path("logs")
logs_dir.mkdir(exist_ok=True)
# Generate timestamped log filename: TDM.YYYY-MM-DDTHH-MM-SS.log
timestamp = datetime.now().isoformat(timespec='seconds').replace(':', '-')
log_file = logs_dir / f"TDM.{timestamp}.log"
# Add file handler for timestamped log
file_handler = logging.FileHandler(log_file)
file_handler.setFormatter(FILE_FORMATTER)
logger.addHandler(file_handler)
logger.info(f"Logging to file: {log_file}")
# Keep old log.txt for backward compatibility if --log flag is used
if settings.log:
legacy_handler = logging.FileHandler(LOG_PATH)
legacy_handler.setFormatter(FILE_FORMATTER)
logger.addHandler(legacy_handler)
logger.info(f"Legacy log file: {LOG_PATH}")
logging.getLogger("TwitchDrops.gql").setLevel(settings.debug_gql)
logging.getLogger("TwitchDrops.websocket").setLevel(settings.debug_ws)
logger.info("=== TwitchDropsMiner Starting ===")
logger.info(f"Version: {__version__}")
logger.info(f"Python version: {sys.version}")
logger.info(f"Platform: {sys.platform}")
exit_status = 0
logger.info("Creating Twitch client")
client = Twitch(settings)
# Initialize web GUI
logger.info("Initializing web GUI mode")
from src.web import app as webapp
from src.web.gui_manager import WebGUIManager
# Set up web GUI
logger.debug("Creating WebGUIManager")
client.gui = WebGUIManager(client)
# Set up webapp references
logger.debug("Setting up webapp managers")
webapp.set_managers(client.gui, client)
# Start web server in background
logger.info("Starting web server on http://0.0.0.0:8080")
web_server_task = asyncio.create_task(
webapp.run_server(host="0.0.0.0", port=8080)
)
logger.info("Web server task created")
loop = asyncio.get_running_loop()
if sys.platform == "linux":
logger.debug("Setting up signal handlers for SIGINT and SIGTERM")
loop.add_signal_handler(signal.SIGINT, lambda *_: client.gui.close())
loop.add_signal_handler(signal.SIGTERM, lambda *_: client.gui.close())
logger.info("Starting main client run loop")
try:
await client.run()
logger.info("Client run completed normally")
except CaptchaRequired:
logger.error("Captcha required - cannot continue")
exit_status = 1
client.prevent_close()
client.print(_("error", "captcha"))
except Exception:
logger.exception("Fatal error encountered during client run")
exit_status = 1
client.prevent_close()
client.print("Fatal error encountered:\n")
client.print(traceback.format_exc())
finally:
logger.info("=== Starting shutdown sequence ===")
if sys.platform == "linux":
logger.debug("Removing signal handlers (Linux)")
loop.remove_signal_handler(signal.SIGINT)
loop.remove_signal_handler(signal.SIGTERM)
logger.info("Notifying client of exit")
client.print(_("gui", "status", "exiting"))
# Shutdown web server
if web_server_task and not web_server_task.done():
logger.info("Shutting down web server")
# Trigger graceful shutdown and wait for it to finish
await webapp.shutdown_server()
# Wait for server to actually exit (with timeout)
try:
await asyncio.wait_for(web_server_task, timeout=5.0)
logger.info("Web server task completed gracefully")
except asyncio.TimeoutError:
logger.warning("Web server didn't exit in time, forcing cancellation")
web_server_task.cancel()
try:
await web_server_task
except asyncio.CancelledError:
logger.info("Web server task force-cancelled")
except Exception as e:
logger.error(f"Error while shutting down web server: {e}")
else:
logger.debug(f"Web server task status: task={web_server_task is not None}, done={web_server_task.done() if web_server_task else 'N/A'}")
logger.info("Shutting down Twitch client")
await client.shutdown()
logger.info("Twitch client shutdown completed")
logger.info(f"Shutdown complete - close_requested={client.gui.close_requested}, exit_status={exit_status}")
if not client.gui.close_requested:
logger.warning("User didn't request closure - showing error state")
# user didn't request the closure
client.gui.tray.change_icon("error")
client.print(_("status", "terminated"))
client.gui.status.update(_("gui", "status", "terminated"))
# notify the user about the closure
client.gui.grab_attention(sound=True)
# Wait for user to close the GUI window (only needed if close wasn't already requested)
logger.info("Waiting for GUI to close")
await client.gui.wait_until_closed()
logger.info("GUI closed by user")
else:
logger.info("Close already requested - skipping GUI wait")
# save the application state
# NOTE: we have to do it after wait_until_closed,
# because the user can alter some settings between app termination and closing the window
logger.info("Saving application state")
client.save(force=True)
logger.info("Application state saved")
logger.info("Stopping GUI")
client.gui.stop()
logger.info("GUI stopped")
logger.info("Closing GUI window")
client.gui.close_window()
logger.info(f"=== Exiting with status code: {exit_status} ===")
sys.exit(exit_status)
asyncio.run(main())
# try:
# # use lock_file to check if we're not already running
# success, file = lock_file(LOCK_PATH)
# if not success:
# # already running - exit
# sys.exit(3)
# asyncio.run(main())
# finally:
# file.close()

16
src/api/__init__.py Normal file
View File

@@ -0,0 +1,16 @@
"""
API client modules for Twitch API communication.
This package provides HTTP and GraphQL client implementations for interacting
with Twitch's API endpoints.
"""
from __future__ import annotations
from src.api.http_client import HTTPClient
from src.api.gql_client import GQLClient
__all__ = [
"HTTPClient",
"GQLClient",
]

220
src/api/gql_client.py Normal file
View File

@@ -0,0 +1,220 @@
"""
GraphQL client for Twitch GQL API.
Handles GraphQL requests with rate limiting, error handling, and retry logic.
"""
from __future__ import annotations
import asyncio
import logging
from typing import overload, TYPE_CHECKING
from itertools import chain
from src.utils import RateLimiter, ExponentialBackoff
from src.exceptions import GQLException, MinerException
from src.config import GQL_OPERATIONS
if TYPE_CHECKING:
from src.config import JsonType, GQLOperation, ClientInfo
from src.api.http_client import HTTPClient
from src.auth import _AuthState
logger = logging.getLogger("TwitchDrops")
gql_logger = logging.getLogger("TwitchDrops.gql")
class GQLClient:
"""
GraphQL client for Twitch GQL API.
This client provides:
- Rate-limited GraphQL requests to prevent API bans
- Automatic retry with exponential backoff on errors
- Error handling for various GQL error types
- Support for batched requests
- Data merging utilities for campaign data
"""
def __init__(
self,
http_client: HTTPClient,
auth_state: _AuthState,
client_type: ClientInfo,
):
"""
Initialize the GraphQL client.
Parameters
----------
http_client : HTTPClient
The HTTP client for making requests
auth_state : _AuthState
Authentication state manager
client_type : ClientInfo
Client type information (User-Agent, Client-ID, etc.)
"""
self.http_client = http_client
self._auth_state = auth_state
self._client_type = client_type
# NOTE: GQL is volatile and breaks everything if rate limited.
# Do not modify these safe defaults.
self._qgl_limiter = RateLimiter(capacity=5, window=1)
@overload
async def request(self, ops: GQLOperation) -> JsonType:
...
@overload
async def request(self, ops: list[GQLOperation]) -> list[JsonType]:
...
async def request(
self, ops: GQLOperation | list[GQLOperation]
) -> JsonType | list[JsonType]:
"""
Execute one or more GraphQL operations.
Parameters
----------
ops : GQLOperation | list[GQLOperation]
Single operation or list of operations to execute
Returns
-------
JsonType | list[JsonType]
Response data for the operation(s)
Raises
------
GQLException
If the GQL API returns an error that can't be handled
RuntimeError
If the retry loop is broken unexpectedly
"""
gql_logger.debug(f"GQL Request: {ops}")
backoff = ExponentialBackoff(maximum=60)
# Flag to retry the request once for specific errors
single_retry: bool = True
for delay in backoff:
async with self._qgl_limiter:
auth_state = await self._auth_state.validate()
async with self.http_client.request(
"POST",
"https://gql.twitch.tv/gql",
json=ops,
headers=auth_state.headers(
user_agent=self._client_type.USER_AGENT, gql=True
),
) as response:
response_json: JsonType | list[JsonType] = await response.json()
gql_logger.debug(f"GQL Response: {response_json}")
orig_response = response_json
# Normalize to list for unified error handling
if isinstance(response_json, list):
response_list = response_json
else:
response_list = [response_json]
force_retry: bool = False
for response_json in response_list:
# GQL error handling
if "errors" in response_json:
for error_dict in response_json["errors"]:
if "message" in error_dict:
if (
single_retry
and error_dict["message"]
in (
"service error",
"PersistedQueryNotFound",
)
):
logger.error(
f"Retrying a {error_dict['message']} for "
f"{response_json['extensions']['operationName']}"
)
single_retry = False
if delay < 5:
# Overwrite delay if too short
delay = 5
force_retry = True
break
elif error_dict["message"] == "server error":
# Nullify the key the error path points to
data_dict: JsonType = response_json["data"]
path: list[str] = error_dict.get("path", [])
for key in path[:-1]:
data_dict = data_dict[key]
data_dict[path[-1]] = None
break
elif error_dict["message"] in (
"service timeout",
"service unavailable",
"context deadline exceeded",
):
force_retry = True
break
else:
raise GQLException(response_json["errors"])
# Other error handling
elif "error" in response_json:
raise GQLException(
f"{response_json['error']}: {response_json['message']}"
)
if force_retry:
break
else:
return orig_response
await asyncio.sleep(delay)
raise RuntimeError("Retry loop was broken")
@staticmethod
def merge_data(primary_data: JsonType, secondary_data: JsonType) -> JsonType:
"""
Recursively merge two JSON objects, preferring primary data.
This is used to merge campaign data from inventory and general campaigns endpoints.
Parameters
----------
primary_data : JsonType
Primary data source (takes precedence)
secondary_data : JsonType
Secondary data source (used when key missing in primary)
Returns
-------
JsonType
Merged data dictionary
Raises
------
MinerException
If data types are inconsistent between sources
"""
merged = {}
for key in set(chain(primary_data.keys(), secondary_data.keys())):
in_primary = key in primary_data
if in_primary and key in secondary_data:
vp = primary_data[key]
vs = secondary_data[key]
if not isinstance(vp, type(vs)) or not isinstance(vs, type(vp)):
raise MinerException("Inconsistent merge data")
if isinstance(vp, dict): # Both are dicts
merged[key] = GQLClient.merge_data(vp, vs)
else:
# Use primary value
merged[key] = vp
elif in_primary:
merged[key] = primary_data[key]
else: # In secondary only
merged[key] = secondary_data[key]
return merged

228
src/api/http_client.py Normal file
View File

@@ -0,0 +1,228 @@
"""
HTTP client for Twitch API requests.
Handles HTTP session management, request retries, and connection quality settings.
"""
from __future__ import annotations
import asyncio
import logging
from datetime import datetime, timedelta, timezone
from contextlib import asynccontextmanager
from typing import TYPE_CHECKING
from collections import abc
import aiohttp
from yarl import URL
from src.i18n import _
from src.exceptions import ExitRequest, RequestInvalid
from src.utils import ExponentialBackoff
from src.config import COOKIES_PATH
if TYPE_CHECKING:
from src.config.settings import Settings
from src.web.gui_manager import WebGUIManager
from src.config import ClientInfo
logger = logging.getLogger("TwitchDrops")
class HTTPClient:
"""
Manages HTTP session and handles request retries with exponential backoff.
This client provides:
- Session management with cookie persistence
- Automatic request retries on connection errors
- Connection quality-based timeout configuration
- Proxy support
"""
def __init__(
self,
settings: Settings,
gui: WebGUIManager,
client_type: ClientInfo,
):
"""
Initialize the HTTP client.
Parameters
----------
settings : Settings
Application settings for connection quality and proxy configuration
gui : WebGUIManager
GUI manager for user notifications and close detection
client_type : ClientInfo
Client type information (User-Agent, Client-ID, etc.)
"""
self.settings = settings
self.gui = gui
self._client_type = client_type
self._session: aiohttp.ClientSession | None = None
async def get_session(self) -> aiohttp.ClientSession:
"""
Get or create the HTTP session.
Returns
-------
aiohttp.ClientSession
The active HTTP session
Raises
------
RuntimeError
If the session is closed
"""
if (session := self._session) is not None:
if session.closed:
raise RuntimeError("Session is closed")
return session
# Load cookies
cookie_jar = aiohttp.CookieJar()
try:
if COOKIES_PATH.exists():
cookie_jar.load(COOKIES_PATH)
except Exception:
# If loading cookies fails, clear the jar and continue
cookie_jar.clear()
# Create timeouts based on connection quality
connection_quality = self.settings.connection_quality
if connection_quality < 1:
connection_quality = self.settings.connection_quality = 1
elif connection_quality > 6:
connection_quality = self.settings.connection_quality = 6
timeout = aiohttp.ClientTimeout(
sock_connect=5 * connection_quality,
total=10 * connection_quality,
)
# Create session with connection pooling
connector = aiohttp.TCPConnector(limit=50)
self._session = aiohttp.ClientSession(
timeout=timeout,
connector=connector,
cookie_jar=cookie_jar,
headers={"User-Agent": self._client_type.USER_AGENT},
)
return self._session
@asynccontextmanager
async def request(
self,
method: str,
url: URL | str,
*,
invalidate_after: datetime | None = None,
**kwargs,
) -> abc.AsyncIterator[aiohttp.ClientResponse]:
"""
Make an HTTP request with automatic retries.
Parameters
----------
method : str
HTTP method (GET, POST, etc.)
url : URL | str
Request URL
invalidate_after : datetime | None, optional
Datetime after which the request should not be retried
**kwargs
Additional arguments passed to aiohttp.ClientSession.request
Yields
------
aiohttp.ClientResponse
The HTTP response
Raises
------
ExitRequest
If the application is closing
RequestInvalid
If the request expires during retry loop
aiohttp.ClientConnectorCertificateError
If SSL verification fails
"""
session = await self.get_session()
method = method.upper()
if self.settings.proxy and "proxy" not in kwargs:
kwargs["proxy"] = self.settings.proxy
logger.debug(f"Request: ({method=}, {url=}, {kwargs=})")
session_timeout = timedelta(seconds=session.timeout.total or 0)
backoff = ExponentialBackoff(maximum=3 * 60)
for delay in backoff:
if self.gui.close_requested:
raise ExitRequest()
elif (
invalidate_after is not None
# Account for expiration landing during the request
and datetime.now(timezone.utc) >= (invalidate_after - session_timeout)
):
raise RequestInvalid()
try:
response: aiohttp.ClientResponse | None = None
response = await self.gui.coro_unless_closed(
session.request(method, url, **kwargs)
)
assert response is not None
logger.debug(f"Response: {response.status}: {response}")
if response.status < 500:
# Pre-read the response to avoid getting errors outside the context manager
raw_response = await response.read() # noqa: F841
yield response
return
self.gui.print(_("error", "site_down").format(seconds=round(delay)))
except aiohttp.ClientConnectorCertificateError:
# SSL verification failures should not be retried
raise
except (
aiohttp.ClientConnectionError,
asyncio.TimeoutError,
aiohttp.ClientPayloadError,
):
# Connection problems, retry with backoff
if backoff.steps > 1:
# Don't show quick retries to the user
self.gui.print(_("error", "no_connection").format(seconds=round(delay)))
finally:
if response is not None:
response.release()
# Wait for the backoff delay or until the GUI closes
with asyncio.suppress(asyncio.TimeoutError):
await asyncio.wait_for(self.gui.wait_until_closed(), timeout=delay)
async def close(self) -> None:
"""
Close the HTTP session and save cookies.
This should be called during application shutdown.
"""
if self._session is not None:
cookie_jar = self._session.cookie_jar
assert isinstance(cookie_jar, aiohttp.CookieJar)
# Clear empty cookie entries before saving
# NOTE: Unfortunately, aiohttp provides no easy way of clearing empty cookies,
# so we need to access the private '_cookies' attribute
for cookie_key, cookie in list(cookie_jar._cookies.items()):
if not cookie:
del cookie_jar._cookies[cookie_key]
cookie_jar.save(COOKIES_PATH)
await self._session.close()
self._session = None

5
src/auth/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Authentication module for Twitch Drops Miner."""
from .auth_state import _AuthState
__all__ = ["_AuthState"]

453
src/auth/auth_state.py Normal file
View File

@@ -0,0 +1,453 @@
"""Authentication state management for Twitch Drops Miner."""
from __future__ import annotations
import asyncio
import logging
from typing import TYPE_CHECKING, cast
import aiohttp
from yarl import URL
from src.config import COOKIES_PATH
from src.exceptions import CaptchaRequired, LoginException
from src.i18n import _
from src.utils import CHARS_HEX_LOWER, create_nonce
if TYPE_CHECKING:
from src.core.client import Twitch
from src.web.gui_manager import LoginForm, LoginFormManager
from src.config import ClientInfo, JsonType
logger = logging.getLogger("TwitchDrops")
class _AuthState:
"""
Manages authentication state including tokens, session, and login flow.
This class handles:
- OAuth device code flow for authentication
- Legacy password-based login (deprecated)
- Access token validation and management
- Session and device ID management
- Cookie persistence
"""
def __init__(self, twitch: Twitch):
self._twitch: Twitch = twitch
self._lock = asyncio.Lock()
self._logged_in = asyncio.Event()
self.user_id: int
self.device_id: str
self.session_id: str
self.access_token: str
self.client_version: str
def _hasattrs(self, *attrs: str) -> bool:
"""Check if all specified attributes exist."""
return all(hasattr(self, attr) for attr in attrs)
def _delattrs(self, *attrs: str) -> None:
"""Delete all specified attributes if they exist."""
for attr in attrs:
if hasattr(self, attr):
delattr(self, attr)
def clear(self) -> None:
"""Clear all authentication state."""
self._delattrs(
"user_id",
"device_id",
"session_id",
"access_token",
"client_version",
)
self._logged_in.clear()
async def _oauth_login(self) -> str:
"""
Perform OAuth device code flow authentication.
This implements the OAuth 2.0 Device Authorization Grant flow:
1. Request device code and user code from Twitch
2. Display code to user for entry at twitch.tv/activate
3. Poll token endpoint until user completes authorization
4. Return access token
Returns:
str: The access token
"""
login_form: LoginForm = self._twitch.gui.login
client_info: ClientInfo = self._twitch._client_type
headers = {
"Accept": "application/json",
"Accept-Encoding": "gzip",
"Accept-Language": "en-US",
"Cache-Control": "no-cache",
"Client-Id": client_info.CLIENT_ID,
"Host": "id.twitch.tv",
"Origin": str(client_info.CLIENT_URL),
"Pragma": "no-cache",
"Referer": str(client_info.CLIENT_URL),
"User-Agent": client_info.USER_AGENT,
"X-Device-Id": self.device_id,
}
payload = {
"client_id": client_info.CLIENT_ID,
"scopes": "", # no scopes needed
}
while True:
try:
from datetime import datetime, timedelta, timezone
from src.exceptions import RequestInvalid
now = datetime.now(timezone.utc)
async with self._twitch.request(
"POST", "https://id.twitch.tv/oauth2/device", headers=headers, data=payload
) as response:
# {
# "device_code": "40 chars [A-Za-z0-9]",
# "expires_in": 1800,
# "interval": 5,
# "user_code": "8 chars [A-Z]",
# "verification_uri": "https://www.twitch.tv/activate?device-code=ABCDEFGH"
# }
response_json: JsonType = await response.json()
device_code: str = response_json["device_code"]
user_code: str = response_json["user_code"]
interval: int = response_json["interval"]
verification_uri: URL = URL(response_json["verification_uri"])
expires_at = now + timedelta(seconds=response_json["expires_in"])
# Print the code to the user, open them the activate page so they can type it in
await login_form.ask_enter_code(verification_uri, user_code)
payload = {
"client_id": self._twitch._client_type.CLIENT_ID,
"device_code": device_code,
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
}
while True:
# sleep first, not like the user is gonna enter the code *that* fast
await asyncio.sleep(interval)
async with self._twitch.request(
"POST",
"https://id.twitch.tv/oauth2/token",
headers=headers,
data=payload,
invalidate_after=expires_at,
) as response:
# 200 means success, 400 means the user haven't entered the code yet
if response.status != 200:
continue
response_json = await response.json()
# {
# "access_token": "40 chars [A-Za-z0-9]",
# "refresh_token": "40 chars [A-Za-z0-9]",
# "scope": [...],
# "token_type": "bearer"
# }
self.access_token = cast(str, response_json["access_token"])
return self.access_token
except RequestInvalid:
# the device_code has expired, request a new code
continue
async def _login(self) -> str:
"""
Perform legacy password-based login flow.
This method implements the old Twitch login API flow using username/password.
It handles:
- Username/password authentication
- Two-factor authentication (TOTP and email codes)
- CAPTCHA detection
- Error handling and user feedback
NOTE: This flow is deprecated and may trigger CAPTCHA or be blocked by Twitch.
OAuth device code flow (_oauth_login) is the preferred method.
Returns:
str: The access token
Raises:
CaptchaRequired: When CAPTCHA is detected
LoginException: On login failure
"""
logger.info("Login flow started")
gui_print = self._twitch.gui.print
login_form: LoginFormManager = self._twitch.gui.login
client_info: ClientInfo = self._twitch._client_type
token_kind: str = ''
use_chrome: bool = False
payload: JsonType = {
# username and password are added later
# "username": str,
# "password": str,
# client ID to-be associated with the access token
"client_id": client_info.CLIENT_ID,
"undelete_user": False, # purpose unknown
"remember_me": True, # persist the session via the cookie
# "authy_token": str, # 2FA token
# "twitchguard_code": str, # email code
# "captcha": str, # self-fed captcha
# 'force_twitchguard': False, # force email code confirmation
}
def _safe_loads(s: str):
"""JSON loads that skips extra data after the first valid JSON object."""
import json
class SkipExtraJsonDecoder(json.JSONDecoder):
def decode(self, s: str, *args):
# skip whitespace check
obj, end = self.raw_decode(s)
return obj
return json.loads(s, cls=SkipExtraJsonDecoder)
while True:
login_data = await login_form.ask_login()
payload["username"] = login_data.username
payload["password"] = login_data.password
# reinstate the 2FA token, if present
payload.pop("authy_token", None)
payload.pop("twitchguard_code", None)
if login_data.token:
# if there's no token kind set yet, and the user has entered a token,
# we can immediately assume it's an authenticator token and not an email one
if not token_kind:
token_kind = "authy"
if token_kind == "authy":
payload["authy_token"] = login_data.token
elif token_kind == "email":
payload["twitchguard_code"] = login_data.token
# use fancy headers to mimic the twitch android app
headers = {
"Accept": "application/vnd.twitchtv.v3+json",
"Accept-Encoding": "gzip",
"Accept-Language": "en-US",
"Client-Id": client_info.CLIENT_ID,
"Content-Type": "application/json; charset=UTF-8",
"Host": "passport.twitch.tv",
"User-Agent": client_info.USER_AGENT,
"X-Device-Id": self.device_id,
# "X-Device-Id": ''.join(random.choices('0123456789abcdef', k=32)),
}
async with self._twitch.request(
"POST", "https://passport.twitch.tv/login", headers=headers, json=payload
) as response:
login_response: JsonType = await response.json(loads=_safe_loads)
# Feed this back in to avoid running into CAPTCHA if possible
if "captcha_proof" in login_response:
payload["captcha"] = {"proof": login_response["captcha_proof"]}
# Error handling
if "error_code" in login_response:
error_code: int = login_response["error_code"]
logger.info(f"Login error code: {error_code}")
if error_code == 1000:
logger.info("1000: CAPTCHA is required")
use_chrome = True
break
elif error_code in (2004, 3001):
logger.info("3001: Login failed due to incorrect username or password")
gui_print(_("login", "incorrect_login_pass"))
if error_code == 2004:
# invalid username
login_form.clear(login=True)
login_form.clear(password=True)
continue
elif error_code in (
3012, # Invalid authy token
3023, # Invalid email code
):
logger.info("3012/23: Login failed due to incorrect 2FA code")
if error_code == 3023:
token_kind = "email"
gui_print(_("login", "incorrect_email_code"))
else:
token_kind = "authy"
gui_print(_("login", "incorrect_twofa_code"))
login_form.clear(token=True)
continue
elif error_code in (
3011, # Authy token needed
3022, # Email code needed
):
# 2FA handling
logger.info("3011/22: 2FA token required")
# user didn't provide a token, so ask them for it
if error_code == 3022:
token_kind = "email"
gui_print(_("login", "email_code_required"))
else:
token_kind = "authy"
gui_print(_("login", "twofa_code_required"))
continue
elif error_code >= 5000:
# Special errors, usually from Twitch telling the user to "go away"
# We print the code out to inform the user, and just use chrome flow instead
# {
# "error_code":5023,
# "error":"Please update your app to continue",
# "error_description":"client is not supported for this feature"
# }
# {
# "error_code":5027,
# "error":"Please update your app to continue",
# "error_description":"client blocked from this operation"
# }
gui_print(_("login", "error_code").format(error_code=error_code))
logger.info(str(login_response))
use_chrome = True
break
else:
ext_msg = str(login_response)
logger.info(ext_msg)
raise LoginException(ext_msg)
# Success handling
if "access_token" in login_response:
self.access_token = cast(str, login_response["access_token"])
logger.info("Access token granted")
login_form.clear()
break
if use_chrome:
# await self._chrome_login()
raise CaptchaRequired()
if hasattr(self, "access_token"):
return self.access_token
raise LoginException("Login flow finished without setting the access token")
def headers(self, *, user_agent: str = '', gql: bool = False) -> JsonType:
"""
Build HTTP headers for Twitch API requests.
Args:
user_agent: Optional custom User-Agent string
gql: If True, include GraphQL-specific headers
Returns:
Dictionary of HTTP headers
"""
client_info: ClientInfo = self._twitch._client_type
headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip",
"Accept-Language": "en-US",
"Pragma": "no-cache",
"Cache-Control": "no-cache",
"Client-Id": client_info.CLIENT_ID,
}
if user_agent:
headers["User-Agent"] = user_agent
if hasattr(self, "session_id"):
headers["Client-Session-Id"] = self.session_id
# if hasattr(self, "client_version"):
# headers["Client-Version"] = self.client_version
if hasattr(self, "device_id"):
headers["X-Device-Id"] = self.device_id
if gql:
headers["Origin"] = str(client_info.CLIENT_URL)
headers["Referer"] = str(client_info.CLIENT_URL)
headers["Authorization"] = f"OAuth {self.access_token}"
return headers
async def validate(self):
"""Thread-safe wrapper for _validate()."""
async with self._lock:
await self._validate()
return self
async def _validate(self):
"""
Validate and restore authentication state.
This method:
1. Generates session ID if needed
2. Extracts device ID from Twitch cookies
3. Validates existing access token or initiates login flow
4. Ensures token client ID matches expected client
5. Saves validated cookies to disk
Raises:
RuntimeError: On repeated validation failures
"""
if not hasattr(self, "session_id"):
self.session_id = create_nonce(CHARS_HEX_LOWER, 16)
if not self._hasattrs("device_id", "access_token", "user_id"):
session = await self._twitch.get_session()
jar = cast(aiohttp.CookieJar, session.cookie_jar)
client_info: ClientInfo = self._twitch._client_type
if not self._hasattrs("device_id"):
async with self._twitch.request(
"GET", client_info.CLIENT_URL, headers=self.headers()
) as response:
page_html = await response.text("utf8")
assert page_html is not None
# match = re.search(r'twilightBuildID="([-a-z0-9]+)"', page_html)
# if match is None:
# raise MinerException("Unable to extract client_version")
# self.client_version = match.group(1)
# doing the request ends up setting the "unique_id" value in the cookie
cookie = jar.filter_cookies(client_info.CLIENT_URL)
self.device_id = cookie["unique_id"].value
if not self._hasattrs("access_token", "user_id"):
# looks like we're missing something
login_form: LoginForm = self._twitch.gui.login
logger.info("Checking login")
login_form.update(_("gui", "login", "logging_in"), None)
for client_mismatch_attempt in range(2):
for invalid_token_attempt in range(2):
cookie = jar.filter_cookies(client_info.CLIENT_URL)
if "auth-token" not in cookie:
self.access_token = await self._oauth_login()
cookie["auth-token"] = self.access_token
elif not hasattr(self, "access_token"):
logger.info("Restoring session from cookie")
self.access_token = cookie["auth-token"].value
# validate the auth token, by obtaining user_id
async with self._twitch.request(
"GET",
"https://id.twitch.tv/oauth2/validate",
headers={"Authorization": f"OAuth {self.access_token}"}
) as response:
if response.status == 401:
# the access token we have is invalid - clear the cookie and reauth
logger.info("Restored session is invalid")
assert client_info.CLIENT_URL.host is not None
jar.clear_domain(client_info.CLIENT_URL.host)
continue
elif response.status == 200:
validate_response = await response.json()
break
else:
raise RuntimeError("Login verification failure (step #2)")
# ensure the cookie's client ID matches the currently selected client
if validate_response["client_id"] == client_info.CLIENT_ID:
break
# otherwise, we need to delete the entire cookie file and clear the jar
logger.info("Cookie client ID mismatch")
jar.clear()
COOKIES_PATH.unlink(missing_ok=True)
else:
raise RuntimeError("Login verification failure (step #1)")
self.user_id = int(validate_response["user_id"])
cookie["persistent"] = str(self.user_id)
logger.info(f"Login successful, user ID: {self.user_id}")
login_form.update(_("gui", "login", "logged_in"), self.user_id)
# update our cookie and save it
jar.update_cookies(cookie, client_info.CLIENT_URL)
jar.save(COOKIES_PATH)
self._logged_in.set()
def invalidate(self):
"""Invalidate the current access token."""
self._delattrs("access_token")

113
src/config/__init__.py Normal file
View File

@@ -0,0 +1,113 @@
"""Configuration package for Twitch Drops Miner."""
from __future__ import annotations
# Re-export all public symbols for convenience
from .constants import (
CALL,
FILE_FORMATTER,
OUTPUT_FORMATTER,
LOGGING_LEVELS,
State,
WebsocketTopic,
WEBSOCKET_TOPICS,
JsonType,
URLType,
TopicProcess,
GQLOperation,
MAX_INT,
MAX_EXTRA_MINUTES,
BASE_TOPICS,
MAX_WEBSOCKETS,
WS_TOPICS_LIMIT,
TOPICS_PER_CHANNEL,
MAX_TOPICS,
MAX_CHANNELS,
DEFAULT_LANG,
PING_INTERVAL,
PING_TIMEOUT,
ONLINE_DELAY,
WATCH_INTERVAL,
WINDOW_TITLE,
)
from .paths import (
IS_APPIMAGE,
IS_PACKAGED,
IS_DOCKER,
SYS_SITE_PACKAGES,
SYS_SCRIPTS,
SELF_PATH,
WORKING_DIR,
DATA_DIR,
VENV_PATH,
SITE_PACKAGES_PATH,
SCRIPTS_PATH,
LANG_PATH,
LOG_PATH,
DUMP_PATH,
LOCK_PATH,
CACHE_PATH,
CACHE_DB,
COOKIES_PATH,
SETTINGS_PATH,
_resource_path,
_merge_vars,
)
from .client_info import ClientInfo, ClientType
from .operations import GQL_OPERATIONS
__all__ = [
# constants.py
"CALL",
"FILE_FORMATTER",
"OUTPUT_FORMATTER",
"LOGGING_LEVELS",
"State",
"WebsocketTopic",
"WEBSOCKET_TOPICS",
"JsonType",
"URLType",
"TopicProcess",
"GQLOperation",
"MAX_INT",
"MAX_EXTRA_MINUTES",
"BASE_TOPICS",
"MAX_WEBSOCKETS",
"WS_TOPICS_LIMIT",
"TOPICS_PER_CHANNEL",
"MAX_TOPICS",
"MAX_CHANNELS",
"DEFAULT_LANG",
"PING_INTERVAL",
"PING_TIMEOUT",
"ONLINE_DELAY",
"WATCH_INTERVAL",
"WINDOW_TITLE",
# paths.py
"IS_APPIMAGE",
"IS_PACKAGED",
"IS_DOCKER",
"SYS_SITE_PACKAGES",
"SYS_SCRIPTS",
"SELF_PATH",
"WORKING_DIR",
"DATA_DIR",
"VENV_PATH",
"SITE_PACKAGES_PATH",
"SCRIPTS_PATH",
"LANG_PATH",
"LOG_PATH",
"DUMP_PATH",
"LOCK_PATH",
"CACHE_PATH",
"CACHE_DB",
"COOKIES_PATH",
"SETTINGS_PATH",
"_resource_path",
"_merge_vars",
# client_info.py
"ClientInfo",
"ClientType",
# operations.py
"GQL_OPERATIONS",
]

113
src/config/client_info.py Normal file
View File

@@ -0,0 +1,113 @@
"""Client configuration for Twitch API interactions."""
from __future__ import annotations
import random
from yarl import URL
class ClientInfo:
"""Client configuration including URL, ID, and User-Agent."""
def __init__(self, client_url: URL, client_id: str, user_agents: str | list[str]) -> None:
self.CLIENT_URL: URL = client_url
self.CLIENT_ID: str = client_id
self.USER_AGENT: str
if isinstance(user_agents, list):
self.USER_AGENT = random.choice(user_agents)
else:
self.USER_AGENT = user_agents
def __iter__(self):
return iter((self.CLIENT_URL, self.CLIENT_ID, self.USER_AGENT))
class ClientType:
"""Predefined client configurations for different Twitch platforms."""
WEB = ClientInfo(
URL("https://www.twitch.tv"),
"kimne78kx3ncx6brgo4mv6wki5h1ko",
(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
),
)
MOBILE_WEB = ClientInfo(
URL("https://m.twitch.tv"),
"r8s4dac0uhzifbpu9sjdiwzctle17ff",
[
# Chrome versioning is done fully on android only,
# other platforms only use the major version
(
"Mozilla/5.0 (Linux; Android 16) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-A205U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-A102U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-G960U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; SM-N960U) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; LM-Q720) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
(
"Mozilla/5.0 (Linux; Android 16; LM-X420) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.7204.158 Mobile Safari/537.36"
),
]
)
ANDROID_APP = ClientInfo(
URL("https://www.twitch.tv"),
"kd1unb4b3q4t58fwlpcbzcbnm76a8fp",
[
(
"Dalvik/2.1.0 (Linux; U; Android 16; SM-S911B Build/TP1A.220624.014) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 16; SM-S938B Build/BP2A.250605.031) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; Android 16; SM-X716N Build/UP1A.231005.007) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-G990B Build/AP3A.240905.015.A2) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-G970F Build/AP3A.241105.008) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 15; SM-A566E Build/AP3A.240905.015.A2) "
"tv.twitch.android.app/25.3.0/2503006"
),
(
"Dalvik/2.1.0 (Linux; U; Android 14; SM-X306B Build/UP1A.231005.007) "
"tv.twitch.android.app/25.3.0/2503006"
),
]
)
SMARTBOX = ClientInfo(
URL("https://android.tv.twitch.tv"),
"ue6666qo983tsx6so1t0vnawi233wa",
(
"Mozilla/5.0 (Linux; Android 7.1; Smart Box C1) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
),
)

160
src/config/constants.py Normal file
View File

@@ -0,0 +1,160 @@
"""Core constants, enums, and type definitions for Twitch Drops Miner."""
from __future__ import annotations
import sys
import logging
from enum import Enum, auto
from datetime import timedelta
from typing import Any, Dict, Literal, NewType, TYPE_CHECKING
from copy import deepcopy
from src.version import __version__
if TYPE_CHECKING:
from collections import abc # noqa
from typing_extensions import TypeAlias
# Logging special levels
CALL: int = logging.INFO - 1
logging.addLevelName(CALL, "CALL")
# Logging configuration
LOGGING_LEVELS = {
0: logging.ERROR,
1: logging.WARNING,
2: logging.INFO,
3: CALL,
4: logging.DEBUG,
}
FILE_FORMATTER = logging.Formatter(
"{asctime}.{msecs:03.0f}:\t{levelname:>7}:\t{filename}:{lineno}:\t{message}",
style='{',
datefmt="%Y-%m-%d %H:%M:%S",
)
OUTPUT_FORMATTER = logging.Formatter("{levelname}: {message}", style='{', datefmt="%H:%M:%S")
# Type aliases
JsonType = Dict[str, Any]
URLType = NewType("URLType", str)
TopicProcess: TypeAlias = "abc.Callable[[int, JsonType], Any]"
# Core constants
MAX_INT = sys.maxsize
MAX_EXTRA_MINUTES = 15
BASE_TOPICS = 2
MAX_WEBSOCKETS = 8
WS_TOPICS_LIMIT = 50
TOPICS_PER_CHANNEL = 2
MAX_TOPICS = (MAX_WEBSOCKETS * WS_TOPICS_LIMIT) - BASE_TOPICS
MAX_CHANNELS = MAX_TOPICS // TOPICS_PER_CHANNEL
# Misc
DEFAULT_LANG = "English"
# Intervals and Delays
PING_INTERVAL = timedelta(minutes=3)
PING_TIMEOUT = timedelta(seconds=10)
ONLINE_DELAY = timedelta(seconds=120)
WATCH_INTERVAL = timedelta(seconds=59)
# Strings
WINDOW_TITLE = f"Twitch Drops Miner v{__version__} (by DevilXD)"
class State(Enum):
"""Application state machine states."""
IDLE = auto()
INVENTORY_FETCH = auto()
GAMES_UPDATE = auto()
CHANNELS_FETCH = auto()
CHANNELS_CLEANUP = auto()
CHANNEL_SWITCH = auto()
EXIT = auto()
class GQLOperation(JsonType):
"""GraphQL operation with persisted query hash."""
def __init__(self, name: str, sha256: str, *, variables: JsonType | None = None):
super().__init__(
operationName=name,
extensions={
"persistedQuery": {
"version": 1,
"sha256Hash": sha256,
}
}
)
if variables is not None:
self.__setitem__("variables", variables)
def with_variables(self, variables: JsonType) -> GQLOperation:
"""Create a copy with merged variables."""
from .paths import _merge_vars
modified = deepcopy(self)
if "variables" in self:
existing_variables: JsonType = modified["variables"]
_merge_vars(existing_variables, variables)
else:
modified["variables"] = variables
return modified
class WebsocketTopic:
"""Represents a websocket topic subscription."""
def __init__(
self,
category: Literal["User", "Channel"],
topic_name: str,
target_id: int,
process: TopicProcess,
):
assert isinstance(target_id, int)
self._id: str = self.as_str(category, topic_name, target_id)
self._target_id = target_id
self._process: TopicProcess = process
@classmethod
def as_str(
cls, category: Literal["User", "Channel"], topic_name: str, target_id: int
) -> str:
return f"{WEBSOCKET_TOPICS[category][topic_name]}.{target_id}"
def __call__(self, message: JsonType):
return self._process(self._target_id, message)
def __str__(self) -> str:
return self._id
def __repr__(self) -> str:
return f"Topic({self._id})"
def __eq__(self, other) -> bool:
if isinstance(other, WebsocketTopic):
return self._id == other._id
elif isinstance(other, str):
return self._id == other
return NotImplemented
def __hash__(self) -> int:
return hash((self.__class__.__name__, self._id))
WEBSOCKET_TOPICS: dict[str, dict[str, str]] = {
"User": { # Using user_id
"Presence": "presence", # unused
"Drops": "user-drop-events",
"Notifications": "onsite-notifications",
"CommunityPoints": "community-points-user-v1",
},
"Channel": { # Using channel_id
"Drops": "channel-drop-events", # unused
"StreamState": "video-playback-by-id",
"StreamUpdate": "broadcast-settings-update",
"CommunityPoints": "community-points-channel-v1", # unused
},
}

157
src/config/operations.py Normal file
View File

@@ -0,0 +1,157 @@
"""GraphQL operations for Twitch API interactions."""
from __future__ import annotations
from .constants import GQLOperation
GQL_OPERATIONS: dict[str, GQLOperation] = {
# returns stream information for a particular channel
"GetStreamInfo": GQLOperation(
"VideoPlayerStreamInfoOverlayChannel",
"198492e0857f6aedead9665c81c5a06d67b25b58034649687124083ff288597d",
variables={
"channel": ..., # channel login
},
),
# can be used to claim channel points
"ClaimCommunityPoints": GQLOperation(
"ClaimCommunityPoints",
"46aaeebe02c99afdf4fc97c7c0cba964124bf6b0af229395f1f6d1feed05b3d0",
variables={
"input": {
"claimID": ..., # points claim_id
"channelID": ..., # channel ID as a str
},
},
),
# can be used to claim a drop
"ClaimDrop": GQLOperation(
"DropsPage_ClaimDropRewards",
"a455deea71bdc9015b78eb49f4acfbce8baa7ccbedd28e549bb025bd0f751930",
variables={
"input": {
"dropInstanceID": ..., # drop claim_id
},
},
),
# returns current state of points (balance, claim available) for a particular channel
"ChannelPointsContext": GQLOperation(
"ChannelPointsContext",
"374314de591e69925fce3ddc2bcf085796f56ebb8cad67a0daa3165c03adc345",
variables={
"channelLogin": ..., # channel login
},
),
# returns all in-progress campaigns
"Inventory": GQLOperation(
"Inventory",
"d86775d0ef16a63a33ad52e80eaff963b2d5b72fada7c991504a57496e1d8e4b",
variables={
"fetchRewardCampaigns": False,
}
),
# returns current state of drops (current drop progress)
"CurrentDrop": GQLOperation(
"DropCurrentSessionContext",
"4d06b702d25d652afb9ef835d2a550031f1cf762b193523a92166f40ea3d142b",
variables={
"channelID": ..., # watched channel ID as a str
"channelLogin": "", # always empty string
},
),
# returns all available campaigns
"Campaigns": GQLOperation(
"ViewerDropsDashboard",
"5a4da2ab3d5b47c9f9ce864e727b2cb346af1e3ea8b897fe8f704a97ff017619",
variables={
"fetchRewardCampaigns": False,
}
),
# returns extended information about a particular campaign
"CampaignDetails": GQLOperation(
"DropCampaignDetails",
"039277bf98f3130929262cc7c6efd9c141ca3749cb6dca442fc8ead9a53f77c1",
variables={
"channelLogin": ..., # user login
"dropID": ..., # campaign ID
},
),
# returns drops available for a particular channel
"AvailableDrops": GQLOperation(
"DropsHighlightService_AvailableDrops",
"9a62a09bce5b53e26e64a671e530bc599cb6aab1e5ba3cbd5d85966d3940716f",
variables={
"channelID": ..., # channel ID as a str
},
),
# retuns stream playback access token
"PlaybackAccessToken": GQLOperation(
"PlaybackAccessToken",
"ed230aa1e33e07eebb8928504583da78a5173989fadfb1ac94be06a04f3cdbe9",
variables={
"isLive": True,
"isVod": False,
"login": ..., # channel login
"platform": "web",
"playerType": "site",
"vodID": "",
},
),
# returns live channels for a particular game
"GameDirectory": GQLOperation(
"DirectoryPage_Game",
"98a996c3c3ebb1ba4fd65d6671c6028d7ee8d615cb540b0731b3db2a911d3649",
variables={
"limit": 30, # limit of channels returned
"slug": ..., # game slug
"imageWidth": 50,
"includeCostreaming": False,
"options": {
"broadcasterLanguages": [],
"freeformTags": None,
"includeRestricted": ["SUB_ONLY_LIVE"],
"recommendationsContext": {"platform": "web"},
"sort": "RELEVANCE", # also accepted: "VIEWER_COUNT"
"systemFilters": [],
"tags": [],
"requestID": "JIRA-VXP-2397",
},
"sortTypeIsRecency": False,
},
),
"SlugRedirect": GQLOperation( # can be used to turn game name -> game slug
"DirectoryGameRedirect",
"1f0300090caceec51f33c5e20647aceff9017f740f223c3c532ba6fa59f6b6cc",
variables={
"name": ..., # game name
},
),
"NotificationsView": GQLOperation( # unused, triggers notifications "update-summary"
"OnsiteNotifications_View",
"e8e06193f8df73d04a1260df318585d1bd7a7bb447afa058e52095513f2bfa4f",
variables={
"input": {},
},
),
"NotificationsList": GQLOperation( # unused
"OnsiteNotifications_ListNotifications",
"11cdb54a2706c2c0b2969769907675680f02a6e77d8afe79a749180ad16bfea6",
variables={
"cursor": "",
"displayType": "VIEWER",
"language": "en",
"limit": 10,
"shouldLoadLastBroadcast": False,
},
),
"NotificationsDelete": GQLOperation(
"OnsiteNotifications_DeleteNotification",
"13d463c831f28ffe17dccf55b3148ed8b3edbbd0ebadd56352f1ff0160616816",
variables={
"input": {
"id": "", # ID of the notification to delete
}
},
),
}

121
src/config/paths.py Normal file
View File

@@ -0,0 +1,121 @@
"""Path-related configuration and environment detection."""
from __future__ import annotations
import os
import sys
from pathlib import Path
from typing import Any, Dict
# Type alias for path operations
JsonType = Dict[str, Any]
# Environment detection
IS_APPIMAGE = "APPIMAGE" in os.environ and os.path.exists(os.environ["APPIMAGE"])
IS_PACKAGED = hasattr(sys, "_MEIPASS") or IS_APPIMAGE
IS_DOCKER = os.getenv("DOCKER_ENV") == "1" or os.path.exists("/.dockerenv")
# Site-packages venv path changes depending on the system platform
if sys.platform == "win32":
SYS_SITE_PACKAGES = "Lib/site-packages"
else:
# On Linux, the site-packages path includes a versioned 'pythonX.Y' folder part
# The Lib folder is also spelled in lowercase: 'lib'
version_info = sys.version_info
SYS_SITE_PACKAGES = f"lib/python{version_info.major}.{version_info.minor}/site-packages"
# Scripts venv path changes depending on the system platform
if sys.platform == "win32":
SYS_SCRIPTS = "Scripts"
else:
SYS_SCRIPTS = "bin"
def _resource_path(relative_path: Path | str) -> Path:
"""
Get an absolute path to a bundled resource.
Works for dev and for PyInstaller.
"""
if IS_APPIMAGE:
base_path = Path(sys.argv[0]).resolve().parent
elif IS_PACKAGED:
# PyInstaller's folder where the one-file app is unpacked
meipass: str = getattr(sys, "_MEIPASS")
base_path = Path(meipass)
else:
base_path = WORKING_DIR
return base_path.joinpath(relative_path)
def _merge_vars(base_vars: JsonType, vars: JsonType) -> None:
"""
Merge variables recursively.
NOTE: This modifies base_vars in place.
"""
for k, v in vars.items():
if k not in base_vars:
base_vars[k] = v
elif isinstance(v, dict):
if isinstance(base_vars[k], dict):
_merge_vars(base_vars[k], v)
elif base_vars[k] is Ellipsis:
# unspecified base, use the passed in var
base_vars[k] = v
else:
raise RuntimeError(f"Var is a dict, base is not: '{k}'")
elif isinstance(base_vars[k], dict):
raise RuntimeError(f"Base is a dict, var is not: '{k}'")
else:
# simple overwrite
base_vars[k] = v
# ensure none of the vars are ellipsis (unset value)
for k, v in base_vars.items():
if v is Ellipsis:
raise RuntimeError(f"Unspecified variable: '{k}'")
# Base Paths - environment-specific resolution
if IS_DOCKER:
# Docker environment: use fixed paths
SELF_PATH = Path("/app/main.py")
WORKING_DIR = Path("/app")
DATA_DIR = Path("/app/data")
elif IS_APPIMAGE:
SELF_PATH = Path(os.environ["APPIMAGE"]).resolve()
WORKING_DIR = SELF_PATH.parent
DATA_DIR = WORKING_DIR
else:
# NOTE: pyinstaller will set sys.argv[0] to its own executable when building
# NOTE: sys.argv[0] will point to gui.py when running the gui.py directly for GUI debug
# detect these and use __file__ and main.py redirection instead
SELF_PATH = Path(sys.argv[0]).resolve()
if SELF_PATH.stem == "pyinstaller" or SELF_PATH.name == "gui.py":
SELF_PATH = Path(__file__).with_name("main.py").resolve()
WORKING_DIR = SELF_PATH.parent
DATA_DIR = WORKING_DIR
# Ensure data directory exists in Docker
if IS_DOCKER and not DATA_DIR.exists():
DATA_DIR.mkdir(parents=True, exist_ok=True)
# Development paths
VENV_PATH = Path(WORKING_DIR, "env")
SITE_PACKAGES_PATH = Path(VENV_PATH, SYS_SITE_PACKAGES)
SCRIPTS_PATH = Path(VENV_PATH, SYS_SCRIPTS)
# Translations path
# NOTE: These don't have to be available to the end-user, so the path points to the internal dir
LANG_PATH = _resource_path("lang")
# Persistent storage paths - use DATA_DIR for Docker compatibility
LOG_PATH = Path(DATA_DIR, "log.txt")
DUMP_PATH = Path(DATA_DIR, "dump.dat")
LOCK_PATH = Path(DATA_DIR, "lock.file")
CACHE_PATH = Path(DATA_DIR, "cache")
CACHE_DB = Path(CACHE_PATH, "mapping.json")
COOKIES_PATH = Path(DATA_DIR, "cookies.jar")
SETTINGS_PATH = Path(DATA_DIR, "settings.json")

16
settings.py → src/config/settings.py
View File

@@ -4,8 +4,8 @@ from typing import Any, TypedDict, TYPE_CHECKING
from yarl import URL from yarl import URL
from utils import json_load, json_save from src.utils import json_load, json_save
from constants import SETTINGS_PATH, DEFAULT_LANG, PriorityMode from src.config import SETTINGS_PATH, DEFAULT_LANG
if TYPE_CHECKING: if TYPE_CHECKING:
from main import ParsedArgs from main import ParsedArgs
@@ -15,24 +15,20 @@ class SettingsFile(TypedDict):
proxy: URL proxy: URL
language: str language: str
dark_mode: bool dark_mode: bool
exclude: set[str] games_to_watch: list[str]
priority: list[str]
autostart_tray: bool autostart_tray: bool
connection_quality: int connection_quality: int
tray_notifications: bool tray_notifications: bool
priority_mode: PriorityMode
default_settings: SettingsFile = { default_settings: SettingsFile = {
"proxy": URL(), "proxy": URL(),
"priority": [], "games_to_watch": [],
"exclude": set(),
"dark_mode": False, "dark_mode": False,
"autostart_tray": False, "autostart_tray": False,
"connection_quality": 1, "connection_quality": 1,
"language": DEFAULT_LANG, "language": DEFAULT_LANG,
"tray_notifications": True, "tray_notifications": True,
"priority_mode": PriorityMode.PRIORITY_ONLY,
} }
@@ -49,12 +45,10 @@ class Settings:
proxy: URL proxy: URL
language: str language: str
dark_mode: bool dark_mode: bool
exclude: set[str] games_to_watch: list[str]
priority: list[str]
autostart_tray: bool autostart_tray: bool
connection_quality: int connection_quality: int
tray_notifications: bool tray_notifications: bool
priority_mode: PriorityMode
PASSTHROUGH = ("_settings", "_args", "_altered") PASSTHROUGH = ("_settings", "_args", "_altered")

0
src/core/__init__.py Normal file
View File

971
twitch.py → src/core/client.py
View File

File diff suppressed because it is too large Load Diff

0
exceptions.py → src/exceptions.py
View File

59
src/i18n/__init__.py Normal file
View File

@@ -0,0 +1,59 @@
"""Internationalization (i18n) package for Twitch Drops Miner."""
from __future__ import annotations
from .translator import (
StatusMessages,
ChromeMessages,
LoginMessages,
ErrorMessages,
GUIStatus,
GUITabs,
GUITray,
GUILoginForm,
GUIWebsocket,
GUIProgress,
GUIChannelHeadings,
GUIChannels,
GUIInvFilter,
GUIInvStatus,
GUIInventory,
GUISettingsGeneral,
GUIPriorityModes,
GUISettings,
GUIHelpLinks,
GUIHelp,
GUIMessages,
Translation,
default_translation,
Translator,
_,
)
__all__ = [
"StatusMessages",
"ChromeMessages",
"LoginMessages",
"ErrorMessages",
"GUIStatus",
"GUITabs",
"GUITray",
"GUILoginForm",
"GUIWebsocket",
"GUIProgress",
"GUIChannelHeadings",
"GUIChannels",
"GUIInvFilter",
"GUIInvStatus",
"GUIInventory",
"GUISettingsGeneral",
"GUIPriorityModes",
"GUISettings",
"GUIHelpLinks",
"GUIHelp",
"GUIMessages",
"Translation",
"default_translation",
"Translator",
"_",
]

8
translate.py → src/i18n/translator.py
View File

@@ -3,9 +3,9 @@ from __future__ import annotations
from collections import abc from collections import abc
from typing import Any, TypedDict, TYPE_CHECKING from typing import Any, TypedDict, TYPE_CHECKING
from exceptions import MinerException from src.exceptions import MinerException
from utils import json_load, json_save from src.utils.json_utils import json_load, json_save
from constants import IS_PACKAGED, LANG_PATH, DEFAULT_LANG from src.config import IS_PACKAGED, LANG_PATH, DEFAULT_LANG
if TYPE_CHECKING: if TYPE_CHECKING:
from typing_extensions import NotRequired from typing_extensions import NotRequired
@@ -48,6 +48,7 @@ class ErrorMessages(TypedDict):
class GUIStatus(TypedDict): class GUIStatus(TypedDict):
name: str name: str
idle: str idle: str
ready: str
exiting: str exiting: str
terminated: str terminated: str
cleanup: str cleanup: str
@@ -262,6 +263,7 @@ default_translation: Translation = {
"status": { "status": {
"name": "Status", "name": "Status",
"idle": "Idle", "idle": "Idle",
"ready": "Ready",
"exiting": "Exiting...", "exiting": "Exiting...",
"terminated": "Terminated", "terminated": "Terminated",
"cleanup": "Cleaning up channels...", "cleanup": "Cleaning up channels...",

19
src/models/__init__.py Normal file
View File

@@ -0,0 +1,19 @@
"""Domain models for Twitch drops mining."""
from src.models.game import Game
from src.models.benefit import Benefit, BenefitType
from src.models.drop import BaseDrop, TimedDrop, remove_dimensions
from src.models.campaign import DropsCampaign
from src.models.channel import Channel, Stream
__all__ = [
"Game",
"Benefit",
"BenefitType",
"BaseDrop",
"TimedDrop",
"remove_dimensions",
"DropsCampaign",
"Channel",
"Stream",
]

34
src/models/benefit.py Normal file
View File

@@ -0,0 +1,34 @@
from __future__ import annotations
from enum import Enum
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.config.constants import JsonType, URLType
class BenefitType(Enum):
"""Type of drop benefit (reward)."""
UNKNOWN = "UNKNOWN"
BADGE = "BADGE"
EMOTE = "EMOTE"
DIRECT_ENTITLEMENT = "DIRECT_ENTITLEMENT"
def is_badge_or_emote(self) -> bool:
return self in (BenefitType.BADGE, BenefitType.EMOTE)
class Benefit:
"""Represents a reward/benefit from a completed drop."""
__slots__ = ("id", "name", "type", "image_url")
def __init__(self, data: JsonType):
benefit_data: JsonType = data["benefit"]
self.id: str = benefit_data["id"]
self.name: str = benefit_data["name"]
self.type: BenefitType = (
BenefitType(benefit_data["distributionType"])
if benefit_data["distributionType"] in BenefitType.__members__.keys()
else BenefitType.UNKNOWN
)
self.image_url: URLType = benefit_data["imageAssetURL"]

199
src/models/campaign.py Normal file
View File

@@ -0,0 +1,199 @@
from __future__ import annotations
import logging
from itertools import chain
from typing import TYPE_CHECKING
from functools import cached_property
from datetime import datetime, timezone
from dateutil.parser import isoparse
from src.models.channel import Channel
from src.models.game import Game
from src.models.drop import TimedDrop, remove_dimensions
from src.config.constants import URLType, State
if TYPE_CHECKING:
from collections import abc
from src.core.client import Twitch
from src.config.constants import JsonType
logger = logging.getLogger("TwitchDrops")
class DropsCampaign:
def __init__(self, twitch: Twitch, data: JsonType, claimed_benefits: dict[str, datetime]):
self._twitch: Twitch = twitch
self.id: str = data["id"]
self.name: str = data["name"]
self.game: Game = Game(data["game"])
self.linked: bool = data["self"]["isAccountConnected"]
self.link_url: str = data["accountLinkURL"]
# campaign's image actually comes from the game object
# we use regex to get rid of the dimensions part (ex. ".../game_id-285x380.jpg")
self.image_url: URLType = remove_dimensions(data["game"]["boxArtURL"])
self.starts_at: datetime = isoparse(data["startAt"])
self.ends_at: datetime = isoparse(data["endAt"])
self._valid: bool = data["status"] != "EXPIRED"
allowed: JsonType = data["allow"]
self.allowed_channels: list[Channel] = (
[Channel.from_acl(twitch, channel_data) for channel_data in allowed["channels"]]
if allowed["channels"] and allowed.get("isEnabled", True) else []
)
self.timed_drops: dict[str, TimedDrop] = {
drop_data["id"]: TimedDrop(self, drop_data, claimed_benefits)
for drop_data in data["timeBasedDrops"]
}
def __repr__(self) -> str:
return f"Campaign({self.game!s}, {self.name}, {self.claimed_drops}/{self.total_drops})"
@property
def drops(self) -> abc.Iterable[TimedDrop]:
return self.timed_drops.values()
@property
def time_triggers(self) -> set[datetime]:
return set(
chain(
(self.starts_at, self.ends_at),
*((d.starts_at, d.ends_at) for d in self.timed_drops.values()),
)
)
@property
def active(self) -> bool:
return self._valid and self.starts_at <= datetime.now(timezone.utc) < self.ends_at
@property
def upcoming(self) -> bool:
return self._valid and datetime.now(timezone.utc) < self.starts_at
@property
def expired(self) -> bool:
return not self._valid or self.ends_at <= datetime.now(timezone.utc)
@property
def total_drops(self) -> int:
return len(self.timed_drops)
@property
def eligible(self) -> bool:
return self.linked or self.has_badge_or_emote
@cached_property
def has_badge_or_emote(self) -> bool:
return any(
benefit.type.is_badge_or_emote() for drop in self.drops for benefit in drop.benefits
)
@property
def finished(self) -> bool:
return all(d.is_claimed or d.required_minutes <= 0 for d in self.drops)
@property
def claimed_drops(self) -> int:
return sum(d.is_claimed for d in self.drops)
@property
def remaining_drops(self) -> int:
return sum(not d.is_claimed for d in self.drops)
@property
def required_minutes(self) -> int:
return max(d.total_required_minutes for d in self.drops)
@property
def remaining_minutes(self) -> int:
return max(d.total_remaining_minutes for d in self.drops)
@property
def progress(self) -> float:
return sum(d.progress for d in self.drops) / self.total_drops
@property
def availability(self) -> float:
return min(d.availability for d in self.drops)
@property
def first_drop(self) -> TimedDrop | None:
drops: list[TimedDrop] = sorted(
(drop for drop in self.drops if drop.can_earn()),
key=lambda d: d.remaining_minutes,
)
return drops[0] if drops else None
def _update_real_minutes(self, delta: int) -> None:
for drop in self.drops:
drop._update_real_minutes(delta)
if (first_drop := self.first_drop) is not None:
first_drop.display()
def _base_can_earn(
self, channel: Channel | None = None, ignore_channel_status: bool = False
) -> bool:
return (
self.eligible # account is eligible
and self.active # campaign is active (and valid)
and (
channel is None or ( # channel isn't specified,
# or there's no ACL, or the channel is in the ACL
(not self.allowed_channels or channel in self.allowed_channels)
# and the channel is live and playing the campaign's game
and (
ignore_channel_status
or channel.game is not None and channel.game == self.game
)
)
)
)
def get_drop(self, drop_id: str) -> TimedDrop | None:
"""Get a specific drop by ID from this campaign."""
return self.timed_drops.get(drop_id)
def preconditions_chain(self) -> set[str]:
"""Return all drop IDs that are preconditions for unclaimed drops."""
return set(
chain.from_iterable(
drop.precondition_drops for drop in self.drops if not drop.is_claimed
)
)
def can_earn(
self, channel: Channel | None = None, ignore_channel_status: bool = False
) -> bool:
# True if any of the containing drops can be earned
return (
self._base_can_earn(channel, ignore_channel_status)
and any(drop._base_can_earn() for drop in self.drops)
)
def can_earn_within(self, stamp: datetime) -> bool:
# Same as can_earn, but doesn't check the channel
# and uses a future timestamp to see if we can earn this campaign later
return (
self.eligible
and self._valid
and self.ends_at > datetime.now(timezone.utc)
and self.starts_at < stamp
and any(drop._can_earn_within(stamp) for drop in self.drops)
)
def bump_minutes(self, channel: Channel) -> None:
"""
Bump the minute counter for all earnable drops in this campaign.
Used when websocket updates aren't available.
"""
# NOTE: Use a temporary list to ensure all drops are bumped before checking
if any([drop._bump_minutes(channel) for drop in self.drops]):
# Executes if any drop's extra_current_minutes reach MAX_ESTIMATED_MINUTES
# TODO: Figure out a better way to handle this case
logger.warning(
f"At least one of the drops in campaign \"{self.name}({self.game.name})\" "
"has reached the maximum extra minutes limit!"
)
self._twitch.change_state(State.CHANNEL_SWITCH)
if (first_drop := self.first_drop) is not None:
first_drop.display()

25
channel.py → src/models/channel.py
View File

@@ -11,14 +11,15 @@ from typing import Any, SupportsInt, cast, TYPE_CHECKING
import aiohttp import aiohttp
from yarl import URL from yarl import URL
from utils import Game, json_minify from src.models.game import Game
from exceptions import MinerException, RequestException from src.config.constants import JsonType, GQLOperation, URLType, CALL, ONLINE_DELAY
from constants import CALL, GQL_OPERATIONS, ONLINE_DELAY, URLType from src.config.operations import GQL_OPERATIONS
from src.utils.json_utils import json_minify
from src.exceptions import MinerException, RequestException
if TYPE_CHECKING: if TYPE_CHECKING:
from twitch import Twitch from src.core.client import Twitch
from gui import ChannelList from src.web.gui_manager import ChannelList
from constants import JsonType, GQLOperation
logger = logging.getLogger("TwitchDrops") logger = logging.getLogger("TwitchDrops")
@@ -269,10 +270,12 @@ class Channel:
return self._stream.drops_enabled return self._stream.drops_enabled
return False return False
def display(self, *, add: bool = False): def display(self, *, add: bool = False) -> None:
"""Display or update this channel in the GUI channel list."""
self._gui_channels.display(self, add=add) self._gui_channels.display(self, add=add)
def remove(self): def remove(self) -> None:
"""Remove this channel from the GUI and cancel pending tasks."""
if self._pending_stream_up is not None: if self._pending_stream_up is not None:
self._pending_stream_up.cancel() self._pending_stream_up.cancel()
self._pending_stream_up = None self._pending_stream_up = None
@@ -315,7 +318,7 @@ class Channel:
for campaign_data in available_drops for campaign_data in available_drops
) )
def external_update(self, channel_data: JsonType, available_drops: list[JsonType]): def external_update(self, channel_data: JsonType, available_drops: list[JsonType]) -> None:
""" """
Update stream information based on data provided externally. Update stream information based on data provided externally.
@@ -375,7 +378,7 @@ class Channel:
self._pending_stream_up = None # for 'display' to work properly self._pending_stream_up = None # for 'display' to work properly
await self.update_stream() await self.update_stream()
def check_online(self): def check_online(self) -> None:
""" """
Sets up a task that will wait ONLINE_DELAY duration, Sets up a task that will wait ONLINE_DELAY duration,
and then check for the stream being ONLINE OR OFFLINE. and then check for the stream being ONLINE OR OFFLINE.
@@ -392,7 +395,7 @@ class Channel:
self._pending_stream_up = asyncio.create_task(self._online_delay()) self._pending_stream_up = asyncio.create_task(self._online_delay())
self.display() self.display()
def set_offline(self): def set_offline(self) -> None:
""" """
Sets the channel status to OFFLINE. Cancels PENDING_ONLINE if applicable. Sets the channel status to OFFLINE. Cancels PENDING_ONLINE if applicable.

238
inventory.py → src/models/drop.py
View File

@@ -1,58 +1,31 @@
from __future__ import annotations from __future__ import annotations
import re import re
import math
import logging import logging
from enum import Enum
from itertools import chain
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from functools import cached_property
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from dateutil.parser import isoparse
from translate import _ from src.i18n import _
from channel import Channel from src.models.benefit import Benefit
from utils import timestamp, Game from src.exceptions import GQLException
from exceptions import GQLException from src.config.constants import MAX_EXTRA_MINUTES, State
from constants import GQL_OPERATIONS, MAX_EXTRA_MINUTES, URLType, State from src.config.operations import GQL_OPERATIONS
if TYPE_CHECKING: if TYPE_CHECKING:
from collections import abc from src.core.client import Twitch
from src.models.channel import Channel
from twitch import Twitch from src.models.campaign import DropsCampaign
from constants import JsonType from src.config.constants import JsonType
logger = logging.getLogger("TwitchDrops") logger = logging.getLogger("TwitchDrops")
DIMS_PATTERN = re.compile(r'-\d+x\d+(?=\.(?:jpg|png|gif)$)', re.I) DIMS_PATTERN = re.compile(r'-\d+x\d+(?=\.(?:jpg|png|gif)$)', re.I)
def remove_dimensions(url: URLType) -> URLType: def remove_dimensions(url: str) -> str:
return URLType(DIMS_PATTERN.sub('', url)) """Remove dimension suffix from Twitch image URLs (e.g., -285x380.jpg)."""
return DIMS_PATTERN.sub('', url)
class BenefitType(Enum):
UNKNOWN = "UNKNOWN"
BADGE = "BADGE"
EMOTE = "EMOTE"
DIRECT_ENTITLEMENT = "DIRECT_ENTITLEMENT"
def is_badge_or_emote(self) -> bool:
return self in (BenefitType.BADGE, BenefitType.EMOTE)
class Benefit:
__slots__ = ("id", "name", "type", "image_url")
def __init__(self, data: JsonType):
benefit_data: JsonType = data["benefit"]
self.id: str = benefit_data["id"]
self.name: str = benefit_data["name"]
self.type: BenefitType = (
BenefitType(benefit_data["distributionType"])
if benefit_data["distributionType"] in BenefitType.__members__.keys()
else BenefitType.UNKNOWN
)
self.image_url: URLType = benefit_data["imageAssetURL"]
class BaseDrop: class BaseDrop:
@@ -64,8 +37,8 @@ class BaseDrop:
self.name: str = data["name"] self.name: str = data["name"]
self.campaign: DropsCampaign = campaign self.campaign: DropsCampaign = campaign
self.benefits: list[Benefit] = [Benefit(b) for b in (data["benefitEdges"] or [])] self.benefits: list[Benefit] = [Benefit(b) for b in (data["benefitEdges"] or [])]
self.starts_at: datetime = timestamp(data["startAt"]) self.starts_at: datetime = isoparse(data["startAt"])
self.ends_at: datetime = timestamp(data["endAt"]) self.ends_at: datetime = isoparse(data["endAt"])
self.claim_id: str | None = None self.claim_id: str | None = None
self.is_claimed: bool = False self.is_claimed: bool = False
if "self" in data: if "self" in data:
@@ -150,7 +123,8 @@ class BaseDrop:
and datetime.now(timezone.utc) < self.campaign.ends_at + timedelta(hours=24) and datetime.now(timezone.utc) < self.campaign.ends_at + timedelta(hours=24)
) )
def update_claim(self, claim_id: str): def update_claim(self, claim_id: str) -> None:
"""Update the claim ID for this drop."""
self.claim_id = claim_id self.claim_id = claim_id
async def generate_claim(self) -> None: async def generate_claim(self) -> None:
@@ -280,6 +254,7 @@ class TimedDrop(BaseDrop):
@property @property
def availability(self) -> float: def availability(self) -> float:
import math
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
if self.required_minutes > 0 and self.total_remaining_minutes > 0 and now < self.ends_at: if self.required_minutes > 0 and self.total_remaining_minutes > 0 and now < self.ends_at:
return ((self.ends_at - now).total_seconds() / 60) / self.total_remaining_minutes return ((self.ends_at - now).total_seconds() / 60) / self.total_remaining_minutes
@@ -323,10 +298,12 @@ class TimedDrop(BaseDrop):
self._on_state_changed() self._on_state_changed()
return result return result
def display(self, *, countdown: bool = True, subone: bool = False): def display(self, *, countdown: bool = True, subone: bool = False) -> None:
"""Display this drop in the GUI with progress information."""
self._twitch.gui.display_drop(self, countdown=countdown, subone=subone) self._twitch.gui.display_drop(self, countdown=countdown, subone=subone)
def update_minutes(self, new_minutes: int): def update_minutes(self, new_minutes: int) -> None:
"""Update the current watched minutes for this drop."""
delta: int = new_minutes - self.real_current_minutes delta: int = new_minutes - self.real_current_minutes
if delta == 0: if delta == 0:
return return
@@ -335,174 +312,3 @@ class TimedDrop(BaseDrop):
elif self.real_current_minutes + delta > self.required_minutes: elif self.real_current_minutes + delta > self.required_minutes:
delta = self.required_minutes - self.real_current_minutes delta = self.required_minutes - self.real_current_minutes
self.campaign._update_real_minutes(delta) self.campaign._update_real_minutes(delta)
class DropsCampaign:
def __init__(self, twitch: Twitch, data: JsonType, claimed_benefits: dict[str, datetime]):
self._twitch: Twitch = twitch
self.id: str = data["id"]
self.name: str = data["name"]
self.game: Game = Game(data["game"])
self.linked: bool = data["self"]["isAccountConnected"]
self.link_url: str = data["accountLinkURL"]
# campaign's image actually comes from the game object
# we use regex to get rid of the dimensions part (ex. ".../game_id-285x380.jpg")
self.image_url: URLType = remove_dimensions(data["game"]["boxArtURL"])
self.starts_at: datetime = timestamp(data["startAt"])
self.ends_at: datetime = timestamp(data["endAt"])
self._valid: bool = data["status"] != "EXPIRED"
allowed: JsonType = data["allow"]
self.allowed_channels: list[Channel] = (
[Channel.from_acl(twitch, channel_data) for channel_data in allowed["channels"]]
if allowed["channels"] and allowed.get("isEnabled", True) else []
)
self.timed_drops: dict[str, TimedDrop] = {
drop_data["id"]: TimedDrop(self, drop_data, claimed_benefits)
for drop_data in data["timeBasedDrops"]
}
def __repr__(self) -> str:
return f"Campaign({self.game!s}, {self.name}, {self.claimed_drops}/{self.total_drops})"
@property
def drops(self) -> abc.Iterable[TimedDrop]:
return self.timed_drops.values()
@property
def time_triggers(self) -> set[datetime]:
return set(
chain(
(self.starts_at, self.ends_at),
*((d.starts_at, d.ends_at) for d in self.timed_drops.values()),
)
)
@property
def active(self) -> bool:
return self._valid and self.starts_at <= datetime.now(timezone.utc) < self.ends_at
@property
def upcoming(self) -> bool:
return self._valid and datetime.now(timezone.utc) < self.starts_at
@property
def expired(self) -> bool:
return not self._valid or self.ends_at <= datetime.now(timezone.utc)
@property
def total_drops(self) -> int:
return len(self.timed_drops)
@property
def eligible(self) -> bool:
return self.linked or self.has_badge_or_emote
@cached_property
def has_badge_or_emote(self) -> bool:
return any(
benefit.type.is_badge_or_emote() for drop in self.drops for benefit in drop.benefits
)
@property
def finished(self) -> bool:
return all(d.is_claimed or d.required_minutes <= 0 for d in self.drops)
@property
def claimed_drops(self) -> int:
return sum(d.is_claimed for d in self.drops)
@property
def remaining_drops(self) -> int:
return sum(not d.is_claimed for d in self.drops)
@property
def required_minutes(self) -> int:
return max(d.total_required_minutes for d in self.drops)
@property
def remaining_minutes(self) -> int:
return max(d.total_remaining_minutes for d in self.drops)
@property
def progress(self) -> float:
return sum(d.progress for d in self.drops) / self.total_drops
@property
def availability(self) -> float:
return min(d.availability for d in self.drops)
@property
def first_drop(self) -> TimedDrop | None:
drops: list[TimedDrop] = sorted(
(drop for drop in self.drops if drop.can_earn()),
key=lambda d: d.remaining_minutes,
)
return drops[0] if drops else None
def _update_real_minutes(self, delta: int) -> None:
for drop in self.drops:
drop._update_real_minutes(delta)
if (first_drop := self.first_drop) is not None:
first_drop.display()
def _base_can_earn(
self, channel: Channel | None = None, ignore_channel_status: bool = False
) -> bool:
return (
self.eligible # account is eligible
and self.active # campaign is active (and valid)
and (
channel is None or ( # channel isn't specified,
# or there's no ACL, or the channel is in the ACL
(not self.allowed_channels or channel in self.allowed_channels)
# and the channel is live and playing the campaign's game
and (
ignore_channel_status
or channel.game is not None and channel.game == self.game
)
)
)
)
def get_drop(self, drop_id: str) -> TimedDrop | None:
return self.timed_drops.get(drop_id)
def preconditions_chain(self) -> set[str]:
return set(
chain.from_iterable(
drop.precondition_drops for drop in self.drops if not drop.is_claimed
)
)
def can_earn(
self, channel: Channel | None = None, ignore_channel_status: bool = False
) -> bool:
# True if any of the containing drops can be earned
return (
self._base_can_earn(channel, ignore_channel_status)
and any(drop._base_can_earn() for drop in self.drops)
)
def can_earn_within(self, stamp: datetime) -> bool:
# Same as can_earn, but doesn't check the channel
# and uses a future timestamp to see if we can earn this campaign later
return (
self.eligible
and self._valid
and self.ends_at > datetime.now(timezone.utc)
and self.starts_at < stamp
and any(drop._can_earn_within(stamp) for drop in self.drops)
)
def bump_minutes(self, channel: Channel) -> None:
# NOTE: Use a temporary list to ensure all drops are bumped before checking
if any([drop._bump_minutes(channel) for drop in self.drops]):
# Executes if any drop's extra_current_minutes reach MAX_ESTIMATED_MINUTES
# TODO: Figure out a better way to handle this case
logger.warning(
f"At least one of the drops in campaign \"{self.name}({self.game.name})\" "
"has reached the maximum extra minutes limit!"
)
self._twitch.change_state(State.CHANNEL_SWITCH)
if (first_drop := self.first_drop) is not None:
first_drop.display()

45
src/models/game.py Normal file
View File

@@ -0,0 +1,45 @@
from __future__ import annotations
import re
from functools import cached_property
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.config.constants import JsonType
class Game:
"""Represents a Twitch game/category."""
def __init__(self, data: JsonType):
self.id: int = int(data["id"])
self.name: str = data.get("displayName") or data["name"]
if "slug" in data:
self.slug = data["slug"]
def __str__(self) -> str:
return self.name
def __repr__(self) -> str:
return f"Game({self.id}, {self.name})"
def __eq__(self, other: object) -> bool:
if isinstance(other, self.__class__):
return self.id == other.id
return NotImplemented
def __hash__(self) -> int:
return self.id
@cached_property
def slug(self) -> str:
"""
Converts the game name into a slug, useable for the GQL API.
"""
# remove specific characters
slug_text = re.sub(r'\'', '', self.name.lower())
# remove non alpha-numeric characters
slug_text = re.sub(r'\W+', '-', slug_text)
# strip and collapse dashes
slug_text = re.sub(r'-{2,}', '-', slug_text.strip('-'))
return slug_text

0
src/services/__init__.py Normal file
View File

177
src/services/channel_service.py Normal file
View File

@@ -0,0 +1,177 @@
"""
Channel service for managing channel discovery, online status checks, and priority sorting.
This service handles all channel-related operations including fetching live streams
from directories, bulk online status verification, and channel priority determination.
"""
from __future__ import annotations
import asyncio
import logging
from typing import TYPE_CHECKING
from collections import abc
from src.utils import chunk
from src.config import MAX_INT, GQL_OPERATIONS
from src.models.channel import Channel
from src.exceptions import GQLException, MinerException
if TYPE_CHECKING:
from src.core.client import Twitch
from src.models.game import Game
from src.config import JsonType, GQLOperation
logger = logging.getLogger("TwitchDrops")
class ChannelService:
"""
Service responsible for channel management operations.
Handles:
- Channel priority calculation
- Live stream discovery from game directories
- Bulk online status checks for ACL channels
- Channel sorting by viewers and priority
"""
def __init__(self, twitch: Twitch) -> None:
"""
Initialize the channel service.
Args:
twitch: The Twitch client instance
"""
self._twitch = twitch
def get_priority(self, channel: Channel) -> int:
"""
Return a priority number for a given channel based on games_to_watch order.
Priority is determined by the position of the channel's game in the
wanted_games list. Lower numbers indicate higher priority.
Args:
channel: The channel to evaluate
Returns:
Priority number where:
- 0 has the highest priority (first in games_to_watch list)
- Higher numbers indicate lower priority
- MAX_INT signifies the lowest possible priority (unwanted game or offline)
"""
if (
(game := channel.game) is None # None when OFFLINE or no game set
or game not in self._twitch.wanted_games # we don't care about the played game
):
return MAX_INT
return self._twitch.wanted_games.index(game)
@staticmethod
def get_viewers_key(channel: Channel) -> int:
"""
Sort key for channels by viewer count (descending).
Args:
channel: The channel to evaluate
Returns:
Viewer count, or -1 if not available (offline channels)
"""
if (viewers := channel.viewers) is not None:
return viewers
return -1
async def get_live_streams(
self, game: Game, *, limit: int = 20, drops_enabled: bool = True
) -> list[Channel]:
"""
Fetch live streams for a specific game from Twitch directory.
Args:
game: The game to fetch streams for
limit: Maximum number of streams to return (default: 20)
drops_enabled: Only return channels with drops enabled (default: True)
Returns:
List of Channel objects representing live streams
Raises:
MinerException: If the GQL request fails
"""
filters: list[str] = []
if drops_enabled:
filters.append("DROPS_ENABLED")
try:
response = await self._twitch.gql_request(
GQL_OPERATIONS["GameDirectory"].with_variables({
"limit": limit,
"slug": game.slug,
"options": {
"includeRestricted": ["SUB_ONLY_LIVE"],
"systemFilters": filters,
},
})
)
except GQLException as exc:
raise MinerException(f"Game: {game.slug}") from exc
if "game" in response["data"]:
return [
Channel.from_directory(
self._twitch, stream_channel_data["node"], drops_enabled=drops_enabled
)
for stream_channel_data in response["data"]["game"]["streams"]["edges"]
if stream_channel_data["node"]["broadcaster"] is not None
]
return []
async def bulk_check_online(self, channels: abc.Iterable[Channel]) -> None:
"""
Utilize batch GQL requests to check ONLINE status for multiple channels at once.
This method efficiently checks the online status and drops_enabled flag
for a large number of channels by batching GraphQL requests.
Args:
channels: Iterable of Channel objects to check
"""
acl_streams_map: dict[int, JsonType] = {}
stream_gql_ops: list[GQLOperation] = [channel.stream_gql for channel in channels]
if not stream_gql_ops:
# shortcut for nothing to process
# NOTE: Have to do this here, because "channels" can be any iterable
return
stream_gql_tasks: list[asyncio.Task[list[JsonType]]] = [
asyncio.create_task(self._twitch.gql_request(stream_gql_chunk))
for stream_gql_chunk in chunk(stream_gql_ops, 20)
]
try:
for coro in asyncio.as_completed(stream_gql_tasks):
response_list: list[JsonType] = await coro
for response_json in response_list:
channel_data: JsonType = response_json["data"]["user"]
if channel_data is not None:
acl_streams_map[int(channel_data["id"])] = channel_data
except Exception:
# asyncio.as_completed doesn't cancel tasks on errors
for task in stream_gql_tasks:
task.cancel()
raise
# Update all channels with their stream data
for channel in channels:
channel_id = channel.id
if channel_id not in acl_streams_map:
continue
channel_data = acl_streams_map[channel_id]
if channel_data["stream"] is None:
continue
# Update channel with stream data (no available drops check)
channel.external_update(channel_data, [])

271
src/services/inventory_service.py Normal file
View File

@@ -0,0 +1,271 @@
"""
Inventory service for managing campaigns, drops, and inventory fetching.
This service handles fetching campaign data from Twitch's GraphQL API,
managing the inventory state, and determining active campaigns.
"""
from __future__ import annotations
import asyncio
import json
import logging
from copy import deepcopy
from datetime import datetime, timedelta, timezone
from typing import TYPE_CHECKING, Any
from dateutil.parser import isoparse
from src.utils import chunk
from src.i18n import _
from src.config import DUMP_PATH, GQL_OPERATIONS
from src.models import DropsCampaign
from src.exceptions import ExitRequest
from src.api import GQLClient
if TYPE_CHECKING:
from src.core.client import Twitch
from src.models.channel import Channel
from src.config import JsonType
logger = logging.getLogger("TwitchDrops")
class InventoryService:
"""
Service responsible for inventory and campaign management.
Handles:
- Fetching campaign details from GraphQL
- Fetching inventory (in-progress campaigns)
- Determining active campaign for a channel
- Managing campaign data and claimed benefits
"""
def __init__(self, twitch: Twitch) -> None:
"""
Initialize the inventory service.
Args:
twitch: The Twitch client instance
"""
self._twitch = twitch
async def fetch_campaigns(
self, campaigns_chunk: list[tuple[str, JsonType]]
) -> dict[str, JsonType]:
"""
Fetch detailed campaign data for a chunk of campaign IDs.
Args:
campaigns_chunk: List of (campaign_id, campaign_data) tuples
Returns:
Dictionary mapping campaign IDs to their detailed data
"""
campaign_ids: dict[str, JsonType] = dict(campaigns_chunk)
auth_state = await self._twitch.get_auth()
response_list_raw = await self._twitch.gql_request(
[
GQL_OPERATIONS["CampaignDetails"].with_variables(
{"channelLogin": str(auth_state.user_id), "dropID": cid}
)
for cid in campaign_ids
]
)
# Ensure we have a list
response_list: list[JsonType] = (
response_list_raw if isinstance(response_list_raw, list) else [response_list_raw]
)
fetched_data: dict[str, JsonType] = {
(campaign_data := response_json["data"]["user"]["dropCampaign"])["id"]: campaign_data
for response_json in response_list
}
return GQLClient.merge_data(campaign_ids, fetched_data)
async def fetch_inventory(self) -> None:
"""
Fetch the complete inventory including campaigns and drops.
This method:
1. Fetches in-progress campaigns (inventory)
2. Fetches available campaigns
3. Fetches detailed data for each campaign
4. Creates DropsCampaign objects
5. Updates GUI with campaign information
6. Sets up maintenance triggers for campaign timing changes
"""
status_update = self._twitch.gui.status.update
status_update(_("gui", "status", "fetching_inventory"))
# fetch in-progress campaigns (inventory)
response = await self._twitch.gql_request(GQL_OPERATIONS["Inventory"])
inventory: JsonType = response["data"]["currentUser"]["inventory"]
ongoing_campaigns: list[JsonType] = inventory["dropCampaignsInProgress"] or []
# this contains claimed benefit edge IDs, not drop IDs
claimed_benefits: dict[str, datetime] = {
b["id"]: isoparse(b["lastAwardedAt"])
for b in inventory["gameEventDrops"]
}
inventory_data: dict[str, JsonType] = {c["id"]: c for c in ongoing_campaigns}
# fetch general available campaigns data (campaigns)
response = await self._twitch.gql_request(GQL_OPERATIONS["Campaigns"])
available_list: list[JsonType] = response["data"]["currentUser"]["dropCampaigns"] or []
applicable_statuses = ("ACTIVE", "UPCOMING")
available_campaigns: dict[str, JsonType] = {
c["id"]: c
for c in available_list
if c["status"] in applicable_statuses # that are currently not expired
}
# fetch detailed data for each campaign, in chunks
status_update(_("gui", "status", "fetching_campaigns"))
fetch_campaigns_tasks: list[asyncio.Task[Any]] = [
asyncio.create_task(self.fetch_campaigns(campaigns_chunk))
for campaigns_chunk in chunk(available_campaigns.items(), 20)
]
try:
for coro in asyncio.as_completed(fetch_campaigns_tasks):
chunk_campaigns_data = await coro
# merge the inventory and campaigns datas together
inventory_data = GQLClient.merge_data(inventory_data, chunk_campaigns_data)
except Exception:
# asyncio.as_completed doesn't cancel tasks on errors
for task in fetch_campaigns_tasks:
task.cancel()
raise
# filter out invalid campaigns
for campaign_id in list(inventory_data.keys()):
if inventory_data[campaign_id]["game"] is None:
del inventory_data[campaign_id]
if self._twitch.settings.dump:
# dump the campaigns data to the dump file
with open(DUMP_PATH, 'a', encoding="utf8") as file:
# we need to pre-process the inventory dump a little
dump_data: JsonType = deepcopy(inventory_data)
for campaign_data in dump_data.values():
# replace ACL lists with a simple text description
if (
campaign_data["allow"]
and campaign_data["allow"].get("isEnabled", True)
and campaign_data["allow"]["channels"]
):
# simply count the channels included in the ACL
campaign_data["allow"]["channels"] = (
f"{len(campaign_data['allow']['channels'])} channels"
)
# replace drop instance IDs, so they don't include user IDs
for drop_data in campaign_data["timeBasedDrops"]:
if "self" in drop_data and drop_data["self"]["dropInstanceID"]:
drop_data["self"]["dropInstanceID"] = "..."
json.dump(dump_data, file, indent=4, sort_keys=True)
file.write("\n\n") # add 2x new line spacer
json.dump(claimed_benefits, file, indent=4, sort_keys=True, default=str)
# use the merged data to create campaign objects
campaigns: list[DropsCampaign] = [
DropsCampaign(self._twitch, campaign_data, claimed_benefits)
for campaign_data in inventory_data.values()
]
campaigns.sort(key=lambda c: c.active, reverse=True)
campaigns.sort(key=lambda c: c.upcoming and c.starts_at or c.ends_at)
campaigns.sort(key=lambda c: c.eligible, reverse=True)
self._twitch._drops.clear()
self._twitch.gui.inv.clear()
self._twitch.inventory.clear()
self._twitch._mnt_triggers.clear()
switch_triggers: set[datetime] = set()
next_hour = datetime.now(timezone.utc) + timedelta(hours=1)
# add the campaigns to the internal inventory
for campaign in campaigns:
self._twitch._drops.update({drop.id: drop for drop in campaign.drops})
if campaign.can_earn_within(next_hour):
switch_triggers.update(campaign.time_triggers)
self._twitch.inventory.append(campaign)
self._twitch._campaigns[campaign.id] = campaign
# concurrently add the campaigns into the GUI
# NOTE: this fetches pictures from the CDN, so might be slow without a cache
status_update(
_("gui", "status", "adding_campaigns").format(counter=f"(0/{len(campaigns)})")
)
add_campaign_tasks: list[asyncio.Task[None]] = [
asyncio.create_task(self._twitch.gui.inv.add_campaign(campaign))
for campaign in campaigns
]
try:
for i, coro in enumerate(asyncio.as_completed(add_campaign_tasks), start=1):
await coro
status_update(
_("gui", "status", "adding_campaigns").format(
counter=f"({i}/{len(campaigns)})"
)
)
# this is needed here explicitly, because cache reads from disk don't raise this
if self._twitch.gui.close_requested:
raise ExitRequest()
except Exception:
# asyncio.as_completed doesn't cancel tasks on errors
for task in add_campaign_tasks:
task.cancel()
raise
self._twitch._mnt_triggers.extend(sorted(switch_triggers))
# trim out all triggers that we're already past
now = datetime.now(timezone.utc)
while self._twitch._mnt_triggers and self._twitch._mnt_triggers[0] <= now:
self._twitch._mnt_triggers.popleft()
# NOTE: maintenance task is restarted at the end of each inventory fetch
if self._twitch._mnt_task is not None and not self._twitch._mnt_task.done():
self._twitch._mnt_task.cancel()
self._twitch._mnt_task = asyncio.create_task(
self._twitch._maintenance_service.run_maintenance_task()
)
def get_active_campaign(self, channel: Channel | None = None) -> DropsCampaign | None:
"""
Determine the active campaign for a given channel (or watching channel).
Returns the campaign with the least remaining minutes that can be earned
on the specified channel. This is used to determine which drop is actively
being progressed.
Args:
channel: The channel to check (defaults to watching channel)
Returns:
The active DropsCampaign, or None if no campaign can be earned
"""
if not self._twitch.wanted_games:
return None
watching_channel = self._twitch.watching_channel.get_with_default(channel)
if watching_channel is None:
# if we aren't watching anything, we can't earn any drops
return None
campaigns: list[DropsCampaign] = []
for campaign in self._twitch.inventory:
if campaign.can_earn(watching_channel):
campaigns.append(campaign)
if campaigns:
campaigns.sort(key=lambda c: c.remaining_minutes)
return campaigns[0]
return None

93
src/services/maintenance.py Normal file
View File

@@ -0,0 +1,93 @@
"""
Maintenance service for periodic inventory reloads and cleanup triggers.
This service manages scheduled tasks that trigger inventory fetches and channel cleanups
based on campaign timing (starts/ends) and hourly reload cycles.
"""
from __future__ import annotations
import asyncio
import logging
from datetime import datetime, timedelta, timezone
from typing import TYPE_CHECKING
from src.utils import task_wrapper
from src.config import CALL, State
if TYPE_CHECKING:
from src.core.client import Twitch
logger = logging.getLogger("TwitchDrops")
class MaintenanceService:
"""
Service responsible for periodic maintenance tasks.
Handles:
- Hourly inventory reloads
- Campaign-triggered channel cleanups (when drops start/end)
- Task scheduling based on time triggers
"""
def __init__(self, twitch: Twitch) -> None:
"""
Initialize the maintenance service.
Args:
twitch: The Twitch client instance
"""
self._twitch = twitch
@task_wrapper(critical=True)
async def run_maintenance_task(self) -> None:
"""
Execute the maintenance task loop.
This task monitors time triggers for channel cleanup and performs
periodic inventory reloads approximately every 60 minutes. The task
exits after each reload cycle and is restarted by fetch_inventory.
The maintenance logic:
1. Wait until the next trigger (either a campaign time trigger or next hour)
2. If the trigger is a campaign timing change, request channel cleanup
3. After reaching the next hour boundary, request inventory reload
"""
now = datetime.now(timezone.utc)
next_period = now + timedelta(minutes=1)
while True:
# exit if there's no need to repeat the loop
now = datetime.now(timezone.utc)
if now >= next_period:
break
next_trigger = next_period
while self._twitch._mnt_triggers and self._twitch._mnt_triggers[0] <= next_trigger:
next_trigger = self._twitch._mnt_triggers.popleft()
trigger_type: str = "Reload" if next_trigger == next_period else "Cleanup"
logger.log(
CALL,
(
"Maintenance task waiting until: "
f"{next_trigger.astimezone().strftime('%X')} ({trigger_type})"
)
)
await asyncio.sleep((next_trigger - now).total_seconds())
# exit after waiting, before the actions
now = datetime.now(timezone.utc)
if now >= next_period:
break
if next_trigger != next_period:
logger.log(CALL, "Maintenance task requests channels cleanup")
self._twitch.change_state(State.CHANNELS_CLEANUP)
# this triggers a restart of this task every (up to) 60 minutes
logger.log(CALL, "Maintenance task requests a reload")
self._twitch.change_state(State.INVENTORY_FETCH)

273
src/services/message_handlers.py Normal file
View File

@@ -0,0 +1,273 @@
"""
Message handler service for processing websocket updates.
This service handles all websocket message types including drop progress,
drop claims, notifications, stream state changes, and broadcast updates.
"""
from __future__ import annotations
import asyncio
import logging
from typing import TYPE_CHECKING
from src.utils import task_wrapper
from src.config import CALL, State, GQL_OPERATIONS
from src.i18n import _
if TYPE_CHECKING:
from src.core.client import Twitch
from src.models.channel import Channel, Stream
from src.models import TimedDrop
from src.config import JsonType
logger = logging.getLogger("TwitchDrops")
class MessageHandlerService:
"""
Service responsible for processing websocket messages.
Handles:
- Drop progress updates (websocket)
- Drop claim notifications (websocket)
- User notifications (websocket)
- Stream state changes (viewcount, stream-up, stream-down)
- Broadcast settings updates (game/title changes)
- Channel update callbacks
"""
def __init__(self, twitch: Twitch) -> None:
"""
Initialize the message handler service.
Args:
twitch: The Twitch client instance
"""
self._twitch = twitch
@task_wrapper
async def process_stream_state(self, channel_id: int, message: JsonType) -> None:
"""
Process websocket stream state updates (viewcount, stream-up, stream-down).
Args:
channel_id: The channel ID that sent the update
message: The websocket message payload
"""
msg_type: str = message["type"]
channel: Channel | None = self._twitch.channels.get(channel_id)
if channel is None:
logger.error(f"Stream state change for a non-existing channel: {channel_id}")
return
if msg_type == "viewcount":
if not channel.online:
# if it's not online for some reason, set it so
channel.check_online()
else:
viewers = message["viewers"]
channel.viewers = viewers
channel.display()
# logger.debug(f"{channel.name} viewers: {viewers}")
elif msg_type == "stream-down":
channel.set_offline()
elif msg_type == "stream-up":
channel.check_online()
elif msg_type == "commercial":
# skip these
pass
else:
logger.warning(f"Unknown stream state: {msg_type}")
@task_wrapper
async def process_stream_update(self, channel_id: int, message: JsonType) -> None:
"""
Process websocket broadcast settings updates (game/title changes).
Args:
channel_id: The channel ID that sent the update
message: The websocket message payload containing:
- channel_id: Channel ID string
- type: "broadcast_settings_update"
- channel: Channel login name
- old_status: Previous stream title
- status: New stream title
- old_game: Previous game name
- game: New game name
- old_game_id: Previous game ID
- game_id: New game ID
"""
channel: Channel | None = self._twitch.channels.get(channel_id)
if channel is None:
logger.error(f"Broadcast settings update for a non-existing channel: {channel_id}")
return
if message["old_game"] != message["game"]:
game_change = f", game changed: {message['old_game']} -> {message['game']}"
else:
game_change = ''
logger.log(CALL, f"Channel update from websocket: {channel.name}{game_change}")
# There's no information about channel tags here, but this event is triggered
# when the tags change. We can use this to just update the stream data after the change.
# Use 'check_online' to introduce a delay, allowing for multiple title and tags
# changes before we update. This eventually calls 'on_channel_update' below.
channel.check_online()
def on_channel_update(
self, channel: Channel, stream_before: Stream | None, stream_after: Stream | None
) -> None:
"""
Called by a Channel when its status is updated (ONLINE, OFFLINE, title/tags change).
This method determines whether a channel switch is needed based on the
status change and channel watching eligibility.
Args:
channel: The channel that was updated
stream_before: The previous stream state (None if was offline)
stream_after: The new stream state (None if now offline)
Note:
'stream_before' gets deallocated once this function finishes.
"""
watching_channel: Channel | None = self._twitch.watching_channel.get_with_default(None)
is_watching_this: bool = watching_channel is not None and watching_channel == channel
# Channel going from OFFLINE to ONLINE
if stream_before is None and stream_after is not None:
if self._twitch.can_watch(channel) and self._twitch.should_switch(channel):
self._twitch.print(_("status", "goes_online").format(channel=channel.name))
self._twitch.watch(channel)
else:
logger.info(f"{channel.name} goes ONLINE")
# Channel going from ONLINE to OFFLINE
elif stream_before is not None and stream_after is None:
if is_watching_this:
self._twitch.print(_("status", "goes_offline").format(channel=channel.name))
self._twitch.change_state(State.CHANNEL_SWITCH)
else:
logger.info(f"{channel.name} goes OFFLINE")
# Channel staying ONLINE but with updates
elif stream_before is not None and stream_after is not None:
drops_status: str = (
f"(🎁: {stream_before.drops_enabled and '✔' or '❌'} -> "
f"{stream_after.drops_enabled and '✔' or '❌'})"
)
if is_watching_this and not self._twitch.can_watch(channel):
# Watching this channel but can't watch it anymore
logger.info(f"{channel.name} status updated, switching... {drops_status}")
self._twitch.change_state(State.CHANNEL_SWITCH)
elif not is_watching_this:
# Not watching this channel
logger.info(f"{channel.name} status updated {drops_status}")
if self._twitch.can_watch(channel) and self._twitch.should_switch(channel):
self._twitch.watch(channel)
# Channel was OFFLINE and stays OFFLINE
else:
logger.log(CALL, f"{channel.name} stays OFFLINE")
channel.display()
@task_wrapper
async def process_drops(self, user_id: int, message: JsonType) -> None:
"""
Process websocket drop progress and claim updates.
Args:
user_id: The user ID that sent the message
message: The websocket message payload, examples:
- {"type": "drop-progress", data: {"current_progress_min": 3, "required_progress_min": 10}}
- {"type": "drop-claim", data: {"drop_instance_id": ...}}
"""
msg_type: str = message["type"]
if msg_type not in ("drop-progress", "drop-claim"):
return
drop_id: str = message["data"]["drop_id"]
drop: TimedDrop | None = self._twitch._drops.get(drop_id)
watching_channel: Channel | None = self._twitch.watching_channel.get_with_default(None)
if msg_type == "drop-claim":
if drop is None:
logger.error(
f"Received a drop claim ID for a non-existing drop: {drop_id}\n"
f"Drop claim ID: {message['data']['drop_instance_id']}"
)
return
drop.update_claim(message["data"]["drop_instance_id"])
campaign = drop.campaign
await drop.claim()
drop.display()
# About 4-20s after claiming the drop, next drop can be started
# by re-sending the watch payload. We can test for it by fetching the current drop
# via GQL, and then comparing drop IDs.
await asyncio.sleep(4)
if watching_channel is not None:
for attempt in range(8):
context = await self._twitch.gql_request(
GQL_OPERATIONS["CurrentDrop"].with_variables(
{"channelID": str(watching_channel.id)}
)
)
drop_data: JsonType | None = (
context["data"]["currentUser"]["dropCurrentSession"]
)
if drop_data is None or drop_data["dropID"] != drop.id:
break
await asyncio.sleep(2)
if campaign.can_earn(watching_channel):
self._twitch.restart_watching()
else:
self._twitch.change_state(State.INVENTORY_FETCH)
return
assert msg_type == "drop-progress"
if drop is not None:
drop_text = (
f"{drop.name} ({drop.campaign.game}, "
f"{message['data']['current_progress_min']}/"
f"{message['data']['required_progress_min']})"
)
else:
drop_text = "<Unknown>"
logger.log(CALL, f"Drop update from websocket: {drop_text}")
if drop is not None and drop.can_earn(self._twitch.watching_channel.get_with_default(None)):
# the received payload is for the drop we expected
drop.update_minutes(message["data"]["current_progress_min"])
@task_wrapper
async def process_notifications(self, user_id: int, message: JsonType) -> None:
"""
Process websocket notification updates.
Handles notification for drop rewards that are ready to claim.
Args:
user_id: The user ID that sent the notification
message: The websocket message payload
"""
if message["type"] == "create-notification":
data: JsonType = message["data"]["notification"]
if data["type"] == "user_drop_reward_reminder_notification":
self._twitch.change_state(State.INVENTORY_FETCH)
await self._twitch.gql_request(
GQL_OPERATIONS["NotificationsDelete"].with_variables(
{"input": {"id": data["id"]}}
)
)

256
src/services/watch_service.py Normal file
View File

@@ -0,0 +1,256 @@
"""
Watch service for managing channel watching and drop progress monitoring.
This service handles the core watching loop that sends watch payloads to Twitch,
monitors drop progress, and determines when to switch channels.
"""
from __future__ import annotations
import asyncio
import logging
from time import time
from typing import TYPE_CHECKING, NoReturn
from contextlib import suppress
from src.utils import task_wrapper, AwaitableValue
from src.i18n import _
from src.config import CALL, WATCH_INTERVAL, GQL_OPERATIONS
from src.exceptions import GQLException
if TYPE_CHECKING:
from src.core.client import Twitch
from src.models.channel import Channel
from src.models import TimedDrop
from src.config import JsonType
logger = logging.getLogger("TwitchDrops")
class WatchService:
"""
Service responsible for watching channels and monitoring drop progress.
Handles:
- Starting/stopping channel watching
- Watch loop that sends periodic watch payloads
- Drop progress monitoring via GQL and websocket
- Channel switch eligibility checks
- Watch loop sleep with restart capability
"""
def __init__(self, twitch: Twitch) -> None:
"""
Initialize the watch service.
Args:
twitch: The Twitch client instance
"""
self._twitch = twitch
def can_watch(self, channel: Channel) -> bool:
"""
Determines if the given channel qualifies as a watching candidate.
A channel can be watched if:
- There are wanted games configured
- The channel is online
- Drops are enabled on the channel
- The channel is streaming a wanted game
- At least one campaign can be progressed on this channel
Args:
channel: The channel to evaluate
Returns:
True if the channel can be watched, False otherwise
"""
if not self._twitch.wanted_games:
return False
# exit early if stream is offline or drops aren't enabled
if not channel.online or not channel.drops_enabled:
return False
# check if we can progress any campaign for the played game
if channel.game is None or channel.game not in self._twitch.wanted_games:
return False
for campaign in self._twitch.inventory:
if campaign.can_earn(channel):
return True
return False
def should_switch(self, channel: Channel) -> bool:
"""
Determines if the given channel qualifies as a switch candidate.
A channel should be switched to if:
- We're not currently watching anything
- The channel's game has higher priority than the watching channel's game
- The channel has the same game priority but is ACL-based and watching isn't
Args:
channel: The channel to evaluate as a switch candidate
Returns:
True if we should switch to this channel, False otherwise
"""
watching_channel = self._twitch.watching_channel.get_with_default(None)
if watching_channel is None:
return True
channel_order = self._twitch._channel_service.get_priority(channel)
watching_order = self._twitch._channel_service.get_priority(watching_channel)
return (
# this channel's game is higher order than the watching one's
channel_order < watching_order
or channel_order == watching_order # or the order is the same
# and this channel is ACL-based and the watching channel isn't
and channel.acl_based > watching_channel.acl_based
)
def watch(self, channel: Channel, *, update_status: bool = True) -> None:
"""
Start watching a specific channel.
Updates GUI elements and sets the watching channel. Optionally prints
a status message and updates the status bar.
Args:
channel: The channel to start watching
update_status: Whether to print status message and update status bar
"""
self._twitch.gui.tray.change_icon("active")
self._twitch.gui.channels.set_watching(channel)
self._twitch.watching_channel.set(channel)
if update_status:
status_text: str = _("status", "watching").format(channel=channel.name)
self._twitch.print(status_text)
self._twitch.gui.status.update(status_text)
def stop_watching(self) -> None:
"""
Stop watching the current channel.
Clears the watching channel and updates GUI elements.
"""
self._twitch.gui.clear_drop()
self._twitch.watching_channel.clear()
self._twitch.gui.channels.clear_watching()
def restart_watching(self) -> None:
"""
Restart the watch loop (forces immediate re-send of watch payload).
Stops the progress timer and signals the watch loop to restart.
"""
self._twitch.gui.progress.stop_timer()
self._twitch._watching_restart.set()
async def watch_sleep(self, delay: float) -> None:
"""
Sleep for a delay that can be interrupted by restart_watching().
Uses wait_for with a timeout to allow an asyncio.sleep-like behavior
that can be ended prematurely via the watching restart event.
Args:
delay: Time in seconds to sleep
"""
self._twitch._watching_restart.clear()
with suppress(asyncio.TimeoutError):
await asyncio.wait_for(self._twitch._watching_restart.wait(), timeout=delay)
@task_wrapper(critical=True)
async def watch_loop(self) -> NoReturn:
"""
Main watch loop that sends watch payloads and monitors drop progress.
This loop:
1. Waits for a channel to watch
2. Sends watch payload to the channel
3. Waits ~20 seconds for websocket progress update
4. If no update received, queries drop progress via GQL or estimates it
5. Sleeps until next watch interval (~20 seconds)
6. Repeats
The loop handles cases where Twitch temporarily stops reporting progress
by falling back to GQL queries or minute bumping.
"""
interval: float = WATCH_INTERVAL.total_seconds()
while True:
channel: Channel = await self._twitch.watching_channel.get()
if not channel.online:
# if the channel isn't online anymore, we stop watching it
self.stop_watching()
continue
# logger.log(CALL, f"Sending watch payload to: {channel.name}")
succeeded: bool = await channel.send_watch()
last_sent: float = time()
if not succeeded:
logger.log(CALL, f"Watch requested failed for channel: {channel.name}")
# wait ~20 seconds for a progress update
await asyncio.sleep(20)
if self._twitch.gui.progress.minute_almost_done():
# If the previous update was more than ~60s ago, and the progress tracker
# isn't counting down anymore, that means Twitch has temporarily
# stopped reporting drop's progress. To ensure the timer keeps at least somewhat
# accurate time, we can use GQL to query for the current drop,
# or even "pretend" mining as a last resort option.
handled: bool = False
# Solution 1: use GQL to query for the currently mined drop status
try:
context = await self._twitch.gql_request(
GQL_OPERATIONS["CurrentDrop"].with_variables(
{"channelID": str(channel.id)}
)
)
drop_data: JsonType | None = (
context["data"]["currentUser"]["dropCurrentSession"]
)
except GQLException:
drop_data = None
if drop_data is not None:
gql_drop: TimedDrop | None = self._twitch._drops.get(drop_data["dropID"])
if gql_drop is not None and gql_drop.can_earn(channel):
gql_drop.update_minutes(drop_data["currentMinutesWatched"])
drop_text: str = (
f"{gql_drop.name} ({gql_drop.campaign.game}, "
f"{gql_drop.current_minutes}/{gql_drop.required_minutes})"
)
logger.log(CALL, f"Drop progress from GQL: {drop_text}")
handled = True
# Solution 2: If GQL fails, figure out which campaign we're most likely mining
# right now, and then bump up the minutes on it's drops
if not handled:
active_campaign = self._twitch._inventory_service.get_active_campaign(channel)
if active_campaign is not None:
active_campaign.bump_minutes(channel)
# NOTE: This usually gets overwritten below
drop_text = f"Unknown drop ({active_campaign.game})"
if (active_drop := active_campaign.first_drop) is not None:
active_drop.display()
drop_text = (
f"{active_drop.name} ({active_drop.campaign.game}, "
f"{active_drop.current_minutes}/{active_drop.required_minutes})"
)
logger.log(CALL, f"Drop progress from active search: {drop_text}")
handled = True
else:
logger.log(CALL, "No active drop could be determined")
await self.watch_sleep(interval - min(time() - last_sent, interval))

64
src/utils/__init__.py Normal file
View File

@@ -0,0 +1,64 @@
"""Utility modules for TwitchDropsMiner."""
from __future__ import annotations
# String utilities
from .string_utils import (
CHARS_ASCII,
CHARS_HEX_LOWER,
CHARS_HEX_UPPER,
create_nonce,
chunk,
deduplicate,
)
# JSON utilities
from .json_utils import (
json_minify,
json_load,
json_save,
merge_json,
SERIALIZE_ENV,
)
# Async helpers
from .async_helpers import (
first_to_complete,
format_traceback,
task_wrapper,
invalidate_cache,
AwaitableValue,
)
# Rate limiting
from .rate_limiter import RateLimiter
# Backoff
from .backoff import ExponentialBackoff
__all__ = [
# String utilities
"CHARS_ASCII",
"CHARS_HEX_LOWER",
"CHARS_HEX_UPPER",
"create_nonce",
"chunk",
"deduplicate",
# JSON utilities
"json_minify",
"json_load",
"json_save",
"merge_json",
"SERIALIZE_ENV",
# Async helpers
"first_to_complete",
"format_traceback",
"task_wrapper",
"invalidate_cache",
"AwaitableValue",
# Rate limiting
"RateLimiter",
# Backoff
"ExponentialBackoff",
]

137
src/utils/async_helpers.py Normal file
View File

@@ -0,0 +1,137 @@
"""Async programming utilities and helpers."""
from __future__ import annotations
import io
import sys
import asyncio
import logging
import traceback
from pathlib import Path
from functools import wraps
from contextlib import suppress
from typing import Any, Literal, Generic, TypeVar, ParamSpec
from collections import abc
from src.exceptions import ExitRequest, ReloadRequest
_T = TypeVar("_T") # type
_D = TypeVar("_D") # default
_P = ParamSpec("_P") # params
logger = logging.getLogger("TwitchDrops")
async def first_to_complete(coros: abc.Iterable[abc.Coroutine[Any, Any, _T]]) -> _T:
"""Wait for the first coroutine to complete, canceling the rest."""
# In Python 3.11, we need to explicitly wrap awaitables
tasks: list[asyncio.Task[_T]] = [asyncio.ensure_future(coro) for coro in coros]
done: set[asyncio.Task[Any]]
pending: set[asyncio.Task[Any]]
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
for task in pending:
task.cancel()
return await next(iter(done))
def format_traceback(exc: BaseException, **kwargs: Any) -> str:
"""
Like `traceback.print_exc` but returns a string. Uses the passed-in exception.
Any additional `**kwargs` are passed to the underlaying `traceback.format_exception`.
"""
return ''.join(traceback.format_exception(type(exc), exc, exc.__traceback__, **kwargs))
def task_wrapper(
afunc: abc.Callable[_P, abc.Coroutine[Any, Any, _T]] | None = None, *, critical: bool = False
):
"""
Decorator for async tasks that handles exceptions gracefully.
Args:
afunc: The async function to wrap
critical: If True, a critical task failure will trigger application termination
Handles ExitRequest and ReloadRequest silently, logs other exceptions.
Critical tasks will attempt to find and close the Twitch instance on failure.
"""
def decorator(
afunc: abc.Callable[_P, abc.Coroutine[Any, Any, _T]]
) -> abc.Callable[_P, abc.Coroutine[Any, Any, _T]]:
@wraps(afunc)
async def wrapper(*args: _P.args, **kwargs: _P.kwargs):
try:
await afunc(*args, **kwargs)
except (ExitRequest, ReloadRequest):
pass
except Exception:
logger.exception(f"Exception in {afunc.__name__} task")
if critical:
# critical task's death should trigger a termination.
# there isn't an easy and sure way to obtain the Twitch instance here,
# but we can improvise finding it
from src.core.client import Twitch # cyclic import
probe = args and args[0] or None # extract from 'self' arg
if isinstance(probe, Twitch):
probe.close()
elif probe is not None:
probe = getattr(probe, "_twitch", None) # extract from '_twitch' attr
if isinstance(probe, Twitch):
probe.close()
raise # raise up to the wrapping task
return wrapper
if afunc is None:
return decorator
return decorator(afunc)
def invalidate_cache(instance: object, *attrnames: str) -> None:
"""
Invalidate cached_property attributes on an instance.
Used to clear functools.cached_property values.
"""
for name in attrnames:
with suppress(AttributeError):
delattr(instance, name)
class AwaitableValue(Generic[_T]):
"""
A value that can be set once and awaited by multiple consumers.
Provides async/await interface for waiting on a value to become available.
Useful for coordination between async tasks.
"""
def __init__(self):
self._value: _T
self._event = asyncio.Event()
def has_value(self) -> bool:
"""Check if the value has been set."""
return self._event.is_set()
def wait(self) -> abc.Coroutine[Any, Any, Literal[True]]:
"""Return a coroutine that waits for the value to be set."""
return self._event.wait()
def get_with_default(self, default: _D) -> _T | _D:
"""Get the value if set, otherwise return the default."""
if self._event.is_set():
return self._value
return default
async def get(self) -> _T:
"""Wait for and return the value."""
await self._event.wait()
return self._value
def set(self, value: _T) -> None:
"""Set the value and notify all waiters."""
self._value = value
self._event.set()
def clear(self) -> None:
"""Clear the value, allowing it to be set again."""
self._event.clear()

86
src/utils/backoff.py Normal file
View File

@@ -0,0 +1,86 @@
"""Exponential backoff implementation for retry logic."""
from __future__ import annotations
import random
from collections import abc
class ExponentialBackoff:
"""
Iterator that generates exponentially increasing delays with variance.
Useful for implementing retry logic with exponential backoff.
Includes configurable variance to prevent thundering herd problems.
Usage:
backoff = ExponentialBackoff(base=2, maximum=300)
for delay in backoff:
await asyncio.sleep(delay)
if try_operation():
backoff.reset()
break
"""
def __init__(
self,
*,
base: float = 2,
variance: float | tuple[float, float] = 0.1,
shift: float = 0,
maximum: float = 300,
):
"""
Initialize exponential backoff.
Args:
base: Exponential base (must be > 1)
variance: Random variance to apply. Can be:
- Single float: applies symmetric variance (1 ± variance)
- Tuple: (min_multiplier, max_multiplier) for asymmetric variance
shift: Constant value added to each delay
maximum: Maximum delay value to return
Raises:
ValueError: If base <= 1
"""
if base <= 1:
raise ValueError("Base has to be greater than 1")
self.steps: int = 0
self.base: float = float(base)
self.shift: float = float(shift)
self.maximum: float = float(maximum)
self.variance_min: float
self.variance_max: float
if isinstance(variance, tuple):
self.variance_min, self.variance_max = variance
else:
self.variance_min = 1 - variance
self.variance_max = 1 + variance
@property
def exp(self) -> int:
"""Current exponent value (steps - 1, minimum 0)."""
return max(0, self.steps - 1)
def __iter__(self) -> abc.Iterator[float]:
return self
def __next__(self) -> float:
"""Generate the next delay value."""
value: float = (
pow(self.base, self.steps)
* random.uniform(self.variance_min, self.variance_max)
+ self.shift
)
if value > self.maximum:
return self.maximum
# NOTE: variance can cause the returned value to be lower than the previous one already,
# so this should be safe to move past the first return,
# to prevent the exponent from getting very big after reaching max and many iterations
self.steps += 1
return value
def reset(self) -> None:
"""Reset the backoff to initial state."""
self.steps = 0

6
cache.py → src/utils/cache.py
View File

@@ -7,15 +7,15 @@ import io
import json import json
from typing import Dict, TypedDict, NewType, TYPE_CHECKING from typing import Dict, TypedDict, NewType, TYPE_CHECKING
from utils import json_load, json_save from src.utils import json_load, json_save
from constants import URLType, CACHE_PATH, CACHE_DB from src.config import URLType, CACHE_PATH, CACHE_DB
from PIL import Image as Image_module from PIL import Image as Image_module
from PIL.ImageTk import PhotoImage from PIL.ImageTk import PhotoImage
if TYPE_CHECKING: if TYPE_CHECKING:
from gui import GUIManager from src.web.gui_manager import GUIManager
from PIL.Image import Image from PIL.Image import Image
from typing_extensions import TypeAlias from typing_extensions import TypeAlias

158
src/utils/json_utils.py Normal file
View File

@@ -0,0 +1,158 @@
"""JSON serialization and deserialization utilities."""
from __future__ import annotations
import json
from datetime import datetime, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Callable, Mapping, TypeVar, cast
from yarl import URL
from src.config import JsonType
_JSON_T = TypeVar("_JSON_T", bound=Mapping[Any, Any])
_MISSING = object()
# Serialization environment - maps type names to deserialization functions
SERIALIZE_ENV: dict[str, Callable[[Any], object]] = {
"set": set,
"URL": URL,
"datetime": lambda d: datetime.fromtimestamp(d, timezone.utc),
}
def json_minify(data: JsonType | list[JsonType]) -> str:
"""Return minified JSON string (no whitespace) for payload usage."""
return json.dumps(data, separators=(',', ':'))
def _serialize(obj: Any) -> Any:
"""
Custom JSON encoder for special types.
Converts datetime, set, Enum, and URL objects to serializable format.
Stores both the type name and the converted data for proper deserialization.
"""
# convert data
d: int | str | float | list[Any] | JsonType
if isinstance(obj, datetime):
if obj.tzinfo is None:
# assume naive objects are UTC
obj = obj.replace(tzinfo=timezone.utc)
d = obj.timestamp()
elif isinstance(obj, set):
d = list(obj)
elif isinstance(obj, Enum):
# NOTE: IntEnum cannot be used, as it will get serialized as a plain integer,
# then loaded back as an integer as well.
d = obj.value
elif isinstance(obj, URL):
d = str(obj)
else:
raise TypeError(obj)
# store with type
return {
"__type": type(obj).__name__,
"data": d,
}
def _remove_missing(obj: JsonType) -> JsonType:
"""
Remove _MISSING sentinel values from a dictionary recursively.
This modifies obj in place, but returns it for convenience.
Used during deserialization to clean up unrecognized types.
"""
for key, value in obj.copy().items():
if value is _MISSING:
del obj[key]
elif isinstance(value, dict):
_remove_missing(value)
if not value:
# the dict is empty now, so remove it's key entirely
del obj[key]
return obj
def _deserialize(obj: JsonType) -> Any:
"""
Custom JSON decoder hook for special types.
Reconstructs objects from serialized format using SERIALIZE_ENV.
Returns _MISSING sentinel for unrecognized types (to be cleaned up later).
"""
if "__type" in obj:
obj_type = obj["__type"]
if obj_type in SERIALIZE_ENV:
return SERIALIZE_ENV[obj_type](obj["data"])
else:
return _MISSING
return obj
def merge_json(obj: JsonType, template: Mapping[Any, Any]) -> None:
"""
Merge a JSON object with a template, ensuring all expected keys exist.
NOTE: This modifies object in place.
- Removes keys not present in template
- Overwrites values with wrong type from template
- Recursively merges nested dictionaries
- Adds missing keys from template
"""
for k, v in list(obj.items()):
if k not in template:
# unknown key: overwrite from template
del obj[k]
elif type(v) is not type(template[k]):
# types don't match: overwrite from template
obj[k] = template[k]
elif isinstance(v, dict):
assert isinstance(template[k], dict)
merge_json(v, template[k])
# ensure the object is not missing any keys
for k in template.keys():
if k not in obj:
obj[k] = template[k]
def json_load(path: Path, defaults: _JSON_T, *, merge: bool = True) -> _JSON_T:
"""
Load JSON from a file with defaults and optional merging.
Args:
path: Path to JSON file
defaults: Default values to use if file doesn't exist or merge is enabled
merge: If True, merge loaded data with defaults template
Returns:
Loaded and optionally merged JSON data
"""
defaults_dict: JsonType = dict(defaults)
if path.exists():
with open(path, 'r', encoding="utf8") as file:
combined: JsonType = _remove_missing(json.load(file, object_hook=_deserialize))
if merge:
merge_json(combined, defaults_dict)
else:
combined = defaults_dict
return cast(_JSON_T, combined)
def json_save(path: Path, contents: Mapping[Any, Any], *, sort: bool = False) -> None:
"""
Save data to a JSON file with custom serialization.
Args:
path: Path to save JSON file
contents: Data to serialize
sort: If True, sort keys alphabetically
"""
with open(path, 'w', encoding="utf8") as file:
json.dump(contents, file, default=_serialize, sort_keys=sort, indent=4)

75
src/utils/rate_limiter.py Normal file
View File

@@ -0,0 +1,75 @@
"""Rate limiting utilities for API request management."""
from __future__ import annotations
import asyncio
class RateLimiter:
"""
Async context manager for rate limiting operations.
Enforces a maximum number of operations within a sliding time window.
Tracks both total operations in the window and concurrent operations.
Usage:
limiter = RateLimiter(capacity=10, window=60)
async with limiter:
# perform rate-limited operation
pass
"""
def __init__(self, *, capacity: int, window: int):
"""
Initialize rate limiter.
Args:
capacity: Maximum number of operations allowed
window: Time window in seconds
"""
self.total: int = 0
self.concurrent: int = 0
self.window: int = window
self.capacity: int = capacity
self._reset_task: asyncio.Task[None] | None = None
self._cond: asyncio.Condition = asyncio.Condition()
def __repr__(self) -> str:
return f"{self.__class__.__name__}({self.concurrent}/{self.total}/{self.capacity})"
def __del__(self) -> None:
if self._reset_task is not None:
self._reset_task.cancel()
def _can_proceed(self) -> bool:
"""Check if an operation can proceed based on current limits."""
return max(self.total, self.concurrent) < self.capacity
async def __aenter__(self):
"""Enter context - wait if rate limit is reached."""
async with self._cond:
await self._cond.wait_for(self._can_proceed)
self.total += 1
self.concurrent += 1
if self._reset_task is None:
self._reset_task = asyncio.create_task(self._rtask())
async def __aexit__(self, exc_type, exc, tb):
"""Exit context - decrement concurrent counter and notify waiters."""
self.concurrent -= 1
async with self._cond:
self._cond.notify(self.capacity - self.concurrent)
async def _reset(self) -> None:
"""Reset the total counter after the window expires."""
if self._reset_task is not None:
self._reset_task = None
async with self._cond:
self.total = 0
if self.concurrent < self.capacity:
self._cond.notify(self.capacity - self.concurrent)
async def _rtask(self) -> None:
"""Background task that resets counters after the time window."""
await asyncio.sleep(self.window)
await self._reset()

34
src/utils/string_utils.py Normal file
View File

@@ -0,0 +1,34 @@
"""String manipulation utility functions."""
from __future__ import annotations
import random
import string
from collections import OrderedDict
from collections import abc
from typing import TypeVar
# Character sets for nonce generation
CHARS_ASCII = string.ascii_letters + string.digits
CHARS_HEX_LOWER = string.digits + "abcdef"
CHARS_HEX_UPPER = string.digits + "ABCDEF"
_T = TypeVar("_T")
def create_nonce(chars: str, length: int) -> str:
"""Generate a random nonce string of specified length from given characters."""
return ''.join(random.choices(chars, k=length))
def chunk(to_chunk: abc.Iterable[_T], chunk_length: int) -> abc.Generator[list[_T], None, None]:
"""Split an iterable into chunks of a specified length."""
list_to_chunk: list[_T] = list(to_chunk)
for i in range(0, len(list_to_chunk), chunk_length):
yield list_to_chunk[i:i + chunk_length]
def deduplicate(iterable: abc.Iterable[_T]) -> list[_T]:
"""Remove duplicates from an iterable while preserving order."""
return list(OrderedDict.fromkeys(iterable).keys())

0
version.py → src/version.py
View File

0
src/web/__init__.py Normal file
View File

0
src/web/api/__init__.py Normal file
View File

299
src/web/app.py Normal file
View File

@@ -0,0 +1,299 @@
from __future__ import annotations
import asyncio
import logging
from pathlib import Path
from typing import TYPE_CHECKING
from fastapi import FastAPI, HTTPException
from fastapi.staticfiles import StaticFiles
from fastapi.responses import HTMLResponse, FileResponse
from fastapi.middleware.cors import CORSMiddleware
import socketio
from pydantic import BaseModel
if TYPE_CHECKING:
from src.web.gui_manager import WebGUIManager
from src.core.client import Twitch
logger = logging.getLogger("TwitchDrops")
# Create FastAPI app
app = FastAPI(title="Twitch Drops Miner Web", version="1.0.0")
# Add CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # In production, specify exact origins
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Create Socket.IO server
sio = socketio.AsyncServer(
async_mode='asgi',
cors_allowed_origins='*',
logger=False,
engineio_logger=False
)
# Wrap with ASGI app
socket_app = socketio.ASGIApp(sio, app)
# Global references (set by main.py)
gui_manager: WebGUIManager | None = None
twitch_client: Twitch | None = None
_server_instance: 'uvicorn.Server | None' = None
def set_managers(gui: WebGUIManager, twitch: Twitch):
"""Called by main.py to set up references"""
global gui_manager, twitch_client
gui_manager = gui
twitch_client = twitch
gui.set_socketio(sio)
# Pydantic models for API
class LoginRequest(BaseModel):
username: str
password: str
token: str = ''
class ChannelSelectRequest(BaseModel):
channel_id: int
class SettingsUpdate(BaseModel):
games_to_watch: list[str] | None = None
dark_mode: bool | None = None
proxy: str | None = None
tray_notifications: bool | None = None
connection_quality: int | None = None
# ==================== REST API Endpoints ====================
@app.get("/", response_class=HTMLResponse)
async def serve_index():
"""Serve the main web interface"""
# Web files are in project_root/web/, we're in project_root/src/web/
web_dir = Path(__file__).parent.parent.parent / "web"
index_file = web_dir / "index.html"
logger.debug(f"Looking for web files: __file__={__file__}, web_dir={web_dir}, index_file={index_file}, exists={index_file.exists()}")
if index_file.exists():
return FileResponse(index_file)
return HTMLResponse(
content=f"<h1>Twitch Drops Miner</h1><p>Web interface files not found. Please check installation.</p><p>Debug: Looking for {index_file}</p>",
status_code=500
)
@app.get("/api/status")
async def get_status():
"""Get current application status"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
return {
"status": gui_manager.status.get(),
"login": gui_manager.login.get_status(),
"close_requested": gui_manager.close_requested
}
@app.get("/api/channels")
async def get_channels():
"""Get list of tracked channels"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
return {
"channels": gui_manager.channels.get_channels()
}
@app.post("/api/channels/select")
async def select_channel(request: ChannelSelectRequest):
"""Select a channel to watch"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
gui_manager.select_channel(request.channel_id)
return {"success": True}
@app.get("/api/campaigns")
async def get_campaigns():
"""Get campaign inventory"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
return {
"campaigns": gui_manager.inv.get_campaigns()
}
@app.get("/api/console")
async def get_console_history():
"""Get console output history"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
return {
"lines": gui_manager.output.get_history()
}
@app.get("/api/settings")
async def get_settings():
"""Get current settings"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
return gui_manager.settings.get_settings()
@app.post("/api/settings")
async def update_settings(settings: SettingsUpdate):
"""Update application settings"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
settings_dict = settings.dict(exclude_unset=True)
gui_manager.settings.update_settings(settings_dict)
return {"success": True, "settings": gui_manager.settings.get_settings()}
@app.post("/api/login")
async def submit_login(login_data: LoginRequest):
"""Submit login credentials"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
gui_manager.login.submit_login(
login_data.username,
login_data.password,
login_data.token
)
return {"success": True}
@app.post("/api/oauth/confirm")
async def confirm_oauth():
"""Confirm OAuth code has been entered by user"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
# Just set the event to signal the user has acknowledged the code
gui_manager.login._login_event.set()
return {"success": True}
@app.post("/api/reload")
async def trigger_reload():
"""Trigger application reload"""
if not twitch_client:
raise HTTPException(status_code=503, detail="Twitch client not initialized")
from src.config import State
twitch_client.change_state(State.INVENTORY_FETCH)
return {"success": True}
@app.post("/api/close")
async def trigger_close():
"""Trigger application shutdown"""
if not gui_manager:
raise HTTPException(status_code=503, detail="GUI not initialized")
gui_manager.close()
return {"success": True}
# ==================== Socket.IO Events ====================
@sio.event
async def connect(sid, environ):
"""Client connected"""
logger.info(f"Web client connected: {sid}")
# Send initial state to new client
if gui_manager:
await sio.emit("initial_state", {
"status": gui_manager.status.get(),
"channels": gui_manager.channels.get_channels(),
"campaigns": gui_manager.inv.get_campaigns(),
"console": gui_manager.output.get_history(),
"settings": gui_manager.settings.get_settings(),
"login": gui_manager.login.get_status()
}, room=sid)
@sio.event
async def disconnect(sid):
"""Client disconnected"""
logger.info(f"Web client disconnected: {sid}")
@sio.event
async def request_login(sid):
"""Client requested login form submission"""
logger.info(f"Login request from client: {sid}")
# The actual login data comes via REST API
@sio.event
async def request_reload(sid):
"""Client requested application reload"""
if twitch_client:
from src.config import State
twitch_client.change_state(State.INVENTORY_FETCH)
# Mount static files (CSS, JS, images)
# Web files are in project_root/web/, we're in project_root/src/web/
web_dir = Path(__file__).parent.parent.parent / "web"
if web_dir.exists():
static_dir = web_dir / "static"
if static_dir.exists():
app.mount("/static", StaticFiles(directory=static_dir), name="static")
# Development server runner
async def run_server(host: str = "0.0.0.0", port: int = 8080):
"""Run the web server (used for development/testing)"""
global _server_instance
import uvicorn
config = uvicorn.Config(
socket_app,
host=host,
port=port,
log_level="info",
access_log=False
)
server = uvicorn.Server(config)
_server_instance = server
try:
await server.serve()
finally:
_server_instance = None
async def shutdown_server():
"""Gracefully shutdown the web server"""
global _server_instance
if _server_instance:
logger.info("Setting server.should_exit = True")
_server_instance.should_exit = True
# Give the server a moment to process the shutdown signal
# The uvicorn server checks should_exit periodically
await asyncio.sleep(0.1)
if __name__ == "__main__":
# For standalone testing
asyncio.run(run_server())

233
src/web/gui_manager.py Normal file
View File

@@ -0,0 +1,233 @@
"""Main web GUI manager coordinating all UI components."""
from __future__ import annotations
import asyncio
import logging
from typing import TYPE_CHECKING
from src.models.game import Game
from src.i18n import _
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.web.managers.status import StatusManager, WebsocketStatusManager
from src.web.managers.console import ConsoleOutputManager
from src.web.managers.campaigns import CampaignProgressManager
from src.web.managers.channels import ChannelListManager
from src.web.managers.inventory import InventoryManager
from src.web.managers.login import LoginFormManager
from src.web.managers.tray import TrayIconStub
from src.web.managers.settings import SettingsManager
from src.web.managers.cache import ImageCache
if TYPE_CHECKING:
from socketio import AsyncServer
from src.core.client import Twitch
from src.models import TimedDrop
logger = logging.getLogger("TwitchDrops")
class WebGUIManager:
"""Web-based GUI manager coordinating all UI components.
This class serves as the main coordinator for the web-based interface,
managing all component managers and providing the same interface as the
desktop GUIManager for compatibility with the core application logic.
The WebGUIManager uses Socket.IO for real-time bidirectional communication
with browser clients, enabling live updates of drop progress, channel lists,
and other dynamic content.
"""
def __init__(self, twitch: Twitch):
self._twitch: Twitch = twitch
self._broadcaster = WebSocketBroadcaster()
self._close_requested = asyncio.Event()
# Create component managers
self.status = StatusManager(self._broadcaster)
self.websockets = WebsocketStatusManager(self._broadcaster)
self.output = ConsoleOutputManager(self._broadcaster)
self.progress = CampaignProgressManager(self._broadcaster)
self.channels = ChannelListManager(self._broadcaster)
self.inv = InventoryManager(self._broadcaster, ImageCache(self))
self.login = LoginFormManager(self._broadcaster, self)
self.tray = TrayIconStub(self._broadcaster)
self.settings = SettingsManager(self._broadcaster, twitch.settings)
# Selected channel tracking (set by web client)
self._selected_channel_id: int | None = None
# Start message
logger.info("Web GUI Manager initialized")
def set_socketio(self, sio: AsyncServer):
"""Set the Socket.IO instance for real-time communication.
Called by webapp during initialization to connect the broadcaster
to the Socket.IO server.
Args:
sio: The Socket.IO AsyncServer instance
"""
self._broadcaster.set_socketio(sio)
@property
def close_requested(self) -> bool:
"""Check if application closure has been requested.
Returns:
True if close was requested via GUI
"""
return self._close_requested.is_set()
def start(self):
"""Start the GUI (logs ready message in web mode)."""
logger.info("Web GUI started - access via browser")
self.status.update(_("gui", "status", "ready"))
def close(self, *args) -> int:
"""Request application closure.
Returns:
Exit code (0 for normal shutdown)
"""
self._close_requested.set()
# notify client we're supposed to close
self._twitch.close()
logger.info("Close requested via web GUI")
return 0
def prevent_close(self):
"""Prevent window from closing (no-op in web mode).
In web mode, users can still navigate away from the page.
"""
pass
def stop(self):
"""Stop the GUI."""
logger.info("Web GUI stopped")
def close_window(self):
"""Close the GUI window (no-op in web mode)."""
pass
async def wait_until_closed(self):
"""Wait until the GUI is closed by the user."""
await self._close_requested.wait()
async def coro_unless_closed(self, coro):
"""Run a coroutine unless the GUI is closed.
Races the provided coroutine against the close event, canceling
the coroutine if close is requested and raising ExitRequest.
Args:
coro: Coroutine to run
Returns:
Result of the coroutine if it completes first
Raises:
ExitRequest: If close is requested during execution
"""
# Race the coroutine against the close event
tasks = [asyncio.ensure_future(coro), asyncio.ensure_future(self._close_requested.wait())]
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
# Cancel any pending tasks
for task in pending:
task.cancel()
# If close was requested, raise ExitRequest
if self._close_requested.is_set():
from src.exceptions import ExitRequest
raise ExitRequest()
# Otherwise return the result
return await next(iter(done))
def save(self, *, force: bool = False):
"""Save GUI state and settings.
Args:
force: Force save even if no changes detected
"""
self._twitch.settings.save(force=force)
def print(self, message: str):
"""Print message to console output.
Args:
message: Message to display in console
"""
self.output.print(message)
def set_games(self, games: set[Game]):
"""Set available games for settings panel.
Args:
games: Set of Game objects from discovered campaigns
"""
self.settings.set_games(games)
def display_drop(self, drop: TimedDrop, *, countdown: bool = True, subone: bool = False):
"""Display drop mining progress with countdown.
Args:
drop: The drop currently being mined
countdown: Whether to show countdown timer
subone: Subtract one minute from remaining time
"""
remaining = drop.remaining_minutes * 60 # Convert to seconds
if subone:
remaining -= 60
self.progress.update(drop, remaining)
def clear_drop(self):
"""Clear the drop progress display."""
self.progress.stop_timer()
def grab_attention(self, *, sound: bool = True):
"""Get user's attention via notification.
Args:
sound: Whether to play notification sound
"""
asyncio.create_task(
self._broadcaster.emit("attention_required", {"sound": sound})
)
def select_channel(self, channel_id: int):
"""Select a channel (called by webapp when user clicks channel).
Args:
channel_id: Twitch channel ID to select
"""
self._selected_channel_id = channel_id
def get_selected_channel_id(self) -> int | None:
"""Get the currently selected channel ID and clear the selection.
Returns:
Channel ID if one was selected, None otherwise
"""
result = self._selected_channel_id
self._selected_channel_id = None # Clear after reading
return result
def apply_theme(self, dark_mode: bool):
"""Apply UI theme (handled client-side in web mode).
Args:
dark_mode: Whether to use dark theme
"""
asyncio.create_task(
self._broadcaster.emit("theme_change", {"dark_mode": dark_mode})
)
# Type aliases for backwards compatibility with code that imports from gui
LoginForm = LoginFormManager
ChannelList = ChannelListManager
WebsocketStatus = WebsocketStatusManager
GUIManager = WebGUIManager

41
src/web/managers/__init__.py Normal file
View File

@@ -0,0 +1,41 @@
"""Web GUI manager modules for the Twitch Drops Miner web interface.
This package contains all component managers for the web-based GUI:
- WebSocketBroadcaster: Real-time message broadcasting to clients
- StatusManager: Main application status display
- WebsocketStatusManager: WebSocket connection pool status
- ConsoleOutputManager: Console log output buffering and display
- CampaignProgressManager: Active drop mining progress and countdown
- ChannelListManager: Available channels tracking and display
- InventoryManager: Drop campaigns and inventory management
- LoginFormManager: Authentication and OAuth flow handling
- SettingsManager: Application settings configuration
- TrayIconStub: System tray stub (browser notifications in web mode)
- ImageCache: Minimal image caching for campaign artwork
"""
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.web.managers.status import StatusManager, WebsocketStatusManager
from src.web.managers.console import ConsoleOutputManager
from src.web.managers.campaigns import CampaignProgressManager
from src.web.managers.channels import ChannelListManager
from src.web.managers.inventory import InventoryManager
from src.web.managers.login import LoginFormManager, LoginData
from src.web.managers.settings import SettingsManager
from src.web.managers.tray import TrayIconStub
from src.web.managers.cache import ImageCache
__all__ = [
"WebSocketBroadcaster",
"StatusManager",
"WebsocketStatusManager",
"ConsoleOutputManager",
"CampaignProgressManager",
"ChannelListManager",
"InventoryManager",
"LoginFormManager",
"LoginData",
"SettingsManager",
"TrayIconStub",
"ImageCache",
]

33
src/web/managers/broadcaster.py Normal file
View File

@@ -0,0 +1,33 @@
"""WebSocket broadcaster for real-time updates to web clients."""
from __future__ import annotations
from typing import Any, TYPE_CHECKING
if TYPE_CHECKING:
from socketio import AsyncServer
class WebSocketBroadcaster:
"""Manages broadcasting messages to all connected web clients via Socket.IO.
This class acts as a central hub for sending real-time updates from the application
to all connected browser clients through Socket.IO events.
"""
def __init__(self):
self._sio: AsyncServer | None = None # Will be set by webapp
def set_socketio(self, sio: AsyncServer):
"""Set the Socket.IO server instance for broadcasting."""
self._sio = sio
async def emit(self, event: str, data: Any):
"""Emit an event to all connected clients.
Args:
event: The event name to emit
data: The data payload to send with the event
"""
if self._sio:
await self._sio.emit(event, data)

32
src/web/managers/cache.py Normal file
View File

@@ -0,0 +1,32 @@
"""Image cache for web interface (minimal implementation)."""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.web.gui_manager import WebGUIManager
class ImageCache:
"""Minimal image cache for web mode.
In the web interface, campaign images are referenced by their URLs directly
rather than being cached locally. This class maintains API compatibility
with desktop GUI implementations while simply passing through URLs.
"""
def __init__(self, manager: WebGUIManager):
self._manager = manager
async def get(self, url: str) -> str:
"""Get image URL (returns the URL directly in web mode).
Args:
url: The image URL to retrieve
Returns:
The same URL (no caching needed for web display)
"""
# In web mode, we just return the URL
return url

61
src/web/managers/campaigns.py Normal file
View File

@@ -0,0 +1,61 @@
"""Campaign progress manager for tracking active drop mining progress."""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.models import TimedDrop
class CampaignProgressManager:
"""Manages active drop mining progress display and countdown timer.
Tracks the currently mined drop and broadcasts real-time progress updates
including remaining time and completion percentage to the web interface.
"""
def __init__(self, broadcaster: WebSocketBroadcaster):
self._broadcaster = broadcaster
self._current_drop: TimedDrop | None = None
self._remaining_seconds: int = 0
def update(self, drop: TimedDrop | None, remaining_seconds: int):
"""Update the current drop progress and remaining time.
Args:
drop: The drop currently being mined, or None if no active drop
remaining_seconds: Seconds remaining until the next progress minute
"""
self._current_drop = drop
self._remaining_seconds = remaining_seconds
if drop:
asyncio.create_task(
self._broadcaster.emit("drop_progress", {
"drop_id": drop.id,
"drop_name": drop.name,
"campaign_name": drop.campaign.name,
"game_name": drop.campaign.game.name,
"current_minutes": drop.current_minutes,
"required_minutes": drop.required_minutes,
"progress": drop.progress,
"remaining_seconds": remaining_seconds
})
)
def stop_timer(self):
"""Stop the progress timer and clear the current drop."""
self._current_drop = None
asyncio.create_task(
self._broadcaster.emit("drop_progress_stop", {})
)
def minute_almost_done(self) -> bool:
"""Check if the current progress minute is almost complete.
Returns:
True if remaining seconds is at or below zero
"""
return self._remaining_seconds <= 0

100
src/web/managers/channels.py Normal file
View File

@@ -0,0 +1,100 @@
"""Channel list manager for tracking and displaying available channels."""
from __future__ import annotations
import asyncio
from typing import Any, TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.models.channel import Channel
class ChannelListManager:
"""Manages the list of available channels in the web interface.
Tracks all discovered channels with their online status, game, viewers,
and drop eligibility. Broadcasts real-time updates when channels change
or when the watched channel switches.
"""
def __init__(self, broadcaster: WebSocketBroadcaster):
self._broadcaster = broadcaster
self._channels: dict[int, dict[str, Any]] = {}
self._watching_id: int | None = None
self._selected_id: int | None = None
def display(self, channel: Channel, *, add: bool = False):
"""Add or update a channel in the display list.
Args:
channel: The channel to display
add: If True, emit channel_add event; otherwise emit channel_update
"""
channel_data = {
"id": channel.id,
"name": channel.name,
"game": channel.game.name if channel.game else None,
"viewers": channel.viewers,
"online": channel.online,
"drops_enabled": channel.drops_enabled,
"acl_based": channel.acl_based,
"watching": channel.id == self._watching_id
}
self._channels[channel.id] = channel_data
asyncio.create_task(
self._broadcaster.emit("channel_update" if not add else "channel_add", channel_data)
)
def remove(self, channel: Channel):
"""Remove a channel from the display list.
Args:
channel: The channel to remove
"""
if channel.id in self._channels:
del self._channels[channel.id]
asyncio.create_task(
self._broadcaster.emit("channel_remove", {"id": channel.id})
)
def clear(self):
"""Clear all channels from the display list."""
self._channels.clear()
asyncio.create_task(
self._broadcaster.emit("channels_clear", {})
)
def set_watching(self, channel: Channel):
"""Mark a channel as currently being watched.
Args:
channel: The channel now being watched
"""
self._watching_id = channel.id
asyncio.create_task(
self._broadcaster.emit("channel_watching", {"id": channel.id})
)
def clear_watching(self):
"""Clear the currently watched channel indicator."""
self._watching_id = None
asyncio.create_task(
self._broadcaster.emit("channel_watching_clear", {})
)
def get_selection(self) -> Channel | None:
"""Get user's channel selection (handled via webapp API).
Returns:
None (selection is handled through the web API, not here)
"""
return None # Handled via webapp API
def get_channels(self) -> list[dict[str, Any]]:
"""Get all currently tracked channels.
Returns:
List of channel data dictionaries
"""
return list(self._channels.values())

44
src/web/managers/console.py Normal file
View File

@@ -0,0 +1,44 @@
"""Console output manager for logging to web interface."""
from __future__ import annotations
import asyncio
from datetime import datetime
from collections import deque
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
class ConsoleOutputManager:
"""Manages console output display in the web interface.
Buffers log messages and broadcasts them to connected clients in real-time,
maintaining a rolling history of recent messages.
"""
def __init__(self, broadcaster: WebSocketBroadcaster, max_lines: int = 1000):
self._broadcaster = broadcaster
self._buffer: deque[str] = deque(maxlen=max_lines)
def print(self, message: str):
"""Print a message to the console output with timestamp.
Args:
message: The message to display
"""
timestamp = datetime.now().strftime("%H:%M:%S")
line = f"[{timestamp}] {message}"
self._buffer.append(line)
asyncio.create_task(
self._broadcaster.emit("console_output", {"message": line})
)
def get_history(self) -> list[str]:
"""Get the current console history buffer.
Returns:
List of timestamped console messages
"""
return list(self._buffer)

108
src/web/managers/inventory.py Normal file
View File

@@ -0,0 +1,108 @@
"""Inventory manager for tracking drop campaigns and claiming progress."""
from __future__ import annotations
import asyncio
from typing import Any, TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.web.managers.cache import ImageCache
from src.models import DropsCampaign, TimedDrop
class InventoryManager:
"""Manages drop campaign inventory display in the web interface.
Tracks all active, upcoming, and expired campaigns with their drops,
broadcasting real-time updates as drops are mined and claimed.
"""
def __init__(self, broadcaster: WebSocketBroadcaster, cache: ImageCache):
self._broadcaster = broadcaster
self._cache = cache
self._campaigns: dict[str, dict[str, Any]] = {}
def clear(self):
"""Clear all campaigns from inventory."""
self._campaigns.clear()
asyncio.create_task(
self._broadcaster.emit("inventory_clear", {})
)
async def add_campaign(self, campaign: DropsCampaign):
"""Add a campaign to the inventory display.
Args:
campaign: The drop campaign to add
"""
# Get campaign image from cache
image_url = str(campaign.image_url)
drops_data = []
for drop in campaign.drops:
drops_data.append({
"id": drop.id,
"name": drop.name,
"current_minutes": drop.current_minutes,
"required_minutes": drop.required_minutes,
"progress": drop.progress,
"is_claimed": drop.is_claimed,
"can_claim": drop.can_claim,
"rewards": drop.rewards_text(),
"starts_at": drop.starts_at.isoformat(),
"ends_at": drop.ends_at.isoformat()
})
campaign_data = {
"id": campaign.id,
"name": campaign.name,
"game_name": campaign.game.name,
"image_url": image_url,
"starts_at": campaign.starts_at.isoformat(),
"ends_at": campaign.ends_at.isoformat(),
"linked": campaign.linked,
"active": campaign.active,
"upcoming": campaign.upcoming,
"expired": campaign.expired,
"claimed_drops": campaign.claimed_drops,
"total_drops": campaign.total_drops,
"drops": drops_data
}
self._campaigns[campaign.id] = campaign_data
await self._broadcaster.emit("campaign_add", campaign_data)
def update_drop(self, drop: TimedDrop):
"""Update a specific drop's progress within its campaign.
Args:
drop: The drop to update
"""
campaign_id = drop.campaign.id
if campaign_id in self._campaigns:
# Find and update the drop in the campaign
for drop_data in self._campaigns[campaign_id]["drops"]:
if drop_data["id"] == drop.id:
drop_data.update({
"current_minutes": drop.current_minutes,
"required_minutes": drop.required_minutes,
"progress": drop.progress,
"is_claimed": drop.is_claimed,
"can_claim": drop.can_claim
})
asyncio.create_task(
self._broadcaster.emit("drop_update", {
"campaign_id": campaign_id,
"drop": drop_data
})
)
break
def get_campaigns(self) -> list[dict[str, Any]]:
"""Get all campaigns in inventory.
Returns:
List of campaign data dictionaries
"""
return list(self._campaigns.values())

132
src/web/managers/login.py Normal file
View File

@@ -0,0 +1,132 @@
"""Login form manager for handling Twitch authentication."""
from __future__ import annotations
import asyncio
from dataclasses import dataclass
from typing import Any, TYPE_CHECKING
from src.i18n import _
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.web.gui_manager import WebGUIManager
@dataclass
class LoginData:
"""Container for login credentials submitted by the user."""
username: str
password: str
token: str
class LoginFormManager:
"""Manages login form and OAuth authentication flow in the web interface.
Handles both traditional username/password login and OAuth device code flow,
coordinating between the web client and the Twitch authentication system.
"""
def __init__(self, broadcaster: WebSocketBroadcaster, manager: WebGUIManager):
self._broadcaster = broadcaster
self._manager = manager
self._login_event = asyncio.Event()
self._login_data: LoginData | None = None
self._status = "Logged out"
self._user_id: int | None = None
self._oauth_pending: dict[str, str] | None = None # Store OAuth code for late-connecting clients
def clear(self, login: bool = False, password: bool = False, token: bool = False):
"""Clear login form fields on the client side.
Args:
login: Clear the login/username field
password: Clear the password field
token: Clear the 2FA token field
"""
asyncio.create_task(
self._broadcaster.emit("login_clear", {
"login": login,
"password": password,
"token": token
})
)
def update(self, status: str, user_id: int | None):
"""Update login status display.
Args:
status: Status message to display (e.g., "Logged in as...", "Login required")
user_id: Twitch user ID if logged in, None otherwise
"""
self._status = status
self._user_id = user_id
asyncio.create_task(
self._broadcaster.emit("login_status", {
"status": status,
"user_id": user_id
})
)
async def ask_login(self) -> LoginData:
"""Request login credentials from the user.
Returns:
LoginData containing submitted credentials
"""
self.update(_("gui", "login", "required"), None)
self._login_event.clear()
await self._broadcaster.emit("login_required", {})
# Use coro_unless_closed to handle shutdown during login
await self._manager.coro_unless_closed(self._login_event.wait())
return self._login_data
async def ask_enter_code(self, page_url, user_code: str):
"""Request OAuth device code entry from the user.
Displays the activation URL and code to the user, waiting for them
to complete the OAuth flow on Twitch's website.
Args:
page_url: URL where user should enter the code (e.g., twitch.tv/activate)
user_code: The device code to enter
"""
self.update(_("gui", "login", "required"), None)
self._login_event.clear()
# Store OAuth code for late-connecting clients
self._oauth_pending = {
"url": str(page_url),
"code": user_code
}
await self._broadcaster.emit("oauth_code_required", self._oauth_pending)
# Use coro_unless_closed to handle shutdown during login
await self._manager.coro_unless_closed(self._login_event.wait())
# Clear OAuth state after confirmation
self._oauth_pending = None
def submit_login(self, username: str, password: str, token: str = ''):
"""Submit login credentials (called by webapp when user submits form).
Args:
username: Twitch username or email
password: Account password
token: Optional 2FA token
"""
self._login_data = LoginData(username, password, token)
self._login_event.set()
def get_status(self) -> dict[str, Any]:
"""Get current login status for client synchronization.
Returns:
Dictionary with status, user_id, and optional oauth_pending data
"""
result = {
"status": self._status,
"user_id": self._user_id
}
# Include OAuth code if pending
if self._oauth_pending:
result["oauth_pending"] = self._oauth_pending
return result

75
src/web/managers/settings.py Normal file
View File

@@ -0,0 +1,75 @@
"""Settings manager for application configuration."""
from __future__ import annotations
import asyncio
from typing import Any, TYPE_CHECKING
from src.models.game import Game
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
from src.config.settings import Settings
class SettingsManager:
"""Manages application settings in the web interface.
Provides access to and modification of user preferences including
game priorities, proxy configuration, and UI preferences.
"""
def __init__(self, broadcaster: WebSocketBroadcaster, settings: Settings):
self._broadcaster = broadcaster
self._settings = settings
self._available_games: list[str] = []
def get_settings(self) -> dict[str, Any]:
"""Get current settings for display.
Returns:
Dictionary containing all user-configurable settings
"""
return {
"language": self._settings.language,
"dark_mode": self._settings.dark_mode,
"games_to_watch": list(self._settings.games_to_watch),
"games_available": self._available_games,
"proxy": str(self._settings.proxy),
"tray_notifications": self._settings.tray_notifications,
"connection_quality": self._settings.connection_quality
}
def update_settings(self, settings_data: dict[str, Any]):
"""Update settings from user input.
Args:
settings_data: Dictionary of settings to update
"""
if "games_to_watch" in settings_data:
self._settings.games_to_watch = settings_data["games_to_watch"]
if "dark_mode" in settings_data:
self._settings.dark_mode = settings_data["dark_mode"]
if "connection_quality" in settings_data:
self._settings.connection_quality = settings_data["connection_quality"]
if "proxy" in settings_data:
self._settings.proxy = settings_data["proxy"]
if "tray_notifications" in settings_data:
self._settings.tray_notifications = settings_data["tray_notifications"]
self._settings.alter()
asyncio.create_task(
self._broadcaster.emit("settings_updated", self.get_settings())
)
def set_games(self, games: set[Game]):
"""Update the list of available games for settings panel.
Args:
games: Set of Game objects discovered from campaigns
"""
# Store and broadcast available games for settings panel
game_names = sorted([g.name for g in games])
self._available_games = game_names
asyncio.create_task(
self._broadcaster.emit("games_available", {"games": game_names})
)

74
src/web/managers/status.py Normal file
View File

@@ -0,0 +1,74 @@
"""Status managers for application and WebSocket connection status."""
from __future__ import annotations
import asyncio
from typing import Any, TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
class StatusManager:
"""Manages main application status display in the web interface.
Tracks and broadcasts the current status message shown to users
(e.g., "Mining drops...", "Fetching inventory...", etc.).
"""
def __init__(self, broadcaster: WebSocketBroadcaster):
self._broadcaster = broadcaster
self._current_status = "Initializing..."
def update(self, status: str):
"""Update the current status and broadcast to all clients."""
self._current_status = status
asyncio.create_task(
self._broadcaster.emit("status_update", {"status": status})
)
def get(self) -> str:
"""Get the current status message."""
return self._current_status
class WebsocketStatusManager:
"""Manages WebSocket connection status tracking and display.
Tracks the status and topic count of each WebSocket connection in the pool,
providing real-time updates about connection health to the web interface.
"""
def __init__(self, broadcaster: WebSocketBroadcaster):
self._broadcaster = broadcaster
self._websockets: dict[int, dict[str, Any]] = {}
def update(self, idx: int, status: str | None = None, topics: int | None = None):
"""Update a specific websocket's status and/or topic count.
Args:
idx: WebSocket index/ID
status: Optional status string (e.g., "Connected", "Reconnecting")
topics: Optional number of topics this WebSocket is subscribed to
"""
if status is None and topics is None:
return # Nothing to update
if idx not in self._websockets:
self._websockets[idx] = {"status": "Unknown", "topics": 0}
if status is not None:
self._websockets[idx]["status"] = status
if topics is not None:
self._websockets[idx]["topics"] = topics
# Broadcast the update
asyncio.create_task(
self._broadcaster.emit("websocket_status", {
"idx": idx,
"status": self._websockets[idx]["status"],
"topics": self._websockets[idx]["topics"],
"total_websockets": len(self._websockets),
"total_topics": sum(ws["topics"] for ws in self._websockets.values())
})
)

55
src/web/managers/tray.py Normal file
View File

@@ -0,0 +1,55 @@
"""Tray icon stub for web-based GUI (no system tray in browser)."""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.web.managers.broadcaster import WebSocketBroadcaster
class TrayIconStub:
"""Stub implementation for system tray icon functionality.
In web mode, traditional system tray operations are not applicable.
This class provides a compatible interface that translates tray
operations into browser notifications and UI indicators.
"""
def __init__(self, broadcaster: WebSocketBroadcaster):
self._broadcaster = broadcaster
def change_icon(self, icon: str):
"""Change tray icon (translated to UI indicator in web mode).
Args:
icon: Icon name/identifier to change to
"""
# Broadcast icon change for potential UI indicators
asyncio.create_task(
self._broadcaster.emit("tray_icon_change", {"icon": icon})
)
def notify(self, message: str, title: str):
"""Send a system notification (translated to browser notification).
Args:
message: Notification message body
title: Notification title
"""
# Send browser notification
asyncio.create_task(
self._broadcaster.emit("notification", {
"title": title,
"message": message
})
)
def minimize(self):
"""Minimize to tray (no-op in web mode)."""
pass
def restore(self):
"""Restore from tray (no-op in web mode)."""
pass

16
src/websocket/__init__.py Normal file
View File

@@ -0,0 +1,16 @@
"""
Websocket layer for Twitch PubSub connections.
This module provides websocket connection management for subscribing to
Twitch PubSub topics and receiving real-time updates about drops, channels,
and stream states.
Classes:
Websocket: Manages a single websocket connection with topic subscriptions
WebsocketPool: Manages multiple websocket connections for topic distribution
"""
from src.websocket.websocket import Websocket
from src.websocket.pool import WebsocketPool
__all__ = ["Websocket", "WebsocketPool"]

132
src/websocket/pool.py Normal file
View File

@@ -0,0 +1,132 @@
from __future__ import annotations
import asyncio
import logging
from typing import Any, Literal, TYPE_CHECKING
from src.exceptions import MinerException
from src.config import MAX_WEBSOCKETS, WS_TOPICS_LIMIT
from src.websocket.websocket import Websocket
if TYPE_CHECKING:
from collections import abc
from src.core.client import Twitch
from src.config import WebsocketTopic
logger = logging.getLogger("TwitchDrops")
class WebsocketPool:
"""
Manages a pool of websocket connections to distribute topics across multiple connections.
Twitch limits the number of topics per websocket, so this pool automatically
creates additional websockets as needed to handle all subscribed topics.
"""
def __init__(self, twitch: Twitch):
"""
Initialize the websocket pool.
Args:
twitch: Twitch client instance
"""
self._twitch: Twitch = twitch
self._running = asyncio.Event()
self.websockets: list[Websocket] = []
@property
def running(self) -> bool:
"""Check if the pool is currently running."""
return self._running.is_set()
def wait_until_connected(self) -> abc.Coroutine[Any, Any, Literal[True]]:
"""Wait until the pool is running and connections are established."""
return self._running.wait()
async def start(self):
"""Start all websockets in the pool."""
self._running.set()
await asyncio.gather(*(ws.start() for ws in self.websockets))
async def stop(self, *, clear_topics: bool = False):
"""
Stop all websockets in the pool.
Args:
clear_topics: If True, clear all topics and remove websockets from GUI
"""
self._running.clear()
await asyncio.gather(*(ws.stop(remove=clear_topics) for ws in self.websockets))
def add_topics(self, topics: abc.Iterable[WebsocketTopic]):
"""
Add topics to the pool, distributing across websockets as needed.
Creates new websocket connections if existing ones are at capacity.
Raises MinerException if the maximum number of topics/websockets is reached.
Args:
topics: Iterable of topics to add
Raises:
MinerException: If maximum topics limit is reached
"""
# ensure no topics end up duplicated
topics_set = set(topics)
if not topics_set:
# nothing to add
return
topics_set.difference_update(*(ws.topics.values() for ws in self.websockets))
if not topics_set:
# none left to add
return
for ws_idx in range(MAX_WEBSOCKETS):
if ws_idx < len(self.websockets):
# just read it back
ws = self.websockets[ws_idx]
else:
# create new
ws = Websocket(self, ws_idx)
if self.running:
ws.start_nowait()
self.websockets.append(ws)
# ask websocket to take any topics it can - this modifies the set in-place
ws.add_topics(topics_set)
# see if there's any leftover topics for the next websocket connection
if not topics_set:
return
# if we're here, there were leftover topics after filling up all websockets
raise MinerException("Maximum topics limit has been reached")
def remove_topics(self, topics: abc.Iterable[str]):
"""
Remove topics from the pool.
Automatically stops and removes websockets that become empty after topic removal.
Recycles topics from removed websockets to maintain efficient connection usage.
Args:
topics: Iterable of topic strings to remove
"""
topics_set = set(topics)
if not topics_set:
# nothing to remove
return
for ws in self.websockets:
ws.remove_topics(topics_set)
# count up all the topics - if we happen to have more websockets connected than needed,
# stop the last one and recycle topics from it - repeat until we have enough
recycled_topics: list[WebsocketTopic] = []
while True:
count = sum(len(ws.topics) for ws in self.websockets)
if count <= (len(self.websockets) - 1) * WS_TOPICS_LIMIT:
ws = self.websockets.pop()
recycled_topics.extend(ws.topics.values())
ws.stop_nowait(remove=True)
else:
break
if recycled_topics:
self.add_topics(recycled_topics)

189
websocket.py → src/websocket/websocket.py
View File

@@ -5,14 +5,14 @@ import asyncio
import logging import logging
from time import time from time import time
from contextlib import suppress from contextlib import suppress
from typing import Any, Literal, TYPE_CHECKING from typing import TYPE_CHECKING
import aiohttp import aiohttp
from translate import _ from src.i18n import _
from exceptions import MinerException, WebsocketClosed from src.exceptions import WebsocketClosed
from constants import PING_INTERVAL, PING_TIMEOUT, MAX_WEBSOCKETS, WS_TOPICS_LIMIT from src.config import PING_INTERVAL, PING_TIMEOUT, WS_TOPICS_LIMIT
from utils import ( from src.utils import (
CHARS_ASCII, CHARS_ASCII,
task_wrapper, task_wrapper,
create_nonce, create_nonce,
@@ -23,11 +23,9 @@ from utils import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from collections import abc from src.core.client import Twitch
from src.web.gui_manager import WebsocketStatus
from twitch import Twitch from src.config import JsonType, WebsocketTopic
from gui import WebsocketStatus
from constants import JsonType, WebsocketTopic
WSMsgType = aiohttp.WSMsgType WSMsgType = aiohttp.WSMsgType
@@ -36,7 +34,23 @@ ws_logger = logging.getLogger("TwitchDrops.websocket")
class Websocket: class Websocket:
def __init__(self, pool: WebsocketPool, index: int): """
Manages a single websocket connection to Twitch's PubSub service.
Handles connection lifecycle, topic subscriptions, ping/pong heartbeat,
and message routing to registered topic handlers.
"""
def __init__(self, pool, index: int):
"""
Initialize a Websocket instance.
Args:
pool: WebsocketPool instance that owns this websocket
index: Numeric index for logging and identification
"""
from src.websocket.pool import WebsocketPool
self._pool: WebsocketPool = pool self._pool: WebsocketPool = pool
self._twitch: Twitch = pool._twitch self._twitch: Twitch = pool._twitch
self._ws_gui: WebsocketStatus = self._twitch.gui.websockets self._ws_gui: WebsocketStatus = self._twitch.gui.websockets
@@ -63,31 +77,49 @@ class Websocket:
@property @property
def connected(self) -> bool: def connected(self) -> bool:
"""Check if the websocket is currently connected."""
return self._ws.has_value() return self._ws.has_value()
def wait_until_connected(self): def wait_until_connected(self):
"""Wait until the websocket is connected."""
return self._ws.wait() return self._ws.wait()
def set_status(self, status: str | None = None, refresh_topics: bool = False): def set_status(self, status: str | None = None, refresh_topics: bool = False):
"""
Update the websocket status in the GUI.
Args:
status: New status message, or None to keep current
refresh_topics: If True, update the topic count in the GUI
"""
self._twitch.gui.websockets.update( self._twitch.gui.websockets.update(
self._idx, status=status, topics=(len(self.topics) if refresh_topics else None) self._idx, status=status, topics=(len(self.topics) if refresh_topics else None)
) )
def request_reconnect(self): def request_reconnect(self):
"""Request a websocket reconnection."""
# reset our ping interval, so we send a PING after reconnect right away # reset our ping interval, so we send a PING after reconnect right away
self._next_ping = time() self._next_ping = time()
self._reconnect_requested.set() self._reconnect_requested.set()
async def start(self): async def start(self):
"""Start the websocket connection and wait until connected."""
async with self._state_lock: async with self._state_lock:
self.start_nowait() self.start_nowait()
await self.wait_until_connected() await self.wait_until_connected()
def start_nowait(self): def start_nowait(self):
"""Start the websocket connection without waiting."""
if self._handle_task is None or self._handle_task.done(): if self._handle_task is None or self._handle_task.done():
self._handle_task = asyncio.create_task(self._handle()) self._handle_task = asyncio.create_task(self._handle())
async def stop(self, *, remove: bool = False): async def stop(self, *, remove: bool = False):
"""
Stop the websocket connection.
Args:
remove: If True, clear topics and remove from GUI
"""
async with self._state_lock: async with self._state_lock:
if self._closed.is_set(): if self._closed.is_set():
return return
@@ -103,16 +135,31 @@ class Websocket:
if remove: if remove:
self.topics.clear() self.topics.clear()
self._topics_changed.set() self._topics_changed.set()
self._twitch.gui.websockets.remove(self._idx) # TODO: WebsocketStatusManager doesn't have a remove() method yet
# self._twitch.gui.websockets.remove(self._idx)
def stop_nowait(self, *, remove: bool = False): def stop_nowait(self, *, remove: bool = False):
# weird syntax but that's what we get for using a decorator for this """
# return type of 'task_wrapper' is a coro, so we need to instance it for the task Stop the websocket connection without waiting.
asyncio.create_task(task_wrapper(self.stop)(remove=remove))
Args:
remove: If True, clear topics and remove from GUI
"""
asyncio.create_task(self.stop(remove=remove))
async def _backoff_connect( async def _backoff_connect(
self, ws_url: str, **kwargs self, ws_url: str, **kwargs
) -> abc.AsyncGenerator[aiohttp.ClientWebSocketResponse, None]: ):
"""
Connect to websocket with exponential backoff retry logic.
Args:
ws_url: Websocket URL to connect to
**kwargs: Additional arguments passed to ExponentialBackoff
Yields:
Connected websocket instances
"""
session = await self._twitch.get_session() session = await self._twitch.get_session()
backoff = ExponentialBackoff(**kwargs) backoff = ExponentialBackoff(**kwargs)
if self._twitch.settings.proxy: if self._twitch.settings.proxy:
@@ -142,6 +189,7 @@ class Websocket:
@task_wrapper(critical=True) @task_wrapper(critical=True)
async def _handle(self): async def _handle(self):
"""Main websocket handler that manages connection lifecycle and message processing."""
# ensure we're logged in before connecting # ensure we're logged in before connecting
self.set_status(_("gui", "websocket", "initializing")) self.set_status(_("gui", "websocket", "initializing"))
await self._twitch.wait_until_login() await self._twitch.wait_until_login()
@@ -187,6 +235,7 @@ class Websocket:
ws_logger.warning(f"Websocket[{self._idx}] reconnecting...") ws_logger.warning(f"Websocket[{self._idx}] reconnecting...")
async def _handle_ping(self): async def _handle_ping(self):
"""Handle ping/pong heartbeat to keep connection alive."""
now = time() now = time()
if now >= self._next_ping: if now >= self._next_ping:
self._next_ping = now + PING_INTERVAL.total_seconds() self._next_ping = now + PING_INTERVAL.total_seconds()
@@ -198,6 +247,7 @@ class Websocket:
self.request_reconnect() self.request_reconnect()
async def _handle_topics(self): async def _handle_topics(self):
"""Handle topic subscription changes (LISTEN/UNLISTEN messages)."""
if not self._topics_changed.is_set(): if not self._topics_changed.is_set():
# nothing to do # nothing to do
return return
@@ -239,7 +289,13 @@ class Websocket:
async def _gather_recv(self, messages: list[JsonType], timeout: float = 0.5): async def _gather_recv(self, messages: list[JsonType], timeout: float = 0.5):
""" """
Gather incoming messages over the timeout specified. Gather incoming messages over the timeout specified.
Note that there's no return value - this modifies `messages` in-place.
Args:
messages: List to append received messages to (modified in-place)
timeout: How long to gather messages for in seconds
Raises:
WebsocketClosed: When the websocket connection closes
""" """
ws = self._ws.get_with_default(None) ws = self._ws.get_with_default(None)
assert ws is not None assert ws is not None
@@ -264,6 +320,12 @@ class Websocket:
ws_logger.error(f"Websocket[{self._idx}] error: Unknown message: {raw_message}") ws_logger.error(f"Websocket[{self._idx}] error: Unknown message: {raw_message}")
def _handle_message(self, message): def _handle_message(self, message):
"""
Route a received MESSAGE to the appropriate topic handler.
Args:
message: Websocket message dict with topic and message data
"""
# request the assigned topic to process the response # request the assigned topic to process the response
topic = self.topics.get(message["data"]["topic"]) topic = self.topics.get(message["data"]["topic"])
if topic is not None: if topic is not None:
@@ -271,9 +333,7 @@ class Websocket:
asyncio.create_task(topic(json.loads(message["data"]["message"]))) asyncio.create_task(topic(json.loads(message["data"]["message"])))
async def _handle_recv(self): async def _handle_recv(self):
""" """Handle receiving and processing messages from the websocket."""
Handle receiving messages from the websocket.
"""
# listen over 0.5s for incoming messages # listen over 0.5s for incoming messages
messages: list[JsonType] = [] messages: list[JsonType] = []
with suppress(asyncio.TimeoutError): with suppress(asyncio.TimeoutError):
@@ -297,6 +357,12 @@ class Websocket:
ws_logger.warning(f"Websocket[{self._idx}] received unknown payload: {message}") ws_logger.warning(f"Websocket[{self._idx}] received unknown payload: {message}")
def add_topics(self, topics_set: set[WebsocketTopic]): def add_topics(self, topics_set: set[WebsocketTopic]):
"""
Add topics to this websocket, up to the limit.
Args:
topics_set: Set of topics to add (modified in-place, removing added topics)
"""
changed: bool = False changed: bool = False
while topics_set and len(self.topics) < WS_TOPICS_LIMIT: while topics_set and len(self.topics) < WS_TOPICS_LIMIT:
topic = topics_set.pop() topic = topics_set.pop()
@@ -306,6 +372,12 @@ class Websocket:
self._topics_changed.set() self._topics_changed.set()
def remove_topics(self, topics_set: set[str]): def remove_topics(self, topics_set: set[str]):
"""
Remove topics from this websocket.
Args:
topics_set: Set of topic strings to remove (modified in-place)
"""
existing = topics_set.intersection(self.topics.keys()) existing = topics_set.intersection(self.topics.keys())
if not existing: if not existing:
# nothing to remove from here # nothing to remove from here
@@ -316,80 +388,15 @@ class Websocket:
self._topics_changed.set() self._topics_changed.set()
async def send(self, message: JsonType): async def send(self, message: JsonType):
"""
Send a JSON message to the websocket.
Args:
message: JSON-serializable message dict
"""
ws = self._ws.get_with_default(None) ws = self._ws.get_with_default(None)
assert ws is not None assert ws is not None
if message["type"] != "PING": if message["type"] != "PING":
message["nonce"] = create_nonce(CHARS_ASCII, 30) message["nonce"] = create_nonce(CHARS_ASCII, 30)
await ws.send_json(message, dumps=json_minify) await ws.send_json(message, dumps=json_minify)
ws_logger.debug(f"Websocket[{self._idx}] sent: {message}") ws_logger.debug(f"Websocket[{self._idx}] sent: {message}")
class WebsocketPool:
def __init__(self, twitch: Twitch):
self._twitch: Twitch = twitch
self._running = asyncio.Event()
self.websockets: list[Websocket] = []
@property
def running(self) -> bool:
return self._running.is_set()
def wait_until_connected(self) -> abc.Coroutine[Any, Any, Literal[True]]:
return self._running.wait()
async def start(self):
self._running.set()
await asyncio.gather(*(ws.start() for ws in self.websockets))
async def stop(self, *, clear_topics: bool = False):
self._running.clear()
await asyncio.gather(*(ws.stop(remove=clear_topics) for ws in self.websockets))
def add_topics(self, topics: abc.Iterable[WebsocketTopic]):
# ensure no topics end up duplicated
topics_set = set(topics)
if not topics_set:
# nothing to add
return
topics_set.difference_update(*(ws.topics.values() for ws in self.websockets))
if not topics_set:
# none left to add
return
for ws_idx in range(MAX_WEBSOCKETS):
if ws_idx < len(self.websockets):
# just read it back
ws = self.websockets[ws_idx]
else:
# create new
ws = Websocket(self, ws_idx)
if self.running:
ws.start_nowait()
self.websockets.append(ws)
# ask websocket to take any topics it can - this modifies the set in-place
ws.add_topics(topics_set)
# see if there's any leftover topics for the next websocket connection
if not topics_set:
return
# if we're here, there were leftover topics after filling up all websockets
raise MinerException("Maximum topics limit has been reached")
def remove_topics(self, topics: abc.Iterable[str]):
topics_set = set(topics)
if not topics_set:
# nothing to remove
return
for ws in self.websockets:
ws.remove_topics(topics_set)
# count up all the topics - if we happen to have more websockets connected than needed,
# stop the last one and recycle topics from it - repeat until we have enough
recycled_topics: list[WebsocketTopic] = []
while True:
count = sum(len(ws.topics) for ws in self.websockets)
if count <= (len(self.websockets) - 1) * WS_TOPICS_LIMIT:
ws = self.websockets.pop()
recycled_topics.extend(ws.topics.values())
ws.stop_nowait(remove=True)
else:
break
if recycled_topics:
self.add_topics(recycled_topics)

436
utils.py
View File

@@ -1,436 +0,0 @@
from __future__ import annotations
import io
import os
import re
import sys
import json
import random
import string
import asyncio
import logging
import traceback
import webbrowser
import tkinter as tk
from enum import Enum
from pathlib import Path
from functools import wraps
from contextlib import suppress
from functools import cached_property
from datetime import datetime, timezone
from collections import abc, OrderedDict
from typing import Any, Literal, Callable, Generic, Mapping, TypeVar, ParamSpec, cast
from yarl import URL
from PIL.ImageTk import PhotoImage
from PIL import Image as Image_module
from exceptions import ExitRequest, ReloadRequest
from constants import IS_PACKAGED, JsonType, PriorityMode
from constants import _resource_path as resource_path # noqa
_T = TypeVar("_T") # type
_D = TypeVar("_D") # default
_P = ParamSpec("_P") # params
_JSON_T = TypeVar("_JSON_T", bound=Mapping[Any, Any])
logger = logging.getLogger("TwitchDrops")
def set_root_icon(root: tk.Tk, image_path: Path | str) -> None:
with Image_module.open(image_path) as image:
icon_photo = PhotoImage(master=root, image=image)
root.iconphoto(True, icon_photo) # type: ignore[arg-type]
# keep a reference to the PhotoImage to avoid the ResourceWarning
root._icon_image = icon_photo # type: ignore[attr-defined]
async def first_to_complete(coros: abc.Iterable[abc.Coroutine[Any, Any, _T]]) -> _T:
# In Python 3.11, we need to explicitly wrap awaitables
tasks = [asyncio.ensure_future(coro) for coro in coros]
done: set[asyncio.Task[Any]]
pending: set[asyncio.Task[Any]]
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
for task in pending:
task.cancel()
return await next(iter(done))
def chunk(to_chunk: abc.Iterable[_T], chunk_length: int) -> abc.Generator[list[_T], None, None]:
list_to_chunk = list(to_chunk)
for i in range(0, len(list_to_chunk), chunk_length):
yield list_to_chunk[i:i + chunk_length]
def format_traceback(exc: BaseException, **kwargs: Any) -> str:
"""
Like `traceback.print_exc` but returns a string. Uses the passed-in exception.
Any additional `**kwargs` are passed to the underlaying `traceback.format_exception`.
"""
return ''.join(traceback.format_exception(type(exc), exc, **kwargs))
def lock_file(path: Path) -> tuple[bool, io.TextIOWrapper]:
file = path.open('w', encoding="utf8")
file.write('ツ')
file.flush()
if sys.platform == "win32":
import msvcrt
try:
# we need to lock at least one byte for this to work
msvcrt.locking(file.fileno(), msvcrt.LK_NBLCK, max(path.stat().st_size, 1))
except Exception:
return False, file
return True, file
if sys.platform == "linux":
import fcntl
try:
fcntl.lockf(file, fcntl.LOCK_EX | fcntl.LOCK_NB)
except Exception:
return False, file
return True, file
# for unsupported systems, just always return True
return True, file
def json_minify(data: JsonType | list[JsonType]) -> str:
"""
Returns minified JSON for payload usage.
"""
return json.dumps(data, separators=(',', ':'))
def timestamp(string: str) -> datetime:
try:
return datetime.strptime(string, "%Y-%m-%dT%H:%M:%S.%fZ").replace(tzinfo=timezone.utc)
except ValueError:
return datetime.strptime(string, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
CHARS_ASCII = string.ascii_letters + string.digits
CHARS_HEX_LOWER = string.digits + "abcdef"
CHARS_HEX_UPPER = string.digits + "ABCDEF"
def create_nonce(chars: str, length: int) -> str:
return ''.join(random.choices(chars, k=length))
def deduplicate(iterable: abc.Iterable[_T]) -> list[_T]:
return list(OrderedDict.fromkeys(iterable).keys())
def task_wrapper(
afunc: abc.Callable[_P, abc.Coroutine[Any, Any, _T]] | None = None, *, critical: bool = False
):
def decorator(
afunc: abc.Callable[_P, abc.Coroutine[Any, Any, _T]]
) -> abc.Callable[_P, abc.Coroutine[Any, Any, _T]]:
@wraps(afunc)
async def wrapper(*args: _P.args, **kwargs: _P.kwargs):
try:
await afunc(*args, **kwargs)
except (ExitRequest, ReloadRequest):
pass
except Exception:
logger.exception(f"Exception in {afunc.__name__} task")
if critical:
# critical task's death should trigger a termination.
# there isn't an easy and sure way to obtain the Twitch instance here,
# but we can improvise finding it
from twitch import Twitch # cyclic import
probe = args and args[0] or None # extract from 'self' arg
if isinstance(probe, Twitch):
probe.close()
elif probe is not None:
probe = getattr(probe, "_twitch", None) # extract from '_twitch' attr
if isinstance(probe, Twitch):
probe.close()
raise # raise up to the wrapping task
return wrapper
if afunc is None:
return decorator
return decorator(afunc)
def invalidate_cache(instance, *attrnames):
"""
To be used to invalidate `functools.cached_property`.
"""
for name in attrnames:
with suppress(AttributeError):
delattr(instance, name)
def _serialize(obj: Any) -> Any:
# convert data
d: int | str | float | list[Any] | JsonType
if isinstance(obj, datetime):
if obj.tzinfo is None:
# assume naive objects are UTC
obj = obj.replace(tzinfo=timezone.utc)
d = obj.timestamp()
elif isinstance(obj, set):
d = list(obj)
elif isinstance(obj, Enum):
# NOTE: IntEnum cannot be used, as it will get serialized as a plain integer,
# then loaded back as an integer as well.
d = obj.value
elif isinstance(obj, URL):
d = str(obj)
else:
raise TypeError(obj)
# store with type
return {
"__type": type(obj).__name__,
"data": d,
}
_MISSING = object()
SERIALIZE_ENV: dict[str, Callable[[Any], object]] = {
"set": set,
"URL": URL,
"PriorityMode": PriorityMode,
"datetime": lambda d: datetime.fromtimestamp(d, timezone.utc),
}
def _remove_missing(obj: JsonType) -> JsonType:
# this modifies obj in place, but we return it just in case
for key, value in obj.copy().items():
if value is _MISSING:
del obj[key]
elif isinstance(value, dict):
_remove_missing(value)
if not value:
# the dict is empty now, so remove it's key entirely
del obj[key]
return obj
def _deserialize(obj: JsonType) -> Any:
if "__type" in obj:
obj_type = obj["__type"]
if obj_type in SERIALIZE_ENV:
return SERIALIZE_ENV[obj_type](obj["data"])
else:
return _MISSING
return obj
def merge_json(obj: JsonType, template: Mapping[Any, Any]) -> None:
# NOTE: This modifies object in place
for k, v in list(obj.items()):
if k not in template:
# unknown key: overwrite from template
del obj[k]
elif type(v) is not type(template[k]):
# types don't match: overwrite from template
obj[k] = template[k]
elif isinstance(v, dict):
assert isinstance(template[k], dict)
merge_json(v, template[k])
# ensure the object is not missing any keys
for k in template.keys():
if k not in obj:
obj[k] = template[k]
def json_load(path: Path, defaults: _JSON_T, *, merge: bool = True) -> _JSON_T:
defaults_dict: JsonType = dict(defaults)
if path.exists():
with open(path, 'r', encoding="utf8") as file:
combined: JsonType = _remove_missing(json.load(file, object_hook=_deserialize))
if merge:
merge_json(combined, defaults_dict)
else:
combined = defaults_dict
return cast(_JSON_T, combined)
def json_save(path: Path, contents: Mapping[Any, Any], *, sort: bool = False) -> None:
with open(path, 'w', encoding="utf8") as file:
json.dump(contents, file, default=_serialize, sort_keys=sort, indent=4)
def webopen(url: URL | str):
url_str = str(url)
if IS_PACKAGED and sys.platform == "linux":
# https://pyinstaller.org/en/stable/
# runtime-information.html#ld-library-path-libpath-considerations
# NOTE: All 4 cases need to be handled here: either of the two values can be there or not.
ld_env = "LD_LIBRARY_PATH"
ld_path_curr = os.environ.get(ld_env)
ld_path_orig = os.environ.get(f"{ld_env}_ORIG")
if ld_path_orig is not None:
os.environ[ld_env] = ld_path_orig
elif ld_path_curr is not None:
# pop current
os.environ.pop(ld_env)
webbrowser.open_new_tab(url_str)
if ld_path_curr is not None:
os.environ[ld_env] = ld_path_curr
elif ld_path_orig is not None:
# pop original
os.environ.pop(ld_env)
else:
webbrowser.open_new_tab(url_str)
class ExponentialBackoff:
def __init__(
self,
*,
base: float = 2,
variance: float | tuple[float, float] = 0.1,
shift: float = 0,
maximum: float = 300,
):
if base <= 1:
raise ValueError("Base has to be greater than 1")
self.steps: int = 0
self.base: float = float(base)
self.shift: float = float(shift)
self.maximum: float = float(maximum)
self.variance_min: float
self.variance_max: float
if isinstance(variance, tuple):
self.variance_min, self.variance_max = variance
else:
self.variance_min = 1 - variance
self.variance_max = 1 + variance
@property
def exp(self) -> int:
return max(0, self.steps - 1)
def __iter__(self) -> abc.Iterator[float]:
return self
def __next__(self) -> float:
value: float = (
pow(self.base, self.steps)
* random.uniform(self.variance_min, self.variance_max)
+ self.shift
)
if value > self.maximum:
return self.maximum
# NOTE: variance can cause the returned value to be lower than the previous one already,
# so this should be safe to move past the first return,
# to prevent the exponent from getting very big after reaching max and many iterations
self.steps += 1
return value
def reset(self) -> None:
self.steps = 0
class RateLimiter:
def __init__(self, *, capacity: int, window: int):
self.total: int = 0
self.concurrent: int = 0
self.window: int = window
self.capacity: int = capacity
self._reset_task: asyncio.Task[None] | None = None
self._cond: asyncio.Condition = asyncio.Condition()
def __repr__(self) -> str:
return f"{self.__class__.__name__}({self.concurrent}/{self.total}/{self.capacity})"
def __del__(self) -> None:
if self._reset_task is not None:
self._reset_task.cancel()
def _can_proceed(self) -> bool:
return max(self.total, self.concurrent) < self.capacity
async def __aenter__(self):
async with self._cond:
await self._cond.wait_for(self._can_proceed)
self.total += 1
self.concurrent += 1
if self._reset_task is None:
self._reset_task = asyncio.create_task(self._rtask())
async def __aexit__(self, exc_type, exc, tb):
self.concurrent -= 1
async with self._cond:
self._cond.notify(self.capacity - self.concurrent)
async def _reset(self) -> None:
if self._reset_task is not None:
self._reset_task = None
async with self._cond:
self.total = 0
if self.concurrent < self.capacity:
self._cond.notify(self.capacity - self.concurrent)
async def _rtask(self) -> None:
await asyncio.sleep(self.window)
await self._reset()
class AwaitableValue(Generic[_T]):
def __init__(self):
self._value: _T
self._event = asyncio.Event()
def has_value(self) -> bool:
return self._event.is_set()
def wait(self) -> abc.Coroutine[Any, Any, Literal[True]]:
return self._event.wait()
def get_with_default(self, default: _D) -> _T | _D:
if self._event.is_set():
return self._value
return default
async def get(self) -> _T:
await self._event.wait()
return self._value
def set(self, value: _T) -> None:
self._value = value
self._event.set()
def clear(self) -> None:
self._event.clear()
class Game:
def __init__(self, data: JsonType):
self.id: int = int(data["id"])
self.name: str = data.get("displayName") or data["name"]
if "slug" in data:
self.slug = data["slug"]
def __str__(self) -> str:
return self.name
def __repr__(self) -> str:
return f"Game({self.id}, {self.name})"
def __eq__(self, other: object) -> bool:
if isinstance(other, self.__class__):
return self.id == other.id
return NotImplemented
def __hash__(self) -> int:
return self.id
@cached_property
def slug(self) -> str:
"""
Converts the game name into a slug, useable for the GQL API.
"""
# remove specific characters
slug_text = re.sub(r'\'', '', self.name.lower())
# remove non alpha-numeric characters
slug_text = re.sub(r'\W+', '-', slug_text)
# strip and collapse dashes
slug_text = re.sub(r'-{2,}', '-', slug_text.strip('-'))
return slug_text

177
web/index.html Normal file
View File

@@ -0,0 +1,177 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Twitch Drops Miner</title>
<link rel="stylesheet" href="/static/styles.css">
<script src="https://cdn.socket.io/4.5.4/socket.io.min.js"></script>
</head>
<body>
<div class="container">
<header>
<h1>🎮 Twitch Drops Miner</h1>
<div class="status-bar">
<span id="status-text">Initializing...</span>
<span id="connection-indicator" class="connected">● Connected</span>
</div>
</header>
<nav class="tabs">
<button class="tab-button active" data-tab="main">Main</button>
<button class="tab-button" data-tab="inventory">Inventory</button>
<button class="tab-button" data-tab="settings">Settings</button>
<button class="tab-button" data-tab="help">Help</button>
</nav>
<!-- Main Tab -->
<div id="main-tab" class="tab-content active">
<div class="main-grid">
<!-- Login Section -->
<section class="panel login-panel">
<h2>Login</h2>
<div id="login-status" class="login-status">Not logged in</div>
<div id="login-form" style="display: none;">
<input type="text" id="username" placeholder="Username" />
<input type="password" id="password" placeholder="Password" />
<input type="text" id="2fa-token" placeholder="2FA Token (optional)" />
<button id="login-button">Login</button>
</div>
<div id="oauth-code-display" style="display: none;">
<p>Enter this code at: <a id="oauth-url" href="#" target="_blank">Twitch Activate</a></p>
<div class="oauth-code" id="oauth-code"></div>
<button id="oauth-confirm">I've entered the code</button>
</div>
</section>
<!-- Current Drop Progress -->
<section class="panel progress-panel">
<h2>Current Drop</h2>
<div id="drop-display">
<div id="no-drop-message">No active drop</div>
<div id="drop-info" style="display: none;">
<div class="drop-name" id="drop-name"></div>
<div class="drop-game" id="drop-game"></div>
<div class="progress-bar">
<div class="progress-fill" id="progress-fill"></div>
</div>
<div class="progress-text" id="progress-text"></div>
<div class="progress-time" id="progress-time"></div>
</div>
</div>
</section>
<!-- Console Output -->
<section class="panel console-panel">
<h2>Console Output</h2>
<div id="console-output" class="console-output"></div>
</section>
<!-- Channels List -->
<section class="panel channels-panel">
<h2>Channels</h2>
<div id="channels-list" class="channels-list"></div>
</section>
</div>
</div>
<!-- Inventory Tab -->
<div id="inventory-tab" class="tab-content">
<div class="inventory-grid" id="inventory-grid">
<p class="empty-message">No campaigns loaded yet...</p>
</div>
</div>
<!-- Settings Tab -->
<div id="settings-tab" class="tab-content">
<div class="settings-container">
<section class="settings-section">
<h2>General Settings</h2>
<label>
<input type="checkbox" id="dark-mode"> Dark Mode
</label>
<label>
Language:
<select id="language">
<option value="en">English</option>
</select>
</label>
<label>
Connection Quality:
<input type="number" id="connection-quality" min="1" max="6" value="1">
</label>
</section>
<section class="settings-section">
<h2>Games to Watch</h2>
<p class="help-text">Select games to watch. Order matters - drag to reorder priority (top = highest priority).</p>
<div class="games-filter">
<input type="text" id="games-filter" placeholder="Search games..." />
<div class="filter-actions">
<button id="select-all-btn" class="small-btn">Select All</button>
<button id="deselect-all-btn" class="small-btn">Deselect All</button>
</div>
</div>
<div class="games-container">
<div class="selected-games">
<h3>Selected Games (drag to reorder)</h3>
<div id="selected-games-list" class="sortable-list"></div>
</div>
<div class="available-games">
<h3>Available Games</h3>
<div id="available-games-list" class="checkbox-list"></div>
</div>
</div>
</section>
<section class="settings-section">
<h2>Actions</h2>
<button id="reload-btn" class="action-button">Reload Campaigns</button>
</section>
</div>
</div>
<!-- Help Tab -->
<div id="help-tab" class="tab-content">
<div class="help-content">
<h2>About Twitch Drops Miner</h2>
<p>This application automatically mines timed Twitch drops without downloading stream data.</p>
<h3>How to Use</h3>
<ol>
<li>Login using your Twitch account (OAuth device code flow)</li>
<li>Link your accounts at <a href="https://www.twitch.tv/drops/campaigns" target="_blank">twitch.tv/drops/campaigns</a></li>
<li>The miner will automatically discover campaigns and start mining</li>
<li>Configure priority games in Settings to focus on what you want</li>
<li>Monitor progress in the Main and Inventory tabs</li>
</ol>
<h3>Features</h3>
<ul>
<li>Stream-less drop mining - saves bandwidth</li>
<li>Game priority and exclusion lists</li>
<li>Tracks up to 199 channels simultaneously</li>
<li>Automatic channel switching</li>
<li>Real-time progress tracking</li>
</ul>
<h3>Important Notes</h3>
<ul>
<li>Do not watch streams on the same account while mining</li>
<li>Keep your cookies.jar file secure</li>
<li>Requires linked game accounts for drops</li>
</ul>
<div class="help-links">
<a href="https://github.com/DevilXD/TwitchDropsMiner" target="_blank">GitHub Repository</a>
</div>
</div>
</div>
</div>
<script src="/static/app.js"></script>
</body>
</html>

724
web/static/app.js Normal file
View File

@@ -0,0 +1,724 @@
// Twitch Drops Miner Web Client
// Socket.IO and API communication
// Global state
const state = {
connected: false,
channels: {},
campaigns: {},
settings: {},
currentDrop: null
};
// Initialize Socket.IO connection
const socket = io({
transports: ['websocket', 'polling'],
reconnection: true,
reconnectionDelay: 1000,
reconnectionDelayMax: 5000,
reconnectionAttempts: Infinity
});
// ==================== Socket.IO Event Handlers ====================
socket.on('connect', () => {
console.log('Connected to server');
state.connected = true;
document.getElementById('connection-indicator').textContent = '● Connected';
document.getElementById('connection-indicator').className = 'connected';
});
socket.on('disconnect', () => {
console.log('Disconnected from server');
state.connected = false;
document.getElementById('connection-indicator').textContent = '● Disconnected';
document.getElementById('connection-indicator').className = 'disconnected';
});
socket.on('initial_state', (data) => {
console.log('Received initial state', data);
if (data.status) updateStatus(data.status);
if (data.channels) data.channels.forEach(ch => updateChannel(ch));
if (data.campaigns) data.campaigns.forEach(camp => addCampaign(camp));
if (data.console) data.console.forEach(line => addConsoleLineRaw(line));
if (data.settings) updateSettingsUI(data.settings);
if (data.login) updateLoginStatus(data.login);
});
socket.on('status_update', (data) => {
updateStatus(data.status);
});
socket.on('console_output', (data) => {
addConsoleLine(data.message);
});
socket.on('channel_add', (data) => {
updateChannel(data);
});
socket.on('channel_update', (data) => {
updateChannel(data);
});
socket.on('channel_remove', (data) => {
removeChannel(data.id);
});
socket.on('channels_clear', () => {
clearChannels();
});
socket.on('channel_watching', (data) => {
setWatchingChannel(data.id);
});
socket.on('channel_watching_clear', () => {
clearWatchingChannel();
});
socket.on('drop_progress', (data) => {
updateDropProgress(data);
});
socket.on('drop_progress_stop', () => {
clearDropProgress();
});
socket.on('campaign_add', (data) => {
addCampaign(data);
});
socket.on('inventory_clear', () => {
clearInventory();
});
socket.on('drop_update', (data) => {
updateDrop(data.campaign_id, data.drop);
});
socket.on('login_required', () => {
showLoginForm();
});
socket.on('oauth_code_required', (data) => {
showOAuthCode(data.url, data.code);
});
socket.on('login_status', (data) => {
updateLoginStatus(data);
});
socket.on('login_clear', (data) => {
if (data.login) document.getElementById('username').value = '';
if (data.password) document.getElementById('password').value = '';
if (data.token) document.getElementById('2fa-token').value = '';
});
socket.on('settings_updated', (data) => {
updateSettingsUI(data);
});
socket.on('games_available', (data) => {
state.availableGames = data.games;
});
socket.on('theme_change', (data) => {
if (data.dark_mode) {
document.body.classList.add('dark-mode');
} else {
document.body.classList.remove('dark-mode');
}
});
socket.on('notification', (data) => {
if ('Notification' in window && Notification.permission === 'granted') {
new Notification(data.title, {
body: data.message,
icon: '/static/icon.png'
});
}
});
socket.on('attention_required', (data) => {
if (data.sound) {
// Play notification sound
const audio = new Audio('/static/notification.mp3');
audio.play().catch(() => {});
}
// Flash title
flashTitle();
});
// ==================== UI Update Functions ====================
function updateStatus(status) {
document.getElementById('status-text').textContent = status;
}
function addConsoleLine(message) {
addConsoleLineRaw(message);
}
function addConsoleLineRaw(line) {
const console = document.getElementById('console-output');
const div = document.createElement('div');
div.textContent = line;
console.appendChild(div);
// Auto-scroll to bottom
console.scrollTop = console.scrollHeight;
// Limit lines
while (console.children.length > 1000) {
console.removeChild(console.firstChild);
}
}
function updateChannel(channelData) {
state.channels[channelData.id] = channelData;
renderChannels();
}
function removeChannel(channelId) {
delete state.channels[channelId];
renderChannels();
}
function clearChannels() {
state.channels = {};
renderChannels();
}
function setWatchingChannel(channelId) {
Object.values(state.channels).forEach(ch => ch.watching = false);
if (state.channels[channelId]) {
state.channels[channelId].watching = true;
}
renderChannels();
}
function clearWatchingChannel() {
Object.values(state.channels).forEach(ch => ch.watching = false);
renderChannels();
}
function renderChannels() {
const container = document.getElementById('channels-list');
container.innerHTML = '';
const channels = Object.values(state.channels);
if (channels.length === 0) {
container.innerHTML = '<p class="empty-message">No channels tracked yet...</p>';
return;
}
// Sort: watching first, then online, then by viewers
channels.sort((a, b) => {
if (a.watching !== b.watching) return b.watching ? 1 : -1;
if (a.online !== b.online) return b.online ? 1 : -1;
return (b.viewers || 0) - (a.viewers || 0);
});
channels.forEach(channel => {
const div = document.createElement('div');
div.className = 'channel-item';
if (channel.watching) div.classList.add('watching');
if (channel.online) div.classList.add('online');
else div.classList.add('offline');
let badges = '';
if (channel.drops_enabled) badges += '<span class="channel-badge drops">DROPS</span>';
if (channel.acl_based) badges += '<span class="channel-badge acl">ACL</span>';
div.innerHTML = `
<div class="channel-name">${channel.name} ${badges}</div>
<div class="channel-info">
${channel.game || 'No game'} •
${channel.viewers !== null ? channel.viewers.toLocaleString() + ' viewers' : 'Offline'}
${channel.watching ? ' • <strong>WATCHING</strong>' : ''}
</div>
`;
div.onclick = () => selectChannel(channel.id);
container.appendChild(div);
});
}
function updateDropProgress(data) {
state.currentDrop = data;
document.getElementById('no-drop-message').style.display = 'none';
document.getElementById('drop-info').style.display = 'block';
document.getElementById('drop-name').textContent = data.drop_name;
document.getElementById('drop-game').textContent = `${data.campaign_name} (${data.game_name})`;
const progress = data.progress * 100;
const fill = document.getElementById('progress-fill');
fill.style.width = `${progress}%`;
fill.textContent = `${Math.round(progress)}%`;
document.getElementById('progress-text').textContent =
`${data.current_minutes} / ${data.required_minutes} minutes`;
// Update remaining time
updateRemainingTime(data.remaining_seconds);
}
function updateRemainingTime(seconds) {
const minutes = Math.floor(seconds / 60);
const secs = seconds % 60;
document.getElementById('progress-time').textContent =
`Time remaining: ${minutes}:${secs.toString().padStart(2, '0')}`;
if (seconds > 0) {
setTimeout(() => updateRemainingTime(seconds - 1), 1000);
}
}
function clearDropProgress() {
state.currentDrop = null;
document.getElementById('no-drop-message').style.display = 'block';
document.getElementById('drop-info').style.display = 'none';
}
function addCampaign(campaignData) {
state.campaigns[campaignData.id] = campaignData;
renderInventory();
}
function clearInventory() {
state.campaigns = {};
renderInventory();
}
function updateDrop(campaignId, dropData) {
if (state.campaigns[campaignId]) {
const drops = state.campaigns[campaignId].drops;
const index = drops.findIndex(d => d.id === dropData.id);
if (index !== -1) {
drops[index] = dropData;
renderInventory();
}
}
}
function renderInventory() {
const container = document.getElementById('inventory-grid');
container.innerHTML = '';
const campaigns = Object.values(state.campaigns);
if (campaigns.length === 0) {
container.innerHTML = '<p class="empty-message">No campaigns loaded yet...</p>';
return;
}
campaigns.forEach(campaign => {
const card = document.createElement('div');
card.className = 'campaign-card';
let statusClass = '';
let statusText = '';
if (campaign.active) {
statusClass = 'active';
statusText = 'Active';
} else if (campaign.upcoming) {
statusClass = 'upcoming';
statusText = 'Upcoming';
} else if (campaign.expired) {
statusClass = 'expired';
statusText = 'Expired';
}
const dropsHtml = campaign.drops.map(drop => `
<div class="drop-item ${drop.is_claimed ? 'claimed' : ''} ${drop.can_claim ? 'active' : ''}">
<div><strong>${drop.name}</strong></div>
<div>${drop.rewards}</div>
<div>${drop.current_minutes} / ${drop.required_minutes} minutes (${Math.round(drop.progress * 100)}%)</div>
${drop.is_claimed ? '<div>✓ Claimed</div>' : ''}
</div>
`).join('');
card.innerHTML = `
<div class="campaign-header">
<div class="campaign-game">${campaign.game_name}</div>
<div class="campaign-name">${campaign.name}</div>
</div>
<div class="campaign-status">
<span>${statusText}</span>
<span>${campaign.claimed_drops} / ${campaign.total_drops} claimed</span>
</div>
<div class="campaign-drops">
${dropsHtml}
</div>
`;
container.appendChild(card);
});
}
function showLoginForm() {
document.getElementById('login-form').style.display = 'block';
document.getElementById('oauth-code-display').style.display = 'none';
}
function showOAuthCode(url, code) {
document.getElementById('login-form').style.display = 'none';
document.getElementById('oauth-code-display').style.display = 'block';
document.getElementById('oauth-url').href = url;
document.getElementById('oauth-code').textContent = code;
}
function updateLoginStatus(data) {
const statusEl = document.getElementById('login-status');
if (data.user_id) {
statusEl.textContent = `Logged in (User ID: ${data.user_id})`;
statusEl.style.color = 'var(--success-color)';
document.getElementById('login-form').style.display = 'none';
document.getElementById('oauth-code-display').style.display = 'none';
} else {
statusEl.textContent = data.status || 'Not logged in';
statusEl.style.color = 'var(--text-secondary)';
// Check if OAuth is pending (for late-connecting clients)
if (data.oauth_pending) {
showOAuthCode(data.oauth_pending.url, data.oauth_pending.code);
}
}
}
function updateSettingsUI(settings) {
state.settings = settings;
document.getElementById('dark-mode').checked = settings.dark_mode || false;
document.getElementById('connection-quality').value = settings.connection_quality || 1;
if (settings.dark_mode) {
document.body.classList.add('dark-mode');
} else {
document.body.classList.remove('dark-mode');
}
// Update available games if provided in settings
if (settings.games_available) {
availableGames = new Set(settings.games_available);
}
// Update games to watch lists
renderGamesToWatch();
}
// ==================== Games to Watch Management ====================
let availableGames = new Set(); // All games from campaigns
let draggedElement = null;
socket.on('games_available', (data) => {
availableGames = new Set(data.games || []);
renderGamesToWatch();
});
function renderGamesToWatch() {
const selectedGames = state.settings.games_to_watch || [];
const filterText = document.getElementById('games-filter')?.value.toLowerCase() || '';
// Render selected games (sortable)
renderSelectedGames(selectedGames);
// Render available games (checkboxes for unselected games)
const unselectedGames = Array.from(availableGames)
.filter(game => !selectedGames.includes(game))
.filter(game => game.toLowerCase().includes(filterText))
.sort();
renderAvailableGames(unselectedGames, filterText);
}
function renderSelectedGames(games) {
const container = document.getElementById('selected-games-list');
if (!container) return;
container.innerHTML = '';
if (games.length === 0) {
container.innerHTML = '<p class="empty-message">No games selected. Check games below to add them.</p>';
return;
}
games.forEach((game, index) => {
const div = document.createElement('div');
div.className = 'sortable-item';
div.draggable = true;
div.dataset.game = game;
div.innerHTML = `
<span class="drag-handle">☰</span>
<span class="priority-number">${index + 1}</span>
<span class="game-name">${game}</span>
<button class="remove-btn" onclick="removeGameFromWatch('${game}')">✕</button>
`;
// Drag event handlers
div.addEventListener('dragstart', handleDragStart);
div.addEventListener('dragover', handleDragOver);
div.addEventListener('drop', handleDrop);
div.addEventListener('dragend', handleDragEnd);
container.appendChild(div);
});
}
function renderAvailableGames(games, filterText) {
const container = document.getElementById('available-games-list');
if (!container) return;
container.innerHTML = '';
if (games.length === 0) {
if (filterText) {
container.innerHTML = '<p class="empty-message">No games match your search.</p>';
} else {
container.innerHTML = '<p class="empty-message">All games are selected or no games available.</p>';
}
return;
}
games.forEach(game => {
const label = document.createElement('label');
label.className = 'game-checkbox';
label.innerHTML = `
<input type="checkbox" value="${game}" onchange="toggleGameWatch('${game}', this.checked)">
<span>${game}</span>
`;
container.appendChild(label);
});
}
// Drag and drop handlers
function handleDragStart(e) {
draggedElement = e.target;
e.target.classList.add('dragging');
e.dataTransfer.effectAllowed = 'move';
e.dataTransfer.setData('text/html', e.target.innerHTML);
}
function handleDragOver(e) {
if (e.preventDefault) {
e.preventDefault();
}
e.dataTransfer.dropEffect = 'move';
const target = e.target.closest('.sortable-item');
if (target && target !== draggedElement) {
const container = target.parentNode;
const allItems = [...container.querySelectorAll('.sortable-item')];
const draggedIndex = allItems.indexOf(draggedElement);
const targetIndex = allItems.indexOf(target);
if (draggedIndex < targetIndex) {
target.parentNode.insertBefore(draggedElement, target.nextSibling);
} else {
target.parentNode.insertBefore(draggedElement, target);
}
}
return false;
}
function handleDrop(e) {
if (e.stopPropagation) {
e.stopPropagation();
}
return false;
}
function handleDragEnd(e) {
e.target.classList.remove('dragging');
// Update the order in state
const container = document.getElementById('selected-games-list');
const items = container.querySelectorAll('.sortable-item');
const newOrder = Array.from(items).map(item => item.dataset.game);
state.settings.games_to_watch = newOrder;
// Re-render to update priority numbers
renderSelectedGames(newOrder);
// Save settings
saveSettings();
}
function toggleGameWatch(gameName, checked) {
const games = state.settings.games_to_watch || [];
if (checked && !games.includes(gameName)) {
games.push(gameName);
} else if (!checked) {
const index = games.indexOf(gameName);
if (index > -1) {
games.splice(index, 1);
}
}
state.settings.games_to_watch = games;
renderGamesToWatch();
saveSettings();
}
function removeGameFromWatch(gameName) {
const games = state.settings.games_to_watch || [];
const index = games.indexOf(gameName);
if (index > -1) {
games.splice(index, 1);
state.settings.games_to_watch = games;
renderGamesToWatch();
saveSettings();
}
}
function selectAllGames() {
state.settings.games_to_watch = Array.from(availableGames).sort();
renderGamesToWatch();
saveSettings();
}
function deselectAllGames() {
state.settings.games_to_watch = [];
renderGamesToWatch();
saveSettings();
}
function flashTitle() {
const originalTitle = document.title;
let count = 0;
const interval = setInterval(() => {
document.title = count % 2 === 0 ? '🔔 Attention!' : originalTitle;
count++;
if (count >= 10) {
document.title = originalTitle;
clearInterval(interval);
}
}, 1000);
}
// ==================== API Functions ====================
async function selectChannel(channelId) {
try {
await fetch('/api/channels/select', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({channel_id: channelId})
});
} catch (error) {
console.error('Failed to select channel:', error);
}
}
async function submitLogin() {
const username = document.getElementById('username').value;
const password = document.getElementById('password').value;
const token = document.getElementById('2fa-token').value;
try {
await fetch('/api/login', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({username, password, token})
});
} catch (error) {
console.error('Failed to submit login:', error);
}
}
async function confirmOAuth() {
// Signal that OAuth code has been entered
try {
await fetch('/api/oauth/confirm', {
method: 'POST'
});
// Hide the OAuth form and show waiting message
document.getElementById('oauth-code-display').style.display = 'none';
document.getElementById('login-status').textContent = 'Waiting for authentication...';
} catch (error) {
console.error('Failed to confirm OAuth:', error);
}
}
async function saveSettings() {
const settings = {
dark_mode: document.getElementById('dark-mode').checked,
connection_quality: parseInt(document.getElementById('connection-quality').value),
games_to_watch: state.settings.games_to_watch || []
};
try {
await fetch('/api/settings', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify(settings)
});
console.log('Settings saved automatically');
} catch (error) {
console.error('Failed to save settings:', error);
}
}
async function reloadCampaigns() {
try {
await fetch('/api/reload', {method: 'POST'});
} catch (error) {
console.error('Failed to reload:', error);
}
}
// ==================== Tab Management ====================
function switchTab(tabName) {
// Hide all tabs
document.querySelectorAll('.tab-content').forEach(tab => {
tab.classList.remove('active');
});
document.querySelectorAll('.tab-button').forEach(btn => {
btn.classList.remove('active');
});
// Show selected tab
document.getElementById(`${tabName}-tab`).classList.add('active');
document.querySelector(`[data-tab="${tabName}"]`).classList.add('active');
}
// ==================== Event Listeners ====================
document.addEventListener('DOMContentLoaded', () => {
// Tab switching
document.querySelectorAll('.tab-button').forEach(button => {
button.addEventListener('click', () => {
switchTab(button.dataset.tab);
});
});
// Login form
document.getElementById('login-button').addEventListener('click', submitLogin);
document.getElementById('oauth-confirm').addEventListener('click', confirmOAuth);
// Settings - auto-save on change
document.getElementById('dark-mode').addEventListener('change', (e) => {
// Apply dark mode immediately for instant feedback
if (e.target.checked) {
document.body.classList.add('dark-mode');
} else {
document.body.classList.remove('dark-mode');
}
// Then save settings
saveSettings();
});
document.getElementById('connection-quality').addEventListener('change', saveSettings);
document.getElementById('reload-btn').addEventListener('click', reloadCampaigns);
// Games to watch management
document.getElementById('select-all-btn').addEventListener('click', selectAllGames);
document.getElementById('deselect-all-btn').addEventListener('click', deselectAllGames);
document.getElementById('games-filter').addEventListener('input', renderGamesToWatch);
// Request notification permission
if ('Notification' in window && Notification.permission === 'default') {
Notification.requestPermission();
}
});

713
web/static/styles.css Normal file
View File

@@ -0,0 +1,713 @@
/* Global Styles */
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
:root {
--bg-primary: #ffffff;
--bg-secondary: #f5f5f5;
--bg-panel: #ffffff;
--text-primary: #333333;
--text-secondary: #666666;
--border-color: #dddddd;
--accent-color: #9146ff;
--success-color: #00c853;
--warning-color: #ffa726;
--error-color: #f44336;
--progress-bg: #e0e0e0;
}
body.dark-mode {
--bg-primary: #1a1a1a;
--bg-secondary: #2a2a2a;
--bg-panel: #2a2a2a;
--text-primary: #e0e0e0;
--text-secondary: #a0a0a0;
--border-color: #444444;
}
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
background-color: var(--bg-primary);
color: var(--text-primary);
line-height: 1.6;
}
.container {
max-width: 1400px;
margin: 0 auto;
padding: 20px;
}
/* Header */
header {
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px;
padding: 20px;
margin-bottom: 20px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
header h1 {
margin-bottom: 10px;
color: var(--accent-color);
}
.status-bar {
display: flex;
justify-content: space-between;
align-items: center;
font-size: 14px;
}
#connection-indicator {
font-weight: bold;
}
#connection-indicator.connected {
color: var(--success-color);
}
#connection-indicator.disconnected {
color: var(--error-color);
}
/* Tabs */
.tabs {
display: flex;
gap: 5px;
margin-bottom: 20px;
}
.tab-button {
flex: 1;
padding: 12px;
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px 8px 0 0;
cursor: pointer;
font-size: 16px;
color: var(--text-primary);
transition: background 0.2s;
}
.tab-button:hover {
background: var(--bg-secondary);
}
.tab-button.active {
background: var(--accent-color);
color: white;
border-color: var(--accent-color);
}
/* Tab Content */
.tab-content {
display: none;
}
.tab-content.active {
display: block;
}
/* Panels */
.panel {
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px;
padding: 20px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
.panel h2 {
margin-bottom: 15px;
font-size: 18px;
border-bottom: 2px solid var(--accent-color);
padding-bottom: 8px;
}
/* Main Tab Grid */
.main-grid {
display: grid;
grid-template-columns: 1fr 1fr;
grid-template-rows: auto auto 1fr;
gap: 20px;
min-height: 70vh;
}
.login-panel {
grid-column: 1;
}
.progress-panel {
grid-column: 2;
}
.console-panel {
grid-column: 1;
grid-row: 2 / 4;
}
.channels-panel {
grid-column: 2;
grid-row: 2 / 4;
}
/* Login */
#login-form input {
display: block;
width: 100%;
padding: 10px;
margin-bottom: 10px;
border: 1px solid var(--border-color);
border-radius: 4px;
background: var(--bg-secondary);
color: var(--text-primary);
}
#login-form button, #oauth-confirm {
width: 100%;
padding: 12px;
background: var(--accent-color);
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 16px;
font-weight: bold;
}
#login-form button:hover, #oauth-confirm:hover {
opacity: 0.9;
}
.oauth-code {
font-size: 32px;
font-weight: bold;
text-align: center;
padding: 20px;
background: var(--bg-secondary);
border-radius: 8px;
margin: 15px 0;
letter-spacing: 4px;
}
/* Drop Progress */
.drop-name {
font-weight: bold;
font-size: 16px;
margin-bottom: 5px;
}
.drop-game {
color: var(--text-secondary);
margin-bottom: 15px;
}
.progress-bar {
width: 100%;
height: 30px;
background: var(--progress-bg);
border-radius: 15px;
overflow: hidden;
margin-bottom: 10px;
}
.progress-fill {
height: 100%;
background: linear-gradient(90deg, var(--accent-color), var(--success-color));
transition: width 0.3s ease;
display: flex;
align-items: center;
justify-content: center;
color: white;
font-weight: bold;
}
.progress-text {
text-align: center;
font-size: 14px;
}
.progress-time {
text-align: center;
color: var(--text-secondary);
font-size: 12px;
}
/* Console Output */
.console-output {
height: 400px;
overflow-y: auto;
font-family: 'Courier New', monospace;
font-size: 12px;
background: var(--bg-secondary);
border: 1px solid var(--border-color);
border-radius: 4px;
padding: 10px;
}
.console-output div {
margin-bottom: 2px;
}
/* Channels List */
.channels-list {
height: 400px;
overflow-y: auto;
}
.channel-item {
padding: 12px;
margin-bottom: 8px;
background: var(--bg-secondary);
border: 2px solid var(--border-color);
border-radius: 6px;
cursor: pointer;
transition: all 0.2s;
}
.channel-item:hover {
border-color: var(--accent-color);
}
.channel-item.watching {
border-color: var(--success-color);
background: rgba(145, 70, 255, 0.1);
}
.channel-item.online {
border-left: 4px solid var(--success-color);
}
.channel-item.offline {
opacity: 0.6;
}
.channel-name {
font-weight: bold;
font-size: 14px;
}
.channel-info {
font-size: 12px;
color: var(--text-secondary);
margin-top: 4px;
}
.channel-badge {
display: inline-block;
padding: 2px 6px;
border-radius: 3px;
font-size: 10px;
margin-left: 5px;
}
.channel-badge.drops {
background: var(--success-color);
color: white;
}
.channel-badge.acl {
background: var(--accent-color);
color: white;
}
/* Inventory Grid */
.inventory-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(350px, 1fr));
gap: 20px;
}
.campaign-card {
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px;
overflow: hidden;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
.campaign-header {
padding: 15px;
background: var(--bg-secondary);
}
.campaign-game {
font-weight: bold;
font-size: 16px;
}
.campaign-name {
font-size: 14px;
color: var(--text-secondary);
}
.campaign-status {
padding: 10px 15px;
font-size: 12px;
display: flex;
justify-content: space-between;
}
.campaign-drops {
padding: 15px;
}
.drop-item {
padding: 10px;
margin-bottom: 8px;
background: var(--bg-secondary);
border-radius: 4px;
border-left: 3px solid var(--border-color);
}
.drop-item.claimed {
border-left-color: var(--success-color);
opacity: 0.7;
}
.drop-item.active {
border-left-color: var(--accent-color);
}
/* Settings */
.settings-container {
max-width: 800px;
}
.settings-section {
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px;
padding: 20px;
margin-bottom: 20px;
}
.settings-section h2 {
margin-bottom: 15px;
}
.settings-section label {
display: block;
margin-bottom: 15px;
}
.settings-section input[type="checkbox"] {
margin-right: 8px;
}
.settings-section select,
.settings-section input[type="number"] {
padding: 8px;
border: 1px solid var(--border-color);
border-radius: 4px;
background: var(--bg-secondary);
color: var(--text-primary);
margin-left: 10px;
}
.list-manager {
margin-top: 20px;
}
.game-list {
min-height: 100px;
max-height: 200px;
overflow-y: auto;
border: 1px solid var(--border-color);
border-radius: 4px;
padding: 10px;
margin-bottom: 10px;
background: var(--bg-secondary);
}
.game-item {
padding: 8px;
margin-bottom: 5px;
background: var(--bg-panel);
border-radius: 4px;
display: flex;
justify-content: space-between;
align-items: center;
}
.game-item button {
padding: 4px 8px;
background: var(--error-color);
color: white;
border: none;
border-radius: 3px;
cursor: pointer;
font-size: 12px;
}
.action-button {
padding: 12px 24px;
margin-right: 10px;
background: var(--bg-secondary);
border: 1px solid var(--border-color);
border-radius: 4px;
cursor: pointer;
font-size: 14px;
color: var(--text-primary);
}
.action-button.primary {
background: var(--accent-color);
color: white;
border-color: var(--accent-color);
}
.action-button:hover {
opacity: 0.9;
}
/* Help */
.help-content {
max-width: 800px;
background: var(--bg-panel);
border: 1px solid var(--border-color);
border-radius: 8px;
padding: 30px;
}
.help-content h2 {
color: var(--accent-color);
margin-bottom: 15px;
}
.help-content h3 {
margin-top: 20px;
margin-bottom: 10px;
}
.help-content ul, .help-content ol {
margin-left: 20px;
margin-bottom: 15px;
}
.help-content li {
margin-bottom: 8px;
}
.help-content a {
color: var(--accent-color);
text-decoration: none;
}
.help-content a:hover {
text-decoration: underline;
}
.help-links {
margin-top: 30px;
padding-top: 20px;
border-top: 1px solid var(--border-color);
}
.help-links a {
display: inline-block;
padding: 10px 20px;
background: var(--accent-color);
color: white;
border-radius: 4px;
margin-right: 10px;
}
/* Games to Watch Styles */
.help-text {
color: var(--text-secondary);
font-size: 14px;
margin-bottom: 15px;
}
.games-filter {
margin-bottom: 20px;
}
.games-filter input {
width: 100%;
padding: 10px;
border: 1px solid var(--border-color);
border-radius: 4px;
background: var(--bg-secondary);
color: var(--text-primary);
font-size: 14px;
margin-bottom: 10px;
}
.filter-actions {
display: flex;
gap: 10px;
}
.small-btn {
padding: 8px 16px;
background: var(--bg-secondary);
border: 1px solid var(--border-color);
border-radius: 4px;
cursor: pointer;
font-size: 13px;
color: var(--text-primary);
transition: all 0.2s;
}
.small-btn:hover {
background: var(--accent-color);
color: white;
border-color: var(--accent-color);
}
.games-container {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 20px;
margin-top: 20px;
}
.selected-games, .available-games {
border: 1px solid var(--border-color);
border-radius: 8px;
padding: 15px;
background: var(--bg-secondary);
}
.selected-games h3, .available-games h3 {
font-size: 14px;
margin-bottom: 15px;
color: var(--text-secondary);
}
/* Sortable list for selected games */
.sortable-list {
min-height: 200px;
max-height: 400px;
overflow-y: auto;
}
.sortable-item {
display: flex;
align-items: center;
gap: 10px;
padding: 10px;
margin-bottom: 8px;
background: var(--bg-panel);
border: 2px solid var(--border-color);
border-radius: 6px;
cursor: move;
transition: all 0.2s;
user-select: none;
}
.sortable-item:hover {
border-color: var(--accent-color);
background: rgba(145, 70, 255, 0.1);
}
.sortable-item.dragging {
opacity: 0.5;
transform: scale(0.95);
}
.drag-handle {
font-size: 18px;
color: var(--text-secondary);
cursor: grab;
}
.drag-handle:active {
cursor: grabbing;
}
.priority-number {
display: inline-flex;
align-items: center;
justify-content: center;
width: 24px;
height: 24px;
background: var(--accent-color);
color: white;
border-radius: 50%;
font-size: 12px;
font-weight: bold;
}
.game-name {
flex: 1;
font-weight: 500;
}
.remove-btn {
padding: 4px 8px;
background: var(--error-color);
color: white;
border: none;
border-radius: 3px;
cursor: pointer;
font-size: 14px;
font-weight: bold;
opacity: 0.8;
transition: opacity 0.2s;
}
.remove-btn:hover {
opacity: 1;
}
/* Checkbox list for available games */
.checkbox-list {
max-height: 400px;
overflow-y: auto;
}
.game-checkbox {
display: flex;
align-items: center;
padding: 8px;
margin-bottom: 5px;
background: var(--bg-panel);
border-radius: 4px;
cursor: pointer;
transition: background 0.2s;
}
.game-checkbox:hover {
background: rgba(145, 70, 255, 0.1);
}
.game-checkbox input[type="checkbox"] {
margin-right: 10px;
cursor: pointer;
}
.game-checkbox span {
user-select: none;
}
/* Utilities */
.empty-message {
text-align: center;
color: var(--text-secondary);
padding: 40px;
}
/* Responsive */
@media (max-width: 1024px) {
.main-grid {
grid-template-columns: 1fr;
grid-template-rows: auto;
}
.login-panel, .progress-panel, .console-panel, .channels-panel {
grid-column: 1;
grid-row: auto;
}
.inventory-grid {
grid-template-columns: 1fr;
}
}