Compare commits

...

10 commits

Author SHA1 Message Date
ec44981a9d Add NVIDIA support for FFmpeg in Docker and enhance chat rendering functionality
All checks were successful
Publish Twitch Archive Container / publish (push) Successful in 7m36s
- Introduced a new docker-compose.nvidia.yml for NVIDIA GPU support.
- Updated dockerstart.bat to allow optional NVIDIA runtime.
- Enhanced ContentDownloader to manage chat rendering status and font settings.
- Improved hardware acceleration detection in utils.py.
- Added tests for hardware acceleration and chat rendering behavior.

Co-authored-by: Copilot <copilot@github.com>
2026-04-25 12:28:59 +02:00
f97e0200d6 Refactor downloader and file manager for improved rclone integration and add healthcheck and smoke test options
- Renamed download flags in ContentDownloader for clarity.
- Enhanced FileManager with methods to build upload paths and verify existing files for rclone uploads.
- Updated StreamProcessor to return success status for stream processing.
- Added rclone smoke test and healthcheck functions to validate configuration and tool availability.
- Improved environment variable handling with a utility function.
- Updated TwitchArchive to incorporate new rclone verification and processing logic.
- Added unit tests for new functionality and refactored existing tests for clarity and coverage.

Co-authored-by: Copilot <copilot@github.com>
2026-04-25 11:54:03 +02:00
e92f36474a Added git lfs 2026-02-22 23:12:11 +01:00
e5e60999bf Refactor global configuration page and navigation; add media library page; enhance streamer configuration with detailed options
- Removed the global configuration form and redirected to the consolidated settings page.
- Updated the dashboard to provide feedback when no streamers are configured and added edit links for each streamer.
- Introduced a new media library page to display media files from the configured archive root.
- Enhanced the streamer configuration page with additional options for overrides and settings, including a confirmation modal for deletion.
- Updated the layout and styles for improved user experience and navigation.
- Switched from file-based password storage to database-backed user credentials management in AuthService.
- Applied EF migrations on application startup to ensure database schema is up-to-date.
2026-02-22 23:06:40 +01:00
1ecf7501f4 Implemented features 2026-02-22 23:06:18 +01:00
4f488bae45 Refactor code structure for improved readability and maintainability 2026-02-21 10:40:12 +01:00
b47641feaa feat: enhance chat downloading with stream monitoring and improved file paths 2026-02-18 18:11:53 +01:00
22a1f5b600 feat: add standalone chat downloader script and batch file for testing 2026-02-15 09:38:58 +01:00
0d3cdfd12c feat: add upload options for pre-merge, merged, and standalone chat videos
- Updated global schema to include options for uploading original videos before merging, merged videos, and standalone chat videos.
- Modified constants to set default values for new upload options.
- Enhanced FileManager to handle new upload options, including conditional file uploads and deletions based on user configuration.
- Introduced unit tests for command-line argument parsing, configuration loading, and merging logic, ensuring robust handling of new features.
- Added tests for filtering logic, default configurations, and enabled streamer handling.
2026-02-11 17:44:34 +01:00
38d51636af Chat monitors stream to end 2026-02-11 13:23:14 +01:00
109 changed files with 6913 additions and 129 deletions

18
.dockerignore Normal file
View file

@ -0,0 +1,18 @@
.git
.github
.forgejo
.vs
.vscode
venv*/
__pycache__/
*.pyc
archive/
dotnet/
tests/
.pytest_cache/
bin/temp/
bin/ffmpeg
bin/ffmpeg.exe
bin/ffprobe
bin/TwitchDownloaderCLI
bin/TwitchDownloaderCLI.exe

16
.env.development Normal file
View file

@ -0,0 +1,16 @@
TWITCH_ARCHIVE_DEV_IMAGE=twitch-archive-local
TWITCH_ARCHIVE_CONTAINER_NAME=twitch-archive-dev
TWITCH_ARCHIVE_APP_ENV_FILE=./.env.development
TWITCH_ARCHIVE_ARCHIVE_BIND=./archive
TWITCH_ARCHIVE_CONFIG_BIND=./config
TWITCH_ARCHIVE_ARGS=-u vinesauce --verbose
TWITCH_ARCHIVE_HEALTHCHECK_STREAMER=vinesauce
TWITCH_ARCHIVE_RCLONE_CONFIG=/app/config/rclone.conf
PYTHONUNBUFFERED=1
TZ=UTC
CLIENT-ID=vdyevjvllziylzwsm3y925p79pwtua
CLIENT-SECRET=y906xadsmf22q54suuzsmfnfav3jc7
OAUTH-PRIVATE-TOKEN=ll4kvlmxuajfgi9lgi5d8mkeglsyvm
SENDER=
RECEIVER=
PASSWD=

16
.env.production Normal file
View file

@ -0,0 +1,16 @@
TWITCH_ARCHIVE_IMAGE=forgejo.maddoscientisto.net/maddo/twitch-archive:latest
TWITCH_ARCHIVE_CONTAINER_NAME=twitch-archive
TWITCH_ARCHIVE_APP_ENV_FILE=./.env.production
TWITCH_ARCHIVE_ARCHIVE_BIND=./archive
TWITCH_ARCHIVE_CONFIG_BIND=./config
TWITCH_ARCHIVE_ARGS=-u vinesauce
TWITCH_ARCHIVE_HEALTHCHECK_STREAMER=vinesauce
TWITCH_ARCHIVE_RCLONE_CONFIG=/app/config/rclone.conf
PYTHONUNBUFFERED=1
TZ=UTC
CLIENT-ID=vdyevjvllziylzwsm3y925p79pwtua
CLIENT-SECRET=y906xadsmf22q54suuzsmfnfav3jc7
OAUTH-PRIVATE-TOKEN=ll4kvlmxuajfgi9lgi5d8mkeglsyvm
SENDER=
RECEIVER=
PASSWD=

View file

@ -0,0 +1,127 @@
name: Publish Twitch Archive Container
on:
push:
branches:
- master
- main
paths:
- docker/python.Dockerfile
- docker/entrypoint.sh
- docker-compose.yml
- docker-compose.override.yml
- requirements.txt
- twitch-archive.py
- run_chat_only.py
- modules/**
- .forgejo/workflows/publish-python-container.yml
workflow_dispatch:
env:
REGISTRY: ${{ vars.FORGEJO_REGISTRY }}
IMAGE_NAMESPACE: ${{ vars.IMAGE_NAMESPACE }}
IMAGE_NAME: ${{ vars.IMAGE_NAME != '' && vars.IMAGE_NAME || 'twitch-archive' }}
BUILD_CONTEXT: .
DOCKERFILE_PATH: docker/python.Dockerfile
jobs:
publish:
runs-on: docker
env:
DOCKER_HOST: ${{ vars.DOCKER_HOST != '' && vars.DOCKER_HOST || 'tcp://172.17.0.1:2375' }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Validate workflow variables
run: |
set -eu
if [ -z "${REGISTRY}" ]; then echo "vars.FORGEJO_REGISTRY is required"; exit 1; fi
if [ -z "${IMAGE_NAMESPACE}" ]; then echo "vars.IMAGE_NAMESPACE is required"; exit 1; fi
if [ ! -f "${DOCKERFILE_PATH}" ]; then echo "Dockerfile not found at ${DOCKERFILE_PATH}"; exit 1; fi
if [ ! -f "requirements.txt" ]; then echo "requirements.txt is missing"; exit 1; fi
- name: Ensure Docker CLI exists
run: |
set -eu
if command -v docker >/dev/null 2>&1; then
docker --version
exit 0
fi
ARCH="$(uname -m)"
case "${ARCH}" in
x86_64) DOCKER_ARCH="x86_64" ;;
aarch64|arm64) DOCKER_ARCH="aarch64" ;;
*) echo "Unsupported architecture for Docker CLI bootstrap: ${ARCH}"; exit 1 ;;
esac
DOCKER_CLI_VERSION="27.5.1"
curl -fsSL "https://download.docker.com/linux/static/stable/${DOCKER_ARCH}/docker-${DOCKER_CLI_VERSION}.tgz" -o docker.tgz
tar -xzf docker.tgz
mkdir -p "${HOME}/.local/bin"
mv docker/docker "${HOME}/.local/bin/docker"
chmod +x "${HOME}/.local/bin/docker"
echo "${HOME}/.local/bin" >> "${FORGEJO_PATH}"
"${HOME}/.local/bin/docker" --version
- name: Ensure Docker Buildx exists
run: |
set -eu
if docker buildx version >/dev/null 2>&1; then
docker buildx version
exit 0
fi
ARCH="$(uname -m)"
case "${ARCH}" in
x86_64) BUILDX_ARCH="amd64" ;;
aarch64|arm64) BUILDX_ARCH="arm64" ;;
*) echo "Unsupported architecture for Docker Buildx bootstrap: ${ARCH}"; exit 1 ;;
esac
BUILDX_VERSION="v0.21.1"
mkdir -p "${HOME}/.docker/cli-plugins"
curl -fsSL "https://github.com/docker/buildx/releases/download/${BUILDX_VERSION}/buildx-${BUILDX_VERSION}.linux-${BUILDX_ARCH}" -o "${HOME}/.docker/cli-plugins/docker-buildx"
chmod +x "${HOME}/.docker/cli-plugins/docker-buildx"
docker buildx version
- name: Check Docker daemon connectivity
run: |
set -eu
echo "Using DOCKER_HOST=${DOCKER_HOST}"
docker version
docker info >/dev/null
- name: Create Buildx builder
run: |
set -eu
docker buildx rm forgejo-builder >/dev/null 2>&1 || true
docker buildx create --name forgejo-builder --driver docker-container --use
docker buildx inspect --bootstrap
- name: Validate registry secrets
run: |
set -eu
if [ -z "${{ secrets.FORGEJO_REGISTRY_USERNAME }}" ]; then echo "secrets.FORGEJO_REGISTRY_USERNAME is required"; exit 1; fi
if [ -z "${{ secrets.FORGEJO_REGISTRY_TOKEN }}" ]; then echo "secrets.FORGEJO_REGISTRY_TOKEN is required"; exit 1; fi
- name: Login to registry
run: |
set -eu
echo "${{ secrets.FORGEJO_REGISTRY_TOKEN }}" | docker login "${REGISTRY}" -u "${{ secrets.FORGEJO_REGISTRY_USERNAME }}" --password-stdin
- name: Build and push image
run: |
set -eu
IMAGE_REF="${REGISTRY}/${IMAGE_NAMESPACE}/${IMAGE_NAME}"
SHA_TAG="${IMAGE_REF}:sha-${FORGEJO_SHA}"
BRANCH_TAG="${IMAGE_REF}:${FORGEJO_REF_NAME}"
docker buildx build \
--file "${DOCKERFILE_PATH}" \
--tag "${SHA_TAG}" \
--tag "${BRANCH_TAG}" \
--tag "${IMAGE_REF}:latest" \
--push \
"${BUILD_CONTEXT}"

18
.gitattributes vendored
View file

@ -1,6 +1,22 @@
# Git LFS tracking for large media and binaries
# Binaries in bin/
bin/* filter=lfs diff=lfs merge=lfs -text
bin/TwitchDownloaderCLI* filter=lfs diff=lfs merge=lfs -text
bin/ffmpeg filter=lfs diff=lfs merge=lfs -text
# Video files under archive (raw/ and common video extensions)
archive/**/video/** filter=lfs diff=lfs merge=lfs -text
archive/**/video/raw/** filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
*.mkv filter=lfs diff=lfs merge=lfs -text
*.mov filter=lfs diff=lfs merge=lfs -text
# Large archives and executables
*.zip filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.exe filter=lfs diff=lfs merge=lfs -text
*.dll filter=lfs diff=lfs merge=lfs -text
# Auto detect text files and perform LF normalization # Auto detect text files and perform LF normalization
* text=auto * text=auto
bin/ffmpeg filter=lfs diff=lfs merge=lfs -text
bin/ffmpeg.exe filter=lfs diff=lfs merge=lfs -text bin/ffmpeg.exe filter=lfs diff=lfs merge=lfs -text
bin/TwitchDownloaderCLI filter=lfs diff=lfs merge=lfs -text bin/TwitchDownloaderCLI filter=lfs diff=lfs merge=lfs -text
bin/TwitchDownloaderCLI.exe filter=lfs diff=lfs merge=lfs -text bin/TwitchDownloaderCLI.exe filter=lfs diff=lfs merge=lfs -text

View file

@ -0,0 +1,77 @@
---
description: 'Blazor component and application patterns'
applyTo: '**/*.razor, **/*.razor.cs, **/*.razor.css'
---
## Blazor Code Style and Structure
- Write idiomatic and efficient Blazor and C# code.
- Follow .NET and Blazor conventions.
- Use Razor Components appropriately for component-based UI development.
- Prefer inline functions for smaller components but separate complex logic into code-behind or service classes.
- Async/await should be used where applicable to ensure non-blocking UI operations.
## Naming Conventions
- Follow PascalCase for component names, method names, and public members.
- Use camelCase for private fields and local variables.
- Prefix interface names with "I" (e.g., IUserService).
## Blazor and .NET Specific Guidelines
- Utilize Blazor's built-in features for component lifecycle (e.g., OnInitializedAsync, OnParametersSetAsync).
- Use data binding effectively with @bind.
- Leverage Dependency Injection for services in Blazor.
- Structure Blazor components and services following Separation of Concerns.
- Always use the latest version C#, currently C# 13 features like record types, pattern matching, and global usings.
## Error Handling and Validation
- Implement proper error handling for Blazor pages and API calls.
- Use logging for error tracking in the backend and consider capturing UI-level errors in Blazor with tools like ErrorBoundary.
- Implement validation using FluentValidation or DataAnnotations in forms.
## Blazor API and Performance Optimization
- Utilize Blazor server-side or WebAssembly optimally based on the project requirements.
- Use asynchronous methods (async/await) for API calls or UI actions that could block the main thread.
- Optimize Razor components by reducing unnecessary renders and using StateHasChanged() efficiently.
- Minimize the component render tree by avoiding re-renders unless necessary, using ShouldRender() where appropriate.
- Use EventCallbacks for handling user interactions efficiently, passing only minimal data when triggering events.
## Caching Strategies
- Implement in-memory caching for frequently used data, especially for Blazor Server apps. Use IMemoryCache for lightweight caching solutions.
- For Blazor WebAssembly, utilize localStorage or sessionStorage to cache application state between user sessions.
- Consider Distributed Cache strategies (like Redis or SQL Server Cache) for larger applications that need shared state across multiple users or clients.
- Cache API calls by storing responses to avoid redundant calls when data is unlikely to change, thus improving the user experience.
## State Management Libraries
- Use Blazor's built-in Cascading Parameters and EventCallbacks for basic state sharing across components.
- Implement advanced state management solutions using libraries like Fluxor or BlazorState when the application grows in complexity.
- For client-side state persistence in Blazor WebAssembly, consider using Blazored.LocalStorage or Blazored.SessionStorage to maintain state between page reloads.
- For server-side Blazor, use Scoped Services and the StateContainer pattern to manage state within user sessions while minimizing re-renders.
## API Design and Integration
- Use HttpClient or other appropriate services to communicate with external APIs or your own backend.
- Implement error handling for API calls using try-catch and provide proper user feedback in the UI.
## Testing and Debugging in Visual Studio
- All unit testing and integration testing should be done in Visual Studio.
- Test Blazor components and services using xUnit, NUnit, or MSTest.
- Use Moq or NSubstitute for mocking dependencies during tests.
- Debug Blazor UI issues using browser developer tools and Visual Studio's debugging tools for backend and server-side issues.
- For performance profiling and optimization, rely on Visual Studio's diagnostics tools.
## Security and Authentication
- Implement Authentication and Authorization in the Blazor app where necessary using ASP.NET Identity or JWT tokens for API authentication.
- Use HTTPS for all web communication and ensure proper CORS policies are implemented.
## API Documentation and Swagger
- Use Swagger/OpenAPI for API documentation for your backend API services.
- Ensure XML documentation for models and API methods for enhancing Swagger documentation.

47
.gitignore vendored
View file

@ -4,6 +4,8 @@ config/global.json
# Streamer-specific configurations (personal settings) # Streamer-specific configurations (personal settings)
config/streamers/*.json config/streamers/*.json
config/rclone.conf
config/rclone.conf.*
# Python cache # Python cache
__pycache__/ __pycache__/
@ -22,3 +24,48 @@ venv3/**
bin/** bin/**
\n+# Ignore any virtual environment directories starting with 'venv' (venv, venv3, venv314, etc.) \n+# Ignore any virtual environment directories starting with 'venv' (venv, venv3, venv314, etc.)
venv*/ venv*/
.vs/ProjectEvaluation/twitch-archive-2.metadata.v10.bin
.vs/ProjectEvaluation/twitch-archive-2.projects.v10.bin
.vs/ProjectEvaluation/twitch-archive-2.strings.v10.bin
.vs/Twitch-Archive-2/CopilotIndices/18.3.508.13148/CodeChunks.db
.vs/Twitch-Archive-2/CopilotIndices/18.3.508.13148/SemanticSymbols.db
.vs/Twitch-Archive-2/DesignTimeBuild/.dtbcache.v2
.vs/Twitch-Archive-2/FileContentIndex/843065c8-d80f-4907-b0ae-6d010b3a5699.vsidx
.vs/Twitch-Archive-2/FileContentIndex/ef7e1a3c-80cd-4867-a9a8-2e5099471227.vsidx
.vs/Twitch-Archive-2/v18/.futdcache.v2
.vs/Twitch-Archive-2/v18/.suo
.vs/Twitch-Archive-2/v18/DocumentLayout.backup.json
.vs/Twitch-Archive-2/v18/DocumentLayout.json
.vscode/settings.json
# C# / Visual Studio
# Build Folders
bin/
obj/
# Visual Studio files
*.user
*.suo
*.userprefs
*.csproj.user
*.pidb
*.pdb
*.cache
*.ilk
*.log
*.vspscc
*.vssscc
# Test results and packages
TestResults/
packages/
*.nupkg
# Database and backup
*.dbmdl
*.bak
*.backup
*.orig
dotnet/.vs/**
.vs/**

101
README.md
View file

@ -1,8 +1,109 @@
# Twitch Archive # Twitch Archive
Inspired by https://github.com/EnterGin/Auto-Stream-Recording-Twitch Inspired by https://github.com/EnterGin/Auto-Stream-Recording-Twitch
## Git LFS
This repository stores large media files (recorded video and some binaries). Use Git LFS to manage large objects.
Quick setup (Windows):
1. Install Git LFS: `git lfs install`
2. Ensure `.gitattributes` is committed (this repo includes one).
3. If you already have large files tracked by normal Git, migrate them:
```powershell
git lfs install
git lfs track "*.mp4" "*.mkv" "bin/*"
git add .gitattributes
git add -A
git commit -m "Migrate large files to LFS"
git push origin main
```
Notes:
- Git LFS needs server-side support. If using GitHub, enable Git LFS on the remote and ensure you have sufficient bandwidth/storage quota.
- You can customize tracked patterns in `.gitattributes`.
Python script to check, download live stream, VOD, chat and upload them to any cloud storage supported by rclone. Python script to check, download live stream, VOD, chat and upload them to any cloud storage supported by rclone.
## Docker
This repository now includes a Python-only container setup for the archiver. The dotnet subapp is not part of this container flow.
Files:
- `docker/python.Dockerfile`: production image for the Python archiver
- `docker-compose.yml`: deployment-oriented compose file
- `docker-compose.override.yml`: local development and testing override
- `.env.production`: production container and app environment template
- `.env.development`: development container and app environment template
- `dockerstart.bat`: Windows helper to run the container like the old batch launcher
### Container layout
- Mount your external archive folder to `/app/archive`
- Mount your external config folder to `/app/config`
- Put your `rclone.conf` file at `/app/config/rclone.conf` on the mounted host path
- The container exports `RCLONE_CONFIG=/app/config/rclone.conf`, so rclone will use that file automatically
### Production deployment
1. Edit `.env.production` with your image name, Twitch credentials, bind paths, and default arguments.
2. Place your streamer JSON files and `rclone.conf` in the mounted config folder.
3. Start the container:
```powershell
docker compose --env-file .env.production up -d
```
4. Follow logs:
```powershell
docker compose --env-file .env.production logs -f twitch-archive
```
### Development and local testing
The override compose file builds the image locally and mounts the repository for faster iteration.
Start it with:
```powershell
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml up --build
```
Run a one-off manual test for another streamer:
```powershell
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml run --rm twitch-archive python twitch-archive.py -u hackerling --verbose
```
Or use the Windows helper:
```powershell
.\dockerstart.bat vinesauce --verbose
```
That batch launcher mirrors the old pattern and expands to a compose `run` command, so you can test any streamer manually.
If the host has the NVIDIA Container Toolkit installed and you want FFmpeg/NVENC inside the container, use the optional NVIDIA override:
```powershell
.\dockerrebuild.bat
.\dockerstart.bat --nvidia vinesauce --verbose
```
The image built by `.\dockerrebuild.bat` already includes the NVIDIA-capable FFmpeg/container toolchain. The optional [docker-compose.nvidia.yml](docker-compose.nvidia.yml) layer is only for runtime GPU passthrough: it requests `gpus: all` and sets `NVIDIA_VISIBLE_DEVICES` plus `NVIDIA_DRIVER_CAPABILITIES=compute,utility,video` for the container.
On systems without NVIDIA support, keep using the normal command without `--nvidia`; the image still builds the same way, it just runs without GPU passthrough.
### Healthcheck and smoke tests
- Container healthcheck command: `python twitch-archive.py --healthcheck -u vinesauce`
- Rclone smoke test command: `python twitch-archive.py -u vinesauce --rclone-smoke-test`
The healthcheck verifies config loading plus `streamlink`, `ffmpeg`, `TwitchDownloaderCLI`, and `rclone` availability. The smoke test writes a tiny file, uploads it with the configured rclone remote, and prints the live rclone output into the container logs.
## ⚡ FFmpeg 8.0 Enhanced ## ⚡ FFmpeg 8.0 Enhanced
Now with FFmpeg 8.0+ support featuring hardware acceleration and performance improvements! Now with FFmpeg 8.0+ support featuring hardware acceleration and performance improvements!
- **5-10x faster encoding** with NVIDIA, Intel, or AMD GPUs - **5-10x faster encoding** with NVIDIA, Intel, or AMD GPUs

214
UpgradePlan.md Normal file
View file

@ -0,0 +1,214 @@
Plan: C# .NET 10 Twitch Archive Rewrite
A complete port of the Python archiver to C# .NET 10 with Blazor Server UI, real-time process output, SQLite state tracking, full DI/service pattern, NLog logging, and a resilient recording engine. Placed under dotnet/ in the existing repo.
Project Layout
Step 1 — Solution & Project scaffolding
Create the SLN and three projects:
TwitchArchive.Core — classlib, targets net10.0
TwitchArchive.Web — Blazor Server (blazorserver), targets net10.0
TwitchArchive.Tests — xUnit, targets net10.0
NuGet packages:
Core: Microsoft.EntityFrameworkCore.Sqlite, Polly, NLog, NLog.Extensions.Logging
Web: all Core packages + Microsoft.AspNetCore.SignalR, NLog.Web.AspNetCore
Tests: xunit, Moq, coverlet.collector, Microsoft.EntityFrameworkCore.InMemory
Step 2 — Configuration models
Mirror the existing JSON schemas as C# POCOs with System.Text.Json attributes:
GlobalConfig.cs — one property per key in config/global.json.example
StreamerConfig.cs — all fields nullable (override semantics), only Username and Enabled required
EffectiveConfig.cs — computed merge of global + per-streamer; exposes resolved values
AppSettings.cs — app-level settings (password hash, tool paths, .env secrets)
IConfigurationService / ConfigurationService:
LoadGlobal() / SaveGlobal(GlobalConfig)
LoadStreamer(string username) / SaveStreamer(StreamerConfig)
GetAllStreamers(), GetEffectiveConfig(string username) (merge logic)
Reads/writes global.json and config/streamers/*.json — same files as Python
Step 3 — Infrastructure layer
TwitchApiClient (injectable, mockable):
GetOAuthTokenAsync() — POST to https://id.twitch.tv/oauth2/token, caches token, refreshes on 401
CheckStreamStatusAsync(string username) — GQL query for live stream + archiveVideo.id
GetLatestVodAsync(string username) — GQL query for most recent VOD
ValidateUsernameAsync(string username) — Helix /users endpoint
Credentials read from environment (CLIENT-ID, CLIENT-SECRET, OAUTH-PRIVATE-TOKEN)
All methods return typed result objects, never throw on network errors — return Result<T> (or OneOf)
HttpResiliencePolicy (Polly):
Wraps HttpClient for TwitchApiClient
WaitAndRetryForever with exponential backoff starting at 15 s, doubling, capped at 10 minutes
Only applies to transient errors (5xx, timeout, HttpRequestException) — not 401/404
Logged via NLog on each retry attempt
ProcessRunner (injectable + mockable for tests):
RunAsync(ProcessRunOptions options, CancellationToken ct) → int exitCode
StartAsync(ProcessRunOptions options, CancellationToken ct) → IRunningProcess handle (for long-lived processes like streamlink)
Reads stdout and stderr line by line asynchronously
Reports each line to IProcessOutputStore (streamer + job context)
Forwards to NLog
ProcessRunOptions: FileName, Arguments, WorkingDirectory, RedirectOutput
Step 4 — Core services (all behind interfaces)
IStreamMonitorService / StreamMonitorService
Wraps TwitchApiClient
CheckIsLiveAsync(string username) → LiveStreamInfo?
GetLatestVodAsync(string username) → VodInfo?
IRecorderService / RecorderService
StartRecordingAsync(string username, string quality, string outputPath, CancellationToken ct) → Task<RecordingResult>
Invokes streamlink via ProcessRunner
Passes --hls-live-restart, --stream-segment-threads, optional OAuth header
Returns when streamlink exits (either stream ended or ct cancelled)
IProcessorService / ProcessorService
ProcessRawStreamAsync(string rawPath, string outputPath, EffectiveConfig cfg, CancellationToken ct)
Builds ffmpeg args: hwaccel, thread count, error recovery flags, faststart, copy codecs
MergeVideoChatAsync(string videoPath, string chatVideoPath, string outputPath, string layout, CancellationToken ct)
IDownloaderService / DownloaderService
DownloadVodAsync(VodInfo vod, string outputPath, EffectiveConfig cfg, CancellationToken ct) → bool
Invokes TwitchDownloaderCLI videodownload
Chat download methods stubbed with NotImplementedException / commented structure; interface is defined now to keep the architecture clean
IUploadService / UploadService
UploadAsync(string localRoot, IEnumerable<string> relativeFilePaths, string rcloneDest, CancellationToken ct) → bool
Writes a temp files-from list, invokes rclone copy --files-from
Returns success/failure; preserves local files on failure
IFileManagerService / FileManagerService
InitializeDirectories(string rootPath, string username)
GetPaths(string rootPath, string username, string filenameBase) → ArchivePaths record (all expected paths)
CleanRawFile(string path, bool cleanRaw)
DeleteLocalFiles(ArchivePaths paths, EffectiveConfig cfg)
GetUniquePath(string path) → adds numeric suffix if file exists
Step 5 — Recording resilience engine
RecoveryPolicy (POCO, unit-testable, no DI deps):
Encodes a state machine with these states:
State Meaning
Monitoring Normal polling at refresh interval
Recording streamlink subprocess active
FastReconnect Stream ended; checking every 10 s for up to 2 minutes
SlowReconnect Still not live after 2 min; checking every 60 s concurrently with post-processing
PostProcessing Confirmed ended; ffmpeg / VOD download / upload running
NetworkFault Twitch API unreachable; exponential back-off (30 s → capped at 10 min)
Transitions:
Recording → streamlink exits → enter FastReconnect, record phase start time
FastReconnect → live confirmed → start new Recording (new filename/segment)
FastReconnect (2 min elapsed) → enter SlowReconnect + kick off PostProcessing concurrently
SlowReconnect → live confirmed → start new Recording
SlowReconnect / Monitoring → API call throws network error → enter NetworkFault
NetworkFault → successful API response → return to previous state (Monitoring or re-enter FastReconnect if we were mid-reconnect)
NetworkFault backoff: 30s * 2^attempt, capped at 600s
RecoveryPolicy is a pure class with a Tick(DateTime now, bool? isLive, bool networkError) method → returns RecoveryDecision (what to do next + sleep duration). Fully unit-testable with no async or DI.
StreamWorker : BackgroundService
One instance per enabled streamer
Holds RecoveryPolicy instance
Main loop: evaluate policy decision → execute the corresponding service call → loop
Started/stopped by StreamWorkerManager
Writes job records to SQLite on start/complete/fail
StreamWorkerManager
StartWorker(string username), StopWorker(string username), RestartWorker(string username)
Called at app startup for all enabled streamers
Called from Web UI on enable/disable/config change
Workers stored in ConcurrentDictionary<string, (StreamWorker, CancellationTokenSource)>
Step 6 — Persistence (SQLite + EF Core)
ArchiveDbContext with three tables:
StreamSessions: Id, StreamerUsername, TwitchStreamId, Title, StartedAt, EndedAt, Status (Recording/Processing/Uploading/Complete/Failed)
ArchiveJobs: Id, SessionId, JobType (enum: RecordLive, ProcessLive, DownloadVod, ProcessVod, UploadCloud, DeleteLocal), Status, StartedAt, CompletedAt, FilePath, ErrorMessage
StreamerStates: Username, IsMonitoring, LastCheckedAt, CurrentRecoveryState
Migrations via EF Core CLI. ISessionRepository / IJobRepository interfaces for testability with in-memory EF provider in tests.
Step 7 — Process output streaming
IProcessOutputStore:
AppendLine(string streamerId, Guid jobId, string line, bool isError)
GetRecentLines(string streamerId, Guid jobId, int count = 500) → IReadOnlyList<OutputLine>
In-memory circular buffer (1000 lines per job, last 20 jobs per streamer)
ProcessOutputHub : Hub (SignalR):
Clients call SubscribeToStreamer(string username) → join group streamer:{username}
Clients call SubscribeToJob(Guid jobId) → join group job:{jobId}
Server pushes ReceiveLine(OutputLine line) from ProcessRunner via IHubContext<ProcessOutputHub>
On subscribe: server immediately sends buffered lines from IProcessOutputStore
Step 8 — Blazor Server Web UI
Authentication: Cookie-based single-password auth via ASP.NET Core minimal auth middleware. Password stored as BCrypt hash in AppSettings. Login.razor page at /login. Protected routes with [Authorize].
Pages & Components:
Dashboard.razor (/) — grid of all configured streamers showing: username, live/offline badge, current recovery state, last recorded session, quick Start/Stop monitoring toggle
StreamerDetail.razor (/streamer/{username}) — live status, current job pipeline steps (record → process → upload with progress), ProcessOutputConsole.razor showing real-time terminal output via SignalR
ProcessOutputConsole.razor — reusable Blazor component; subscribes to SignalR on mount, renders an auto-scrolling <pre> with colored output (stdout = white, stderr = orange/red), handles reconnect
Sessions.razor (/sessions) — paginated list of past archive sessions with job statuses and expandable per-job output
GlobalConfig.razor (/config/global) — EditForm bound to GlobalConfig model with data annotations validation, Save button calls IConfigurationService.SaveGlobal()
StreamerConfig.razor (/config/{username}) — similar form for per-streamer overrides; each field has a nullable toggle (inherit from global vs override)
AddStreamer.razor (/config/new) — minimal form: username + enabled; creates new config/streamers/{username}.json
AppSettings.razor (/settings) — tool paths (streamlink, ffmpeg, TwitchDownloaderCLI, rclone), change password
Step 9 — NLog configuration
nlog.config (XML): two targets:
Console (colored, with level formatting)
File rolling (logs/archive-${shortdate}.log, keep 30 days)
Log structured context: StreamerUsername, JobId, JobType as NLog ScopeContext properties. Service methods open a scope via ILogger.BeginScope(...).
Step 10 — Docker
Dockerfile (multi-stage):
Build stage: mcr.microsoft.com/dotnet/sdk:10.0
Runtime stage: mcr.microsoft.com/dotnet/aspnet:10.0 (Linux)
Install ffmpeg, streamlink (via pip), download TwitchDownloaderCLI binary for linux-x64
rclone installed via shell script or apt
Expose port 8080
ENTRYPOINT ["dotnet", "TwitchArchive.Web.dll"]
docker-compose.yml:
Volume mounts: ./config:/app/config, ./archive:/app/archive, ./logs:/app/logs
Environment variables: CLIENT-ID, CLIENT-SECRET, OAUTH-PRIVATE-TOKEN
Windows dev: run directly with dotnet run; tool paths auto-detected (Windows vs Linux) via RuntimeInformation.IsOSPlatform(OSPlatform.Windows) in ToolPathResolver
Step 11 — Unit tests
TwitchArchive.Tests covers:
RecoveryPolicyTests — state machine transitions, timing phases, network fault backoff; pure synchronous tests
ConfigurationServiceTests — JSON load/save/merge with temp files
TwitchApiClientTests — mocked HttpMessageHandler; OAuth, GQL queries, 401 refresh, network errors
FileManagerServiceTests — path generation, directory creation with temp directories
DownloaderServiceTests / RecorderServiceTests — mocked ProcessRunner; verify correct CLI arguments
UploadServiceTests — mocked ProcessRunner; verify rclone argument construction
SessionRepositoryTests — EF Core in-memory provider
EffectiveConfigTests — global + streamer override merge logic
Verification
dotnet build dotnet/TwitchArchive.sln — zero warnings, zero errors
dotnet test dotnet/TwitchArchive.Tests/ — all tests green
docker compose up --build in the dotnet/ folder → app reachable at http://localhost:8080
Manual: add a test streamer config, enable monitoring, confirm it polls and records a live stream, runs ffmpeg, uploads via rclone
Resilience manual test: kill network during recording → verify FastReconnect phase kicks in, resumes after connectivity restored
Decisions
Blazor Server chosen for real-time terminal output without a separate API; no WASM needed
streamlink for live + TwitchDownloaderCLI for VOD (same split as Python; streamlink gives better live resilience)
Simple BCrypt password auth (not full Identity — this is a single-user tool)
RecoveryPolicy as a pure POCO state machine keeps the resilience logic fully unit-testable without async/mocking
Polly WaitAndRetryForever on HttpClient handles persistent network failure independently of the application-level state machine; they are complementary — Polly handles individual HTTP call retries, RecoveryPolicy handles the overall workflow state
Chat download service interface is defined in Step 4 but methods are stubbed — adding implementation later requires only filling in DownloaderService without touching any other layer
Config files remain in the same format/location so the Python and C# versions can share the same config directory

78
UpgradePlan2.md Normal file
View file

@ -0,0 +1,78 @@
<div class="page"> <nav class="sidebar"><NavMenu /></nav> <main class="main">@Body</main> </div> ``` Add a top bar (`<header class="topbar">`) with the app title "Twitch Archive" and a hamburger toggle `<button>` that toggles a `sidebarCollapsed` bool field to add/remove a CSS class.
Create (or extend) dotnet/src/TwitchArchive.Web/wwwroot/css/app.css:
.page: display:flex; height:100vh
.sidebar: width:220px; flex-shrink:0; background:#1e1e2e; color:#cdd6f4; overflow-y:auto
.main: flex:1; overflow-y:auto; padding:1.5rem
.nav-link: display:block; padding:0.6rem 1rem; color:#cdd6f4; text-decoration:none
.nav-link.active: background:#313244; border-left:3px solid #89b4fa
Media query @media(max-width:768px): .sidebar.collapsed { display:none }, .topbar { display:flex }, else .topbar { display:none }
Step G — Full Blazor UI pages (Plan Step 8)
Goal: implement the missing pages referenced by the NavMenu.
Steps
Dashboard.razor (/) — replace dotnet/src/TwitchArchive.Web/Pages/Index.razor. Display a CSS grid of streamer cards, each showing: username (link to /streamer/{username}), live/offline <span> badge, current RecoveryState text from WorkerManager.GetState(username), last session start from ISessionRepository, Start/Stop buttons. Poll every 10 s via PeriodicTimer in OnInitializedAsync, disposed in IAsyncDisposable.DisposeAsync.
StreamerDetail.razor (/streamer/{username}) — new file. Live status badge, pipeline step bar (Record → Process → Upload using CSS flex row), <ProcessConsole Streamer="@Username" />. Route parameter [Parameter] public string Username.
GlobalConfig.razor (/config/global) — new file. <EditForm> bound to a GlobalConfig loaded via IConfigurationService.LoadGlobal(). On valid submit: await ConfigService.SaveGlobal(model), show a dismissible success alert.
StreamerConfig.razor (/config/{username}) — new file. Per-field nullable override: each field has an <InputCheckbox> "Override" toggle; when unchecked the <InputText> is disabled and shows the global default as placeholder. Save calls SaveStreamer. Delete button removes the config file and navigates to /.
AddStreamer.razor (/config/new) — new file. Two fields: Username (required, lowercase) and Enabled checkbox. On submit: await ConfigService.SaveStreamer(new StreamerConfig { Username, Enabled }), then Nav.NavigateTo($"/config/{model.Username}").
AppSettings.razor (/settings) — new file. Tool-path fields bound to AppSettings. Change-password section: current password (validated against BCrypt hash) + new + confirm fields. On save: update appsettings.json and call IAuthService to refresh the cached hash.
Step H — Authentication (Plan Step 8)
Goal: single-password BCrypt cookie auth protecting all pages except /login.
Steps
Add <PackageReference Include="BCrypt.Net-Next" Version="4.*" /> to TwitchArchive.Web.csproj.
Create dotnet/src/TwitchArchive.Web/Services/IAuthService.cs + AuthService.cs — ValidatePassword(string plain) → bool using BCrypt.Net.BCrypt.Verify against AppSettings.PasswordHash. If hash is empty (first-run), any password is accepted.
Create dotnet/src/TwitchArchive.Web/Pages/Login.razor (/login) — password <InputText type="password"> in an <EditForm>. Posts to /auth/login minimal-API endpoint via form navigation.
Add minimal API endpoint POST /auth/login in Program.cs — reads password from form body, calls IAuthService.ValidatePassword, calls HttpContext.SignInAsync(CookieAuthenticationDefaults.AuthenticationScheme, ...), redirects to /. On failure, redirects to /login?error=1.
In Program.cs add builder.Services.AddAuthentication(...).AddCookie(opt => opt.LoginPath = "/login"), app.UseAuthentication(), app.UseAuthorization().
Update App.razor — replace <RouteView> with <AuthorizeRouteView>, wrap in <CascadingAuthenticationState>. Add @attribute [Authorize] to all pages except Login.razor.
Step I — Docker (Plan Step 10)
Steps
Create dotnet/Dockerfile:
Build stage (sdk:10.0): dotnet publish src/TwitchArchive.Web -c Release -o /app/publish
Runtime stage (aspnet:10.0): apt-get install -y ffmpeg python3-pip rclone, pip3 install streamlink, download TwitchDownloaderCLI linux-x64 binary to /app/bin/ and chmod +x.
EXPOSE 8080, ENV ASPNETCORE_URLS=http://+:8080, ENTRYPOINT ["dotnet","TwitchArchive.Web.dll"]
Create dotnet/docker-compose.yml:
Create dotnet/src/TwitchArchive.Core/Config/ToolPathResolver.cs — static helper using RuntimeInformation.IsOSPlatform(OSPlatform.Windows) to resolve default binary paths; used by AppSettings property defaults and RecorderService/ProcessorService.
Step J — Extended unit tests (Plan Step 11)
Steps
ConfigurationServiceTests.cs — load/save/merge in Path.GetTempPath() temp dir; assert roundtrip and merge precedence.
TwitchApiClientTests.cs — mock HttpMessageHandler; token caching, GQL stream-status (live + offline), GetLatestVodAsync, network error → null.
FileManagerServiceTests.cs — InitializeDirectories, GetPaths, GetUniquePath collision suffix on temp dirs.
RecorderServiceTests.cs — mock IProcessRunner capturing ProcessRunOptions.Arguments; assert --hls-live-restart and resolved quality.
UploadServiceTests.cs — mock IProcessRunner; assert rclone arguments contain copy --files-from and correct dest; assert temp list file is cleaned up.
EffectiveConfigTests.cs — all merge-precedence cases: null streamer field → global default; non-null streamer field → override wins.
SessionRepositoryTests.cs — EF in-memory; AddSessionAsync, GetRecentSessionsAsync, UpdateSessionAsync status change.
Verification
Manual checks:
All NavMenu links resolve without 404; active link is highlighted
Login page blocks unauthenticated access; correct password grants access; wrong password shows error
Dashboard renders streamer cards; Start/Stop toggles update Recovery State badge
Global Config saves to global.json; reload confirms persistence
Add Streamer creates config/streamers/{name}.json and redirects to per-streamer config page
Sessions page shows rows after a recording completes with expandable job list
Live Monitor shows real-time ProcessConsole output via SignalR
Decisions
NavMenu uses Blazor's built-in NavLink with pure CSS sidebar — no Bootstrap dependency to keep the bundle small
Config service reads/writes the same config directory as the Python side — both runtimes can share config files without conversion
EnsureCreated → Migrate() — migrations support future schema changes without data loss
BCrypt single-password auth over full Identity — this is a single-user self-hosted tool; Identity adds unnecessary overhead
IRecorderService extracted from StreamWorker — the recording path becomes independently testable without running the full state machine
Or simply paste the content above into a new file UpgradePlan-Part2.md at the workspace root. Once terminal/file-edit tools are re-enabled, I can write it directly.

View file

@ -13,6 +13,9 @@
"mergeChatLayout": "side-by-side", "mergeChatLayout": "side-by-side",
"vodTimeout": 300, "vodTimeout": 300,
"uploadCloud": true, "uploadCloud": true,
"uploadPreMergeVideo": true,
"uploadMergedVideo": true,
"uploadChatVideo": false,
"deleteFiles": false, "deleteFiles": false,
"onlyRaw": false, "onlyRaw": false,
"cleanRaw": true, "cleanRaw": true,

View file

@ -74,6 +74,21 @@
"default": true, "default": true,
"description": "Upload to rclone remote: false = disabled, true = enabled" "description": "Upload to rclone remote: false = disabled, true = enabled"
}, },
"uploadPreMergeVideo": {
"type": "boolean",
"default": true,
"description": "Upload original videos before merging with chat (LIVE and VOD files): false = skip, true = upload"
},
"uploadMergedVideo": {
"type": "boolean",
"default": true,
"description": "Upload merged videos (video + chat combined): false = skip, true = upload"
},
"uploadChatVideo": {
"type": "boolean",
"default": true,
"description": "Upload standalone chat video files: false = skip, true = upload"
},
"deleteFiles": { "deleteFiles": {
"type": "boolean", "type": "boolean",
"default": false, "default": false,

View file

@ -0,0 +1,6 @@
services:
twitch-archive:
gpus: all
environment:
NVIDIA_VISIBLE_DEVICES: ${NVIDIA_VISIBLE_DEVICES:-all}
NVIDIA_DRIVER_CAPABILITIES: ${NVIDIA_DRIVER_CAPABILITIES:-compute,utility,video}

View file

@ -0,0 +1,17 @@
services:
twitch-archive:
build:
context: .
dockerfile: docker/python.Dockerfile
image: ${TWITCH_ARCHIVE_DEV_IMAGE:-twitch-archive-local}
restart: "no"
env_file:
- ${TWITCH_ARCHIVE_APP_ENV_FILE:-./.env.development}
command:
- sh
- -lc
- python twitch-archive.py ${TWITCH_ARCHIVE_ARGS:--u vinesauce --verbose}
volumes:
- .:/app
- ${TWITCH_ARCHIVE_ARCHIVE_BIND:-./archive}:/app/archive
- ${TWITCH_ARCHIVE_CONFIG_BIND:-./config}:/app/config

28
docker-compose.yml Normal file
View file

@ -0,0 +1,28 @@
services:
twitch-archive:
image: ${TWITCH_ARCHIVE_IMAGE:-forgejo.maddoscientisto.net/maddo/twitch-archive:latest}
container_name: ${TWITCH_ARCHIVE_CONTAINER_NAME:-twitch-archive}
restart: unless-stopped
init: true
env_file:
- ${TWITCH_ARCHIVE_APP_ENV_FILE:-./.env.production}
environment:
PYTHONUNBUFFERED: ${PYTHONUNBUFFERED:-1}
TZ: ${TZ:-UTC}
RCLONE_CONFIG: ${TWITCH_ARCHIVE_RCLONE_CONFIG:-/app/config/rclone.conf}
TWITCH_ARCHIVE_HEALTHCHECK_STREAMER: ${TWITCH_ARCHIVE_HEALTHCHECK_STREAMER:-vinesauce}
command:
- sh
- -lc
- python twitch-archive.py ${TWITCH_ARCHIVE_ARGS:--u vinesauce}
volumes:
- ${TWITCH_ARCHIVE_ARCHIVE_BIND:-./archive}:/app/archive
- ${TWITCH_ARCHIVE_CONFIG_BIND:-./config}:/app/config
healthcheck:
test:
- CMD-SHELL
- python twitch-archive.py --healthcheck -u ${TWITCH_ARCHIVE_HEALTHCHECK_STREAMER:-vinesauce}
interval: 30s
timeout: 10s
retries: 3
start_period: 30s

6
docker/entrypoint.sh Normal file
View file

@ -0,0 +1,6 @@
#!/bin/sh
set -eu
mkdir -p /app/archive /app/config /app/bin/temp
exec "$@"

49
docker/python.Dockerfile Normal file
View file

@ -0,0 +1,49 @@
FROM python:3.12-slim
ARG TWITCH_DOWNLOADER_VERSION=1.56.4
ARG TARGETARCH
ENV PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
RCLONE_CONFIG=/app/config/rclone.conf
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
ffmpeg \
libicu-dev \
rclone \
unzip \
&& rm -rf /var/lib/apt/lists/*
RUN set -eux; \
case "${TARGETARCH}" in \
amd64) twitch_downloader_arch='Linux-x64' ;; \
arm64) twitch_downloader_arch='LinuxArm64' ;; \
arm) twitch_downloader_arch='LinuxArm' ;; \
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
esac; \
curl -fsSL -o /tmp/TwitchDownloaderCLI.zip "https://github.com/lay295/TwitchDownloader/releases/download/${TWITCH_DOWNLOADER_VERSION}/TwitchDownloaderCLI-${TWITCH_DOWNLOADER_VERSION}-${twitch_downloader_arch}.zip"; \
mkdir -p /tmp/twitchdownloader; \
unzip -j /tmp/TwitchDownloaderCLI.zip TwitchDownloaderCLI -d /tmp/twitchdownloader; \
install -m 0755 /tmp/twitchdownloader/TwitchDownloaderCLI /usr/local/bin/TwitchDownloaderCLI; \
rm -rf /tmp/TwitchDownloaderCLI.zip /tmp/twitchdownloader
COPY requirements.txt ./requirements.txt
RUN python -m pip install --upgrade pip \
&& python -m pip install -r requirements.txt
COPY . /app
RUN mkdir -p /app/archive /app/config /app/bin/temp
COPY docker/entrypoint.sh /usr/local/bin/twitch-archive-entrypoint
RUN chmod +x /usr/local/bin/twitch-archive-entrypoint
ENTRYPOINT ["twitch-archive-entrypoint"]
CMD ["python", "twitch-archive.py", "-u", "vinesauce"]

4
dockerrebuild.bat Normal file
View file

@ -0,0 +1,4 @@
@echo off
setlocal
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml build --no-cache --pull twitch-archive

27
dockerstart.bat Normal file
View file

@ -0,0 +1,27 @@
@echo off
setlocal
set NVIDIA_COMPOSE=
if /I "%~1"=="--nvidia" (
set NVIDIA_COMPOSE=-f docker-compose.nvidia.yml
shift
)
if "%~1"=="" (
echo Usage: .\dockerstart.bat [--nvidia] streamer [additional args]
exit /b 1
)
set STREAMER=%~1
shift
set EXTRA_ARGS=
:collect_args
if "%~1"=="" goto run_compose
set EXTRA_ARGS=%EXTRA_ARGS% %~1
shift
goto collect_args
:run_compose
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml %NVIDIA_COMPOSE% run --rm twitch-archive python twitch-archive.py -u %STREAMER%%EXTRA_ARGS%

18
dotnet/Dockerfile Normal file
View file

@ -0,0 +1,18 @@
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY . .
RUN dotnet publish src/TwitchArchive.Web -c Release -o /app/publish
FROM mcr.microsoft.com/dotnet/aspnet:10.0
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg python3-pip rclone curl ca-certificates && rm -rf /var/lib/apt/lists/*
RUN pip3 install --no-cache-dir streamlink
WORKDIR /app
COPY --from=build /app/publish ./
# Download TwitchDownloaderCLI linux-x64 binary (if available)
RUN mkdir -p /app/bin && \
curl -fsSL -o /app/bin/TwitchDownloaderCLI https://github.com/Franiac/TwitchDownloader/releases/latest/download/TwitchDownloaderCLI-linux-x64 && \
chmod +x /app/bin/TwitchDownloaderCLI || true
EXPOSE 8080
ENV ASPNETCORE_URLS=http://+:8080
ENTRYPOINT ["dotnet", "TwitchArchive.Web.dll"]

14
dotnet/README.md Normal file
View file

@ -0,0 +1,14 @@
Twitch Archive (.NET) - Initial Scaffold
This folder contains the initial .NET 10 scaffold for the Twitch Archive rewrite.
Quick commands (from `dotnet/`):
```bash
dotnet build ./src/TwitchArchive.Core
dotnet test ./src/TwitchArchive.Tests
```
This initial commit includes a pure `RecoveryPolicy` implementation and unit tests.
Next steps: add services, process runner, Blazor Server app.

33
dotnet/TwitchArchive.sln Normal file
View file

@ -0,0 +1,33 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 18
VisualStudioVersion = 18.3.11512.155 d18.3
MinimumVisualStudioVersion = 10.0.40219.1
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TwitchArchive.Core", "src\TwitchArchive.Core\TwitchArchive.Core.csproj", "{9E6F1C3D-8B2A-4F21-9E9E-4A1B2C3D4E5F}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TwitchArchive.Web", "src\TwitchArchive.Web\TwitchArchive.Web.csproj", "{C2B1A3D4-5E6F-4721-9ABC-DEF012345679}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TwitchArchive.Tests", "src\TwitchArchive.Tests\TwitchArchive.Tests.csproj", "{D3E4F5A6-B7C8-49D0-9ABC-DEF012345670}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{9E6F1C3D-8B2A-4F21-9E9E-4A1B2C3D4E5F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{9E6F1C3D-8B2A-4F21-9E9E-4A1B2C3D4E5F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{9E6F1C3D-8B2A-4F21-9E9E-4A1B2C3D4E5F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{9E6F1C3D-8B2A-4F21-9E9E-4A1B2C3D4E5F}.Release|Any CPU.Build.0 = Release|Any CPU
{C2B1A3D4-5E6F-4721-9ABC-DEF012345679}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2B1A3D4-5E6F-4721-9ABC-DEF012345679}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C2B1A3D4-5E6F-4721-9ABC-DEF012345679}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C2B1A3D4-5E6F-4721-9ABC-DEF012345679}.Release|Any CPU.Build.0 = Release|Any CPU
{D3E4F5A6-B7C8-49D0-9ABC-DEF012345670}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D3E4F5A6-B7C8-49D0-9ABC-DEF012345670}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D3E4F5A6-B7C8-49D0-9ABC-DEF012345670}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D3E4F5A6-B7C8-49D0-9ABC-DEF012345670}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal

11
dotnet/docker-compose.yml Normal file
View file

@ -0,0 +1,11 @@
version: '3.8'
services:
twitcharchive:
build: .
ports:
- "8080:8080"
volumes:
- ./config:/app/config
- ./archive:/app/archive
environment:
- ASPNETCORE_ENVIRONMENT=Production

View file

@ -0,0 +1,17 @@
using System.Threading;
using System.Threading.Tasks;
using System.Net.Http;
using System.Collections.Generic;
namespace TwitchArchive.Core.Api
{
public record LiveStreamInfo(string? Title, string? CreatedAt, Dictionary<string, object>? Raw);
public record VodInfo(string Id, string Title, string RecordedAt);
public interface ITwitchApiClient
{
Task<string> GetOauthTokenAsync(CancellationToken ct = default);
Task<LiveStreamInfo?> GetStreamStatusAsync(string username, CancellationToken ct = default);
Task<VodInfo?> GetLatestVodAsync(string username, CancellationToken ct = default);
}
}

View file

@ -0,0 +1,111 @@
using System;
using System.Net.Http;
using System.Net.Http.Json;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Polly;
using Polly.Retry;
using System.Collections.Generic;
namespace TwitchArchive.Core.Api
{
public class TwitchApiClient : ITwitchApiClient
{
private readonly HttpClient _http;
private readonly AsyncRetryPolicy<HttpResponseMessage> _retryPolicy;
private string? _cachedToken;
private DateTime _tokenExpiryUtc;
private const string OAuthUrl = "https://id.twitch.tv/oauth2/token";
private const string GqlUrl = "https://gql.twitch.tv/gql";
private const string HelixUsersUrl = "https://api.twitch.tv/helix/users";
public TwitchApiClient(HttpClient http)
{
_http = http;
_retryPolicy = Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
.WaitAndRetryAsync(new[] { TimeSpan.FromSeconds(2), TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(10) });
}
public async Task<string> GetOauthTokenAsync(CancellationToken ct = default)
{
if (!string.IsNullOrEmpty(_cachedToken) && DateTime.UtcNow < _tokenExpiryUtc.AddSeconds(-30))
{
return _cachedToken!;
}
var clientId = Environment.GetEnvironmentVariable("CLIENT-ID");
var clientSecret = Environment.GetEnvironmentVariable("CLIENT-SECRET");
if (string.IsNullOrEmpty(clientId) || string.IsNullOrEmpty(clientSecret))
throw new InvalidOperationException("CLIENT-ID and CLIENT-SECRET must be set in environment");
var url = $"{OAuthUrl}?client_id={clientId}&client_secret={clientSecret}&grant_type=client_credentials";
var resp = await _retryPolicy.ExecuteAsync(() => _http.PostAsync(url, null, ct)).ConfigureAwait(false);
resp.EnsureSuccessStatusCode();
var doc = await resp.Content.ReadFromJsonAsync<JsonElement>(cancellationToken: ct).ConfigureAwait(false);
if (doc.ValueKind == JsonValueKind.Object && doc.TryGetProperty("access_token", out var tokenEl))
{
_cachedToken = tokenEl.GetString();
if (doc.TryGetProperty("expires_in", out var expiresEl) && expiresEl.TryGetInt32(out var secs))
{
_tokenExpiryUtc = DateTime.UtcNow.AddSeconds(secs);
}
return _cachedToken!;
}
throw new InvalidOperationException("Failed to obtain OAuth token from Twitch");
}
public async Task<LiveStreamInfo?> GetStreamStatusAsync(string username, CancellationToken ct = default)
{
// Simple GQL query to fetch stream data
var query = $"query{{user(login: \"{username}\") {{stream{{title createdAt archiveVideo{{id}}}}}}}}";
var payload = JsonSerializer.Serialize(new { query });
var content = new StringContent(payload, System.Text.Encoding.UTF8, "application/json");
var resp = await _retryPolicy.ExecuteAsync(() => _http.PostAsync(GqlUrl, content, ct)).ConfigureAwait(false);
if (!resp.IsSuccessStatusCode) return null;
var json = await resp.Content.ReadFromJsonAsync<JsonDocument>(cancellationToken: ct).ConfigureAwait(false);
if (json == null) return null;
// navigate JSON safely
try
{
var root = json.RootElement;
if (root.TryGetProperty("data", out var data) && data.TryGetProperty("user", out var user) && user.TryGetProperty("stream", out var stream) && stream.ValueKind != JsonValueKind.Null)
{
var title = stream.GetProperty("title").GetString();
var createdAt = stream.GetProperty("createdAt").GetString();
var raw = new Dictionary<string, object>();
return new LiveStreamInfo(title, createdAt, raw);
}
}
catch { }
return null;
}
public async Task<VodInfo?> GetLatestVodAsync(string username, CancellationToken ct = default)
{
var query = $"query {{user(login: \"{username}\") {{videos(first: 1) {{edges {{node {{id title recordedAt}}}}}}}}}}";
var payload = JsonSerializer.Serialize(new { query });
var content = new StringContent(payload, System.Text.Encoding.UTF8, "application/json");
var resp = await _retryPolicy.ExecuteAsync(() => _http.PostAsync(GqlUrl, content, ct)).ConfigureAwait(false);
if (!resp.IsSuccessStatusCode) return null;
var json = await resp.Content.ReadFromJsonAsync<JsonDocument>(cancellationToken: ct).ConfigureAwait(false);
if (json == null) return null;
try
{
var root = json.RootElement;
var node = root.GetProperty("data").GetProperty("user").GetProperty("videos").GetProperty("edges")[0].GetProperty("node");
var id = node.GetProperty("id").GetString()!;
var title = node.GetProperty("title").GetString()!;
var recordedAt = node.GetProperty("recordedAt").GetString()!;
return new VodInfo(id, title, recordedAt);
}
catch { }
return null;
}
}
}

View file

@ -0,0 +1,17 @@
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Design;
using TwitchArchive.Core.Persistence;
namespace TwitchArchive.Core
{
public class ArchiveDbContextFactory : IDesignTimeDbContextFactory<ArchiveDbContext>
{
public ArchiveDbContext CreateDbContext(string[] args)
{
var builder = new DbContextOptionsBuilder<ArchiveDbContext>();
var conn = "Data Source=archive.db";
builder.UseSqlite(conn);
return new ArchiveDbContext(builder.Options);
}
}
}

View file

@ -0,0 +1,22 @@
using System.Text.Json.Serialization;
namespace TwitchArchive.Core.Config
{
public class AppSettings
{
[JsonPropertyName("streamlink_path")]
public string? StreamlinkPath { get; set; }
[JsonPropertyName("ffmpeg_path")]
public string? FfmpegPath { get; set; }
[JsonPropertyName("twitchdownloader_path")]
public string? TwitchDownloaderPath { get; set; }
[JsonPropertyName("rclone_path")]
public string? RclonePath { get; set; }
[JsonPropertyName("password_hash")]
public string? PasswordHash { get; set; }
}
}

View file

@ -0,0 +1,119 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.Json;
namespace TwitchArchive.Core.Config
{
public class ConfigurationService : IConfigurationService
{
private readonly string _basePath;
private readonly string _streamersPath;
private readonly JsonSerializerOptions _jsonOptions = new JsonSerializerOptions { WriteIndented = true };
public ConfigurationService(string? basePath = null)
{
// If a basePath was explicitly provided, use it. Otherwise try to find a
// repository-level `config` folder by walking parent directories from the
// application base. This ensures the web app uses the same config/ files
// as the repository (global.json, config/streamers/*.json) when available.
if (!string.IsNullOrWhiteSpace(basePath))
{
_basePath = basePath;
}
else
{
var found = FindExistingConfigFolder();
_basePath = found ?? Path.Combine(AppContext.BaseDirectory, "config");
}
_streamersPath = Path.Combine(_basePath, "streamers");
Directory.CreateDirectory(_basePath);
Directory.CreateDirectory(_streamersPath);
}
private string? FindExistingConfigFolder()
{
var start = AppContext.BaseDirectory ?? Environment.CurrentDirectory;
var dir = new DirectoryInfo(start);
for (int i = 0; i < 8 && dir != null; i++)
{
var candidate = Path.Combine(dir.FullName, "config");
if (Directory.Exists(candidate))
{
// prefer candidate if it contains global.json or streamers
if (File.Exists(Path.Combine(candidate, "global.json")) || Directory.Exists(Path.Combine(candidate, "streamers")))
return candidate;
}
dir = dir.Parent;
}
return null;
}
public GlobalConfig LoadGlobal()
{
var file = Path.Combine(_basePath, "global.json");
if (!File.Exists(file)) return new GlobalConfig();
var txt = File.ReadAllText(file);
return JsonSerializer.Deserialize<GlobalConfig>(txt, _jsonOptions) ?? new GlobalConfig();
}
public void SaveGlobal(GlobalConfig cfg)
{
var file = Path.Combine(_basePath, "global.json");
var txt = JsonSerializer.Serialize(cfg, _jsonOptions);
File.WriteAllText(file, txt);
}
public StreamerConfig? LoadStreamer(string username)
{
if (string.IsNullOrWhiteSpace(username)) return null;
var file = Path.Combine(_streamersPath, username + ".json");
if (!File.Exists(file)) return null;
var txt = File.ReadAllText(file);
return JsonSerializer.Deserialize<StreamerConfig>(txt, _jsonOptions);
}
public void SaveStreamer(StreamerConfig cfg)
{
if (cfg == null) throw new ArgumentNullException(nameof(cfg));
if (string.IsNullOrWhiteSpace(cfg.Username)) throw new ArgumentException("Username required");
var file = Path.Combine(_streamersPath, cfg.Username + ".json");
var txt = JsonSerializer.Serialize(cfg, _jsonOptions);
File.WriteAllText(file, txt);
}
public IEnumerable<StreamerConfig> GetAllStreamers()
{
if (!Directory.Exists(_streamersPath)) return Enumerable.Empty<StreamerConfig>();
var files = Directory.EnumerateFiles(_streamersPath, "*.json");
var list = new List<StreamerConfig>();
foreach (var f in files)
{
try
{
var txt = File.ReadAllText(f);
var s = JsonSerializer.Deserialize<StreamerConfig>(txt, _jsonOptions);
if (s != null) list.Add(s);
}
catch { }
}
return list;
}
public EffectiveConfig GetEffectiveConfig(string username)
{
var global = LoadGlobal();
var streamer = LoadStreamer(username);
return EffectiveConfig.Merge(global, streamer);
}
public void DeleteStreamer(string username)
{
if (string.IsNullOrWhiteSpace(username)) return;
var file = Path.Combine(_streamersPath, username + ".json");
try { if (File.Exists(file)) File.Delete(file); } catch { }
}
}
}

View file

@ -0,0 +1,71 @@
using System;
namespace TwitchArchive.Core.Config
{
public class EffectiveConfig
{
public string? ArchiveRoot { get; init; }
public string? StreamlinkPath { get; init; }
public string? FfmpegPath { get; init; }
public string? TwitchDownloaderPath { get; init; }
public string? RclonePath { get; init; }
public bool UploadToCloud { get; init; }
public string? UploadDestination { get; init; }
public int RefreshIntervalSeconds { get; init; }
public int StreamSegmentThreads { get; init; }
public string? DefaultQuality { get; init; }
// Defaults that can be overridden per-streamer
public DefaultsSection Defaults { get; init; } = new DefaultsSection();
public static EffectiveConfig Merge(GlobalConfig global, StreamerConfig? streamer)
{
streamer ??= new StreamerConfig();
var d = new TwitchArchive.Core.Config.DefaultsSection();
// start with global defaults
if (global?.Defaults != null) d = global.Defaults;
// apply per-streamer overrides when present
var mergedDefaults = new TwitchArchive.Core.Config.DefaultsSection
{
DownloadVOD = streamer.DownloadVOD ?? d.DownloadVOD,
DownloadCHAT = streamer.DownloadCHAT ?? d.DownloadCHAT,
DownloadLiveCHAT = streamer.DownloadLiveCHAT ?? d.DownloadLiveCHAT,
MergeVideoChat = streamer.MergeVideoChat ?? d.MergeVideoChat,
MergeChatLayout = streamer.MergeChatLayout ?? d.MergeChatLayout,
VodTimeout = streamer.VodTimeout ?? d.VodTimeout,
UploadCloud = streamer.UploadCloud ?? d.UploadCloud,
UploadPreMergeVideo = streamer.UploadPreMergeVideo ?? d.UploadPreMergeVideo,
UploadMergedVideo = streamer.UploadMergedVideo ?? d.UploadMergedVideo,
UploadChatVideo = streamer.UploadChatVideo ?? d.UploadChatVideo,
DeleteFiles = streamer.DeleteFiles ?? d.DeleteFiles,
OnlyRaw = streamer.OnlyRaw ?? d.OnlyRaw,
CleanRaw = streamer.CleanRaw ?? d.CleanRaw,
HlsSegments = streamer.HlsSegments ?? d.HlsSegments,
HlsSegmentsVOD = streamer.HlsSegmentsVOD ?? d.HlsSegmentsVOD,
StreamlinkTtvlol = streamer.StreamlinkTtvlol ?? d.StreamlinkTtvlol,
FfmpegHwaccel = streamer.FfmpegHwaccel ?? d.FfmpegHwaccel,
FfmpegThreads = streamer.FfmpegThreads ?? d.FfmpegThreads,
FfmpegAudioCodec = streamer.FfmpegAudioCodec ?? d.FfmpegAudioCodec,
FfmpegAudioSamplerate = streamer.FfmpegAudioSamplerate ?? d.FfmpegAudioSamplerate,
FfmpegAudioBitrate = streamer.FfmpegAudioBitrate ?? d.FfmpegAudioBitrate,
FfmpegErrorRecovery = streamer.FfmpegErrorRecovery ?? d.FfmpegErrorRecovery,
FfmpegFaststart = streamer.FfmpegFaststart ?? d.FfmpegFaststart,
FfmpegProgress = streamer.FfmpegProgress ?? d.FfmpegProgress
};
return new EffectiveConfig
{
ArchiveRoot = streamer.Username != null ? (global.ArchiveRoot ?? string.Empty) : (global.ArchiveRoot ?? string.Empty),
StreamlinkPath = streamer.StreamlinkPath ?? global.StreamlinkPath,
FfmpegPath = global.FfmpegPath,
TwitchDownloaderPath = global.TwitchDownloaderPath,
RclonePath = global.RclonePath,
UploadToCloud = streamer.UploadToCloud ?? global.UploadToCloud,
UploadDestination = streamer.UploadDestination ?? global.UploadDestination,
RefreshIntervalSeconds = global.RefreshIntervalSeconds,
StreamSegmentThreads = global.StreamSegmentThreads,
DefaultQuality = streamer.Quality ?? global.DefaultQuality,
Defaults = mergedDefaults
};
}
}
}

View file

@ -0,0 +1,124 @@
using System.ComponentModel.DataAnnotations;
using System.Text.Json.Serialization;
namespace TwitchArchive.Core.Config
{
public class GlobalConfig
{
[JsonPropertyName("archive_root")]
public string? ArchiveRoot { get; set; }
[JsonPropertyName("streamlink_path")]
public string? StreamlinkPath { get; set; }
[JsonPropertyName("ffmpeg_path")]
public string? FfmpegPath { get; set; }
[JsonPropertyName("twitchdownloader_path")]
public string? TwitchDownloaderPath { get; set; }
[JsonPropertyName("rclone_path")]
public string? RclonePath { get; set; }
[JsonPropertyName("upload_to_cloud")]
public bool UploadToCloud { get; set; } = false;
[JsonPropertyName("upload_destination")]
public string? UploadDestination { get; set; }
[JsonPropertyName("refresh_interval_seconds")]
[Range(5, 86400, ErrorMessage = "Refresh interval must be between 5 and 86400 seconds.")]
public int RefreshIntervalSeconds { get; set; } = 60;
[JsonPropertyName("stream_segment_threads")]
[Range(1, 64, ErrorMessage = "Stream segment threads must be between 1 and 64.")]
public int StreamSegmentThreads { get; set; } = 4;
[JsonPropertyName("default_quality")]
public string? DefaultQuality { get; set; } = "best";
// Defaults section for per-streamer fallbacks
[JsonPropertyName("defaults")]
public DefaultsSection Defaults { get; set; } = new DefaultsSection();
}
public class DefaultsSection
{
[JsonPropertyName("downloadVOD")]
public bool DownloadVOD { get; set; } = true;
[JsonPropertyName("downloadCHAT")]
public bool DownloadCHAT { get; set; } = true;
[JsonPropertyName("downloadLiveCHAT")]
public bool DownloadLiveCHAT { get; set; } = true;
[JsonPropertyName("mergeVideoChat")]
public bool MergeVideoChat { get; set; } = false;
[JsonPropertyName("mergeChatLayout")]
public string MergeChatLayout { get; set; } = "side-by-side";
[JsonPropertyName("vodTimeout")]
[Range(0, 86400, ErrorMessage = "VOD timeout must be between 0 and 86400 seconds.")]
public int VodTimeout { get; set; } = 300;
[JsonPropertyName("uploadCloud")]
public bool UploadCloud { get; set; } = false;
[JsonPropertyName("uploadPreMergeVideo")]
public bool UploadPreMergeVideo { get; set; } = true;
[JsonPropertyName("uploadMergedVideo")]
public bool UploadMergedVideo { get; set; } = true;
[JsonPropertyName("uploadChatVideo")]
public bool UploadChatVideo { get; set; } = false;
[JsonPropertyName("deleteFiles")]
public bool DeleteFiles { get; set; } = false;
[JsonPropertyName("onlyRaw")]
public bool OnlyRaw { get; set; } = false;
[JsonPropertyName("cleanRaw")]
public bool CleanRaw { get; set; } = true;
[JsonPropertyName("hls_segments")]
[Range(1, 50, ErrorMessage = "HLS segments must be between 1 and 50.")]
public int HlsSegments { get; set; } = 3;
[JsonPropertyName("hls_segmentsVOD")]
[Range(1, 200, ErrorMessage = "HLS segments (VOD) must be between 1 and 200.")]
public int HlsSegmentsVOD { get; set; } = 10;
[JsonPropertyName("streamlink_ttvlol")]
public bool StreamlinkTtvlol { get; set; } = false;
[JsonPropertyName("ffmpeg_hwaccel")]
public string FfmpegHwaccel { get; set; } = "auto";
[JsonPropertyName("ffmpeg_threads")]
[Range(0, 128, ErrorMessage = "FFmpeg threads must be between 0 and 128.")]
public int FfmpegThreads { get; set; } = 0;
[JsonPropertyName("ffmpeg_audio_codec")]
public string FfmpegAudioCodec { get; set; } = "aac";
[JsonPropertyName("ffmpeg_audio_samplerate")]
[Range(8000, 192000, ErrorMessage = "Audio sample rate must be between 8000 and 192000.")]
public int FfmpegAudioSamplerate { get; set; } = 48000;
[JsonPropertyName("ffmpeg_audio_bitrate")]
public string FfmpegAudioBitrate { get; set; } = "192k";
[JsonPropertyName("ffmpeg_error_recovery")]
public bool FfmpegErrorRecovery { get; set; } = true;
[JsonPropertyName("ffmpeg_faststart")]
public bool FfmpegFaststart { get; set; } = true;
[JsonPropertyName("ffmpeg_progress")]
public bool FfmpegProgress { get; set; } = false;
}
}

View file

@ -0,0 +1,18 @@
using System.Collections.Generic;
namespace TwitchArchive.Core.Config
{
public interface IConfigurationService
{
GlobalConfig LoadGlobal();
void SaveGlobal(GlobalConfig cfg);
StreamerConfig? LoadStreamer(string username);
void SaveStreamer(StreamerConfig cfg);
void DeleteStreamer(string username);
IEnumerable<StreamerConfig> GetAllStreamers();
EffectiveConfig GetEffectiveConfig(string username);
}
}

View file

@ -0,0 +1,105 @@
using System.ComponentModel.DataAnnotations;
using System.Text.Json.Serialization;
namespace TwitchArchive.Core.Config
{
public class StreamerConfig
{
[JsonPropertyName("username")]
[Required]
public string Username { get; set; } = string.Empty;
[JsonPropertyName("enabled")]
public bool Enabled { get; set; } = true;
[JsonPropertyName("quality")]
public string? Quality { get; set; }
[JsonPropertyName("upload_to_cloud")]
public bool? UploadToCloud { get; set; }
[JsonPropertyName("upload_destination")]
public string? UploadDestination { get; set; }
[JsonPropertyName("streamlink_path")]
public string? StreamlinkPath { get; set; }
// Per-streamer override options matching GlobalConfig.Defaults
[JsonPropertyName("downloadVOD")]
public bool? DownloadVOD { get; set; }
[JsonPropertyName("downloadCHAT")]
public bool? DownloadCHAT { get; set; }
[JsonPropertyName("downloadLiveCHAT")]
public bool? DownloadLiveCHAT { get; set; }
[JsonPropertyName("mergeVideoChat")]
public bool? MergeVideoChat { get; set; }
[JsonPropertyName("mergeChatLayout")]
public string? MergeChatLayout { get; set; }
[JsonPropertyName("vodTimeout")]
[Range(0, 86400, ErrorMessage = "VOD timeout must be between 0 and 86400 seconds.")]
public int? VodTimeout { get; set; }
[JsonPropertyName("uploadCloud")]
public bool? UploadCloud { get; set; }
[JsonPropertyName("uploadPreMergeVideo")]
public bool? UploadPreMergeVideo { get; set; }
[JsonPropertyName("uploadMergedVideo")]
public bool? UploadMergedVideo { get; set; }
[JsonPropertyName("uploadChatVideo")]
public bool? UploadChatVideo { get; set; }
[JsonPropertyName("deleteFiles")]
public bool? DeleteFiles { get; set; }
[JsonPropertyName("onlyRaw")]
public bool? OnlyRaw { get; set; }
[JsonPropertyName("cleanRaw")]
public bool? CleanRaw { get; set; }
[JsonPropertyName("hls_segments")]
[Range(1, 50, ErrorMessage = "HLS segments must be between 1 and 50.")]
public int? HlsSegments { get; set; }
[JsonPropertyName("hls_segmentsVOD")]
[Range(1, 200, ErrorMessage = "HLS segments (VOD) must be between 1 and 200.")]
public int? HlsSegmentsVOD { get; set; }
[JsonPropertyName("streamlink_ttvlol")]
public bool? StreamlinkTtvlol { get; set; }
[JsonPropertyName("ffmpeg_hwaccel")]
public string? FfmpegHwaccel { get; set; }
[JsonPropertyName("ffmpeg_threads")]
[Range(0, 128, ErrorMessage = "FFmpeg threads must be between 0 and 128.")]
public int? FfmpegThreads { get; set; }
[JsonPropertyName("ffmpeg_audio_codec")]
public string? FfmpegAudioCodec { get; set; }
[JsonPropertyName("ffmpeg_audio_samplerate")]
[Range(8000, 192000, ErrorMessage = "Audio sample rate must be between 8000 and 192000.")]
public int? FfmpegAudioSamplerate { get; set; }
[JsonPropertyName("ffmpeg_audio_bitrate")]
public string? FfmpegAudioBitrate { get; set; }
[JsonPropertyName("ffmpeg_error_recovery")]
public bool? FfmpegErrorRecovery { get; set; }
[JsonPropertyName("ffmpeg_faststart")]
public bool? FfmpegFaststart { get; set; }
[JsonPropertyName("ffmpeg_progress")]
public bool? FfmpegProgress { get; set; }
}
}

View file

@ -0,0 +1,35 @@
using System.Runtime.InteropServices;
namespace TwitchArchive.Core.Config
{
public static class ToolPathResolver
{
public static string DefaultStreamlinkPath()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
return "streamlink.exe";
return "/usr/local/bin/streamlink";
}
public static string DefaultFfmpegPath()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
return "ffmpeg.exe";
return "/usr/bin/ffmpeg";
}
public static string DefaultTwitchDownloaderPath()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
return "TwitchDownloaderCLI.exe";
return "/app/bin/TwitchDownloaderCLI";
}
public static string DefaultRclonePath()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
return "rclone.exe";
return "/usr/bin/rclone";
}
}
}

View file

@ -0,0 +1,17 @@
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Monitoring
{
/// <summary>
/// Dummy implementation used for initial scaffolding — always reports offline (false).
/// Replace this with a Twitch API backed implementation later.
/// </summary>
public class DummyLiveChecker : ILiveChecker
{
public Task<bool?> IsLiveAsync(string username, CancellationToken ct = default)
{
return Task.FromResult<bool?>(false);
}
}
}

View file

@ -0,0 +1,13 @@
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Monitoring
{
public interface ILiveChecker
{
/// <summary>
/// Return true = live, false = offline, null = unknown/error
/// </summary>
Task<bool?> IsLiveAsync(string username, CancellationToken ct = default);
}
}

View file

@ -0,0 +1,29 @@
using System.Threading;
using System.Threading.Tasks;
using TwitchArchive.Core.Api;
namespace TwitchArchive.Core.Monitoring
{
public class TwitchLiveChecker : ILiveChecker
{
private readonly ITwitchApiClient _api;
public TwitchLiveChecker(ITwitchApiClient api)
{
_api = api;
}
public async Task<bool?> IsLiveAsync(string username, CancellationToken ct = default)
{
try
{
var info = await _api.GetStreamStatusAsync(username, ct).ConfigureAwait(false);
return info != null;
}
catch
{
return null; // unknown / network error
}
}
}
}

View file

@ -0,0 +1,15 @@
using Microsoft.EntityFrameworkCore;
using TwitchArchive.Core.Persistence.Models;
namespace TwitchArchive.Core.Persistence
{
public class ArchiveDbContext : DbContext
{
public ArchiveDbContext(DbContextOptions<ArchiveDbContext> options) : base(options) { }
public DbSet<StreamSession> StreamSessions { get; set; } = null!;
public DbSet<ArchiveJob> ArchiveJobs { get; set; } = null!;
public DbSet<StreamerState> StreamerStates { get; set; } = null!;
public DbSet<TwitchArchive.Core.Persistence.Models.UserCredential> UserCredentials { get; set; } = null!;
}
}

View file

@ -0,0 +1,30 @@
using System;
using Microsoft.EntityFrameworkCore;
namespace TwitchArchive.Core.Persistence
{
public static class ArchiveDbInitializer
{
public static void EnsureUserCredentialsTable(IDbContextFactory<ArchiveDbContext> factory)
{
if (factory == null) throw new ArgumentNullException(nameof(factory));
try
{
using var ctx = factory.CreateDbContext();
var conn = ctx.Database.GetDbConnection();
try { conn.Open(); } catch { /* ignore open errors */ }
using var cmd = conn.CreateCommand();
// Create table if it doesn't exist (SQLite syntax)
cmd.CommandText = @"CREATE TABLE IF NOT EXISTS UserCredentials (
Id INTEGER PRIMARY KEY AUTOINCREMENT,
PasswordHash TEXT NOT NULL
);";
cmd.ExecuteNonQuery();
}
catch
{
// Initialization should not crash the app; log if needed
}
}
}
}

View file

@ -0,0 +1,17 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using TwitchArchive.Core.Persistence.Models;
namespace TwitchArchive.Core.Persistence
{
public interface ISessionRepository
{
Task<StreamSession> CreateSessionAsync(string username, string streamId, DateTime startedAt, CancellationToken ct = default);
Task EndSessionAsync(long sessionId, DateTime endedAt, string status, CancellationToken ct = default);
Task<ArchiveJob> CreateJobAsync(long sessionId, string jobType, DateTime startedAt, CancellationToken ct = default);
Task UpdateJobAsync(ArchiveJob job, CancellationToken ct = default);
Task<List<StreamSession>> GetRecentSessionsAsync(int max = 50, CancellationToken ct = default);
Task<List<ArchiveJob>> GetJobsForSessionAsync(long sessionId, CancellationToken ct = default);
}
}

View file

@ -0,0 +1,16 @@
using System;
namespace TwitchArchive.Core.Persistence.Models
{
public class ArchiveJob
{
public long Id { get; set; }
public long StreamSessionId { get; set; }
public string JobType { get; set; } = string.Empty; // enum name
public string Status { get; set; } = string.Empty; // Pending, Running, Completed, Failed
public DateTime StartedAt { get; set; }
public DateTime? CompletedAt { get; set; }
public string? FilePath { get; set; }
public string? ErrorMessage { get; set; }
}
}

View file

@ -0,0 +1,15 @@
using System;
namespace TwitchArchive.Core.Persistence.Models
{
public class StreamSession
{
public long Id { get; set; }
public string StreamerUsername { get; set; } = string.Empty;
public string TwitchStreamId { get; set; } = string.Empty;
public string Title { get; set; } = string.Empty;
public DateTime StartedAt { get; set; }
public DateTime? EndedAt { get; set; }
public string Status { get; set; } = string.Empty; // Recording, Processing, Uploading, Complete, Failed
}
}

View file

@ -0,0 +1,13 @@
using System;
namespace TwitchArchive.Core.Persistence.Models
{
public class StreamerState
{
public long Id { get; set; }
public string Username { get; set; } = string.Empty;
public bool IsMonitoring { get; set; }
public DateTime? LastCheckedAt { get; set; }
public string? CurrentRecoveryState { get; set; }
}
}

View file

@ -0,0 +1,8 @@
namespace TwitchArchive.Core.Persistence.Models
{
public class UserCredential
{
public int Id { get; set; }
public string PasswordHash { get; set; } = string.Empty;
}
}

View file

@ -0,0 +1,62 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using TwitchArchive.Core.Persistence.Models;
namespace TwitchArchive.Core.Persistence
{
public class SessionRepository : ISessionRepository
{
private readonly IDbContextFactory<ArchiveDbContext> _factory;
public SessionRepository(IDbContextFactory<ArchiveDbContext> factory) { _factory = factory ?? throw new ArgumentNullException(nameof(factory)); }
public async Task<StreamSession> CreateSessionAsync(string username, string streamId, DateTime startedAt, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
var s = new StreamSession { StreamerUsername = username, TwitchStreamId = streamId, StartedAt = startedAt, Status = "Recording" };
db.StreamSessions.Add(s);
await db.SaveChangesAsync(ct).ConfigureAwait(false);
return s;
}
public async Task EndSessionAsync(long sessionId, DateTime endedAt, string status, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
var s = await db.StreamSessions.FindAsync(new object[] { sessionId }, ct).ConfigureAwait(false);
if (s == null) return;
s.EndedAt = endedAt;
s.Status = status;
await db.SaveChangesAsync(ct).ConfigureAwait(false);
}
public async Task<ArchiveJob> CreateJobAsync(long sessionId, string jobType, DateTime startedAt, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
var j = new ArchiveJob { StreamSessionId = sessionId, JobType = jobType, StartedAt = startedAt, Status = "Running" };
db.ArchiveJobs.Add(j);
await db.SaveChangesAsync(ct).ConfigureAwait(false);
return j;
}
public async Task UpdateJobAsync(ArchiveJob job, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
db.ArchiveJobs.Update(job);
await db.SaveChangesAsync(ct).ConfigureAwait(false);
}
public async Task<List<StreamSession>> GetRecentSessionsAsync(int max = 50, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
return await db.StreamSessions.OrderByDescending(s => s.StartedAt).Take(max).ToListAsync(ct).ConfigureAwait(false);
}
public async Task<List<ArchiveJob>> GetJobsForSessionAsync(long sessionId, CancellationToken ct = default)
{
await using var db = _factory.CreateDbContext();
return await db.ArchiveJobs.Where(j => j.StreamSessionId == sessionId).OrderBy(j => j.StartedAt).ToListAsync(ct).ConfigureAwait(false);
}
}
}

View file

@ -0,0 +1,131 @@
using System;
namespace TwitchArchive.Core.Recovery
{
public enum RecoveryState
{
Monitoring,
Recording,
FastReconnect,
SlowReconnect,
NetworkFault
}
public enum RecoveryAction
{
None,
StartRecording,
StartProcessing,
SleepOnly
}
public sealed class RecoveryDecision
{
public RecoveryAction Action { get; init; }
public TimeSpan Sleep { get; init; }
public static RecoveryDecision SleepFor(TimeSpan t) => new RecoveryDecision { Action = RecoveryAction.SleepOnly, Sleep = t };
public static RecoveryDecision StartRecordingNow() => new RecoveryDecision { Action = RecoveryAction.StartRecording, Sleep = TimeSpan.Zero };
public static RecoveryDecision StartProcessingAndSleep(TimeSpan sleep) => new RecoveryDecision { Action = RecoveryAction.StartProcessing, Sleep = sleep };
public static RecoveryDecision None() => new RecoveryDecision { Action = RecoveryAction.None, Sleep = TimeSpan.Zero };
}
/// <summary>
/// Pure, testable recovery policy state machine.
/// It decides whether to start recording immediately, wait a short period (fast reconnect),
/// wait longer (slow reconnect), or enter a network fault backoff.
///</summary>
public class RecoveryPolicy
{
private RecoveryState _state = RecoveryState.Monitoring;
private readonly TimeSpan _refreshInterval;
private DateTime? _fastReconnectStartUtc;
private int _networkFaultAttempts;
// constants matching requested behavior
private static readonly TimeSpan FastReconnectInterval = TimeSpan.FromSeconds(10);
private static readonly TimeSpan FastReconnectWindow = TimeSpan.FromMinutes(2);
private static readonly TimeSpan SlowReconnectInterval = TimeSpan.FromSeconds(60);
private static readonly TimeSpan NetworkFaultBase = TimeSpan.FromSeconds(30);
private static readonly TimeSpan NetworkFaultMax = TimeSpan.FromMinutes(10);
public RecoveryPolicy(TimeSpan? refreshInterval = null)
{
_refreshInterval = refreshInterval ?? TimeSpan.FromSeconds(60);
}
/// <summary>
/// Evaluate the policy given current time, whether the streamer is live (null = unknown),
/// and whether there was a network error reaching Twitch APIs.
/// Returns a RecoveryDecision describing the next action and how long to wait if sleeping.
/// </summary>
public RecoveryDecision Tick(DateTime nowUtc, bool? isLive, bool networkError)
{
// Network error handling: enter NetworkFault state with exponential backoff
if (networkError)
{
_networkFaultAttempts++;
_state = RecoveryState.NetworkFault;
var backoff = NetworkFaultBase * Math.Pow(2, Math.Max(0, _networkFaultAttempts - 1));
if (backoff > NetworkFaultMax) backoff = NetworkFaultMax;
return RecoveryDecision.SleepFor(backoff);
}
// If we recover from network fault, reset attempts
if (_state == RecoveryState.NetworkFault && !networkError)
{
_networkFaultAttempts = 0;
_state = RecoveryState.Monitoring;
}
// If live -> start recording immediately
if (isLive == true)
{
_state = RecoveryState.Recording;
_fastReconnectStartUtc = null;
return RecoveryDecision.StartRecordingNow();
}
// If unknown live status, treat conservatively: sleep a short amount (refresh)
if (isLive == null)
{
return RecoveryDecision.SleepFor(_refreshInterval);
}
// isLive == false here
switch (_state)
{
case RecoveryState.Recording:
// Just transitioned from recording to not-live: start fast reconnect window
_state = RecoveryState.FastReconnect;
_fastReconnectStartUtc = nowUtc;
return RecoveryDecision.SleepFor(FastReconnectInterval);
case RecoveryState.FastReconnect:
if (_fastReconnectStartUtc.HasValue && (nowUtc - _fastReconnectStartUtc.Value) < FastReconnectWindow)
{
// stay in fast reconnect, poll frequently
return RecoveryDecision.SleepFor(FastReconnectInterval);
}
else
{
// fast reconnect window expired -> move to slow reconnect and trigger processing
_state = RecoveryState.SlowReconnect;
_fastReconnectStartUtc = null;
return RecoveryDecision.StartProcessingAndSleep(SlowReconnectInterval);
}
case RecoveryState.SlowReconnect:
// In slow reconnect, poll less frequently
return RecoveryDecision.SleepFor(SlowReconnectInterval);
case RecoveryState.Monitoring:
default:
// Normal monitoring cadence
return RecoveryDecision.SleepFor(_refreshInterval);
}
}
public RecoveryState CurrentState => _state;
}
}

View file

@ -0,0 +1,50 @@
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public class DownloaderService : IDownloaderService
{
private readonly IProcessRunner _runner;
public DownloaderService(IProcessRunner runner)
{
_runner = runner;
}
public async Task<bool> DownloadVodAsync(TwitchArchive.Core.Api.VodInfo vod, string outputPath, CancellationToken ct = default)
{
try
{
var bin = "TwitchDownloaderCLI"; // expect to be on PATH or configured elsewhere
Directory.CreateDirectory(Path.GetDirectoryName(outputPath) ?? ".");
var args = $"videodownload -u https://www.twitch.tv/videos/{(vod.Id.StartsWith("v") ? vod.Id.Substring(1) : vod.Id)} -q best -t 10 --ffmpeg-path ffmpeg --collision Rename -o \"{outputPath}\"";
var res = await _runner.RunAsync(new ProcessRunOptions { FileName = bin, Arguments = args, RedirectOutput = true }, ct).ConfigureAwait(false);
return res.ExitCode == 0;
}
catch
{
return false;
}
}
public async Task<bool> DownloadChatJsonAsync(string vodId, string jsonPath, CancellationToken ct = default)
{
try
{
var bin = "TwitchDownloaderCLI";
Directory.CreateDirectory(Path.GetDirectoryName(jsonPath) ?? ".");
if (vodId.StartsWith("v")) vodId = vodId.Substring(1);
var args = $"chatdownload --id {vodId} --embed-images --collision Rename -o \"{jsonPath}\"";
var res = await _runner.RunAsync(new ProcessRunOptions { FileName = bin, Arguments = args, RedirectOutput = true }, ct).ConfigureAwait(false);
return res.ExitCode == 0 && File.Exists(jsonPath);
}
catch
{
return false;
}
}
}
}

View file

@ -0,0 +1,52 @@
using System;
using System.IO;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public record ArchivePaths(string Root, string RawPath, string VideoPath, string ChatJsonPath, string ChatMp4Path, string MetadataPath);
public class FileManagerService
{
private readonly string _root;
public FileManagerService(string? root = null)
{
_root = root ?? Path.Combine(Directory.GetCurrentDirectory(), "archive");
}
public ArchivePaths GetPaths(string username)
{
var root = Path.Combine(_root, username);
var raw = Path.Combine(root, "video", "raw");
var video = Path.Combine(root, "video");
var chatJson = Path.Combine(root, "chat", "json");
var chatMp4 = Path.Combine(root, "chat");
var meta = Path.Combine(root, "metadata");
return new ArchivePaths(root, raw, video, chatJson, chatMp4, meta);
}
public void EnsureDirectories(ArchivePaths paths)
{
Directory.CreateDirectory(paths.RawPath);
Directory.CreateDirectory(paths.VideoPath);
Directory.CreateDirectory(paths.ChatJsonPath);
Directory.CreateDirectory(paths.ChatMp4Path);
Directory.CreateDirectory(paths.MetadataPath);
}
public string GetUniqueFilePath(string path)
{
if (!File.Exists(path)) return path;
var dir = Path.GetDirectoryName(path) ?? string.Empty;
var name = Path.GetFileNameWithoutExtension(path);
var ext = Path.GetExtension(path);
for (int i = 1; i < 10000; i++)
{
var candidate = Path.Combine(dir, $"{name}-{i}{ext}");
if (!File.Exists(candidate)) return candidate;
}
throw new IOException("Could not create unique filename");
}
}
}

View file

@ -0,0 +1,12 @@
using System.Threading;
using System.Threading.Tasks;
using TwitchArchive.Core.Api;
namespace TwitchArchive.Core.Services
{
public interface IDownloaderService
{
Task<bool> DownloadVodAsync(VodInfo vod, string outputPath, CancellationToken ct = default);
Task<bool> DownloadChatJsonAsync(string vodId, string jsonPath, CancellationToken ct = default);
}
}

View file

@ -0,0 +1,19 @@
using System;
using System.Collections.Generic;
namespace TwitchArchive.Core.Services
{
public record OutputLine(Guid JobId, DateTime TimestampUtc, string Line, bool IsError);
public interface IProcessOutputStore
{
void AppendLine(string streamer, OutputLine line);
IReadOnlyList<OutputLine> GetRecentLines(string streamer, Guid jobId, int max = 500);
IReadOnlyList<OutputLine> GetRecentLines(string streamer, int max = 500);
/// <summary>
/// Event raised when a line is appended. Useful for UI subscriptions.
/// </summary>
event Action<string, OutputLine>? LineAppended;
}
}

View file

@ -0,0 +1,31 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public record ProcessOutputLine(DateTime TimestampUtc, string Line, bool IsError);
public class ProcessRunResult
{
public int ExitCode { get; set; }
public List<ProcessOutputLine> Output { get; set; } = new();
}
public class ProcessRunOptions
{
public string FileName { get; init; } = string.Empty;
public string Arguments { get; init; } = string.Empty;
public string? WorkingDirectory { get; init; }
public bool RedirectOutput { get; init; } = true;
// Optional metadata for real-time output forwarding
public string? Streamer { get; init; }
public Guid? JobId { get; init; }
}
public interface IProcessRunner
{
Task<ProcessRunResult> RunAsync(ProcessRunOptions options, CancellationToken cancellationToken = default);
}
}

View file

@ -0,0 +1,11 @@
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public interface IProcessorService
{
Task<bool> ProcessRawStreamAsync(string rawPath, string outputPath, string quality, CancellationToken ct = default);
Task<bool> MergeVideoAndChatAsync(string videoPath, string chatVideoPath, string outputPath, string layout, CancellationToken ct = default);
}
}

View file

@ -0,0 +1,59 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
namespace TwitchArchive.Core.Services
{
public class ProcessOutputStore : IProcessOutputStore
{
// streamer -> jobId -> circular buffer
private readonly ConcurrentDictionary<string, ConcurrentDictionary<Guid, FixedSizedQueue<OutputLine>>> _store = new();
public event Action<string, OutputLine>? LineAppended;
public void AppendLine(string streamer, OutputLine line)
{
var jobs = _store.GetOrAdd(streamer, _ => new ConcurrentDictionary<Guid, FixedSizedQueue<OutputLine>>());
var buf = jobs.GetOrAdd(line.JobId, _ => new FixedSizedQueue<OutputLine>(1000));
buf.Enqueue(line);
try { LineAppended?.Invoke(streamer, line); } catch { }
}
public IReadOnlyList<OutputLine> GetRecentLines(string streamer, Guid jobId, int max = 500)
{
if (!_store.TryGetValue(streamer, out var jobs)) return Array.Empty<OutputLine>();
if (!jobs.TryGetValue(jobId, out var buf)) return Array.Empty<OutputLine>();
var items = buf.ToList();
if (items.Count <= max) return items;
return items.Skip(items.Count - max).ToList();
}
public IReadOnlyList<OutputLine> GetRecentLines(string streamer, int max = 500)
{
if (!_store.TryGetValue(streamer, out var jobs)) return Array.Empty<OutputLine>();
var combined = new List<OutputLine>();
foreach (var kv in jobs)
{
lock (kv.Value)
{
combined.AddRange(kv.Value.ToList());
}
}
combined.Sort((a, b) => a.TimestampUtc.CompareTo(b.TimestampUtc));
if (combined.Count <= max) return combined;
return combined.Skip(combined.Count - max).ToList();
}
private class FixedSizedQueue<T> : Queue<T>
{
private readonly int _maxSize;
public FixedSizedQueue(int maxSize) { _maxSize = maxSize; }
public new void Enqueue(T item)
{
base.Enqueue(item);
while (Count > _maxSize) Dequeue();
}
}
}
}

View file

@ -0,0 +1,101 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public class ProcessRunner : IProcessRunner
{
private readonly IProcessOutputStore? _outputStore;
public ProcessRunner(IProcessOutputStore? outputStore = null)
{
_outputStore = outputStore;
}
public async Task<ProcessRunResult> RunAsync(ProcessRunOptions options, CancellationToken cancellationToken = default)
{
var result = new ProcessRunResult();
using var process = new Process();
process.StartInfo.FileName = options.FileName;
process.StartInfo.Arguments = options.Arguments;
process.StartInfo.UseShellExecute = false;
process.StartInfo.CreateNoWindow = true;
if (options.RedirectOutput)
{
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
}
var outputLines = new List<ProcessOutputLine>();
if (options.RedirectOutput)
{
process.OutputDataReceived += (s, e) =>
{
if (e.Data == null) return;
var item = new ProcessOutputLine(DateTime.UtcNow, e.Data, false);
lock (outputLines) { outputLines.Add(item); }
// forward to store if available
if (_outputStore != null && options.Streamer != null && options.JobId.HasValue)
{
try { _outputStore.AppendLine(options.Streamer, new OutputLine(options.JobId.Value, item.TimestampUtc, item.Line, item.IsError)); } catch { }
}
};
process.ErrorDataReceived += (s, e) =>
{
if (e.Data == null) return;
var item = new ProcessOutputLine(DateTime.UtcNow, e.Data, true);
lock (outputLines) { outputLines.Add(item); }
if (_outputStore != null && options.Streamer != null && options.JobId.HasValue)
{
try { _outputStore.AppendLine(options.Streamer, new OutputLine(options.JobId.Value, item.TimestampUtc, item.Line, item.IsError)); } catch { }
}
};
}
using (cancellationToken.Register(() =>
{
try { if (!process.HasExited) process.Kill(); } catch { }
}))
{
try
{
if (!process.Start())
{
result.ExitCode = -1;
return result;
}
if (options.RedirectOutput)
{
process.BeginOutputReadLine();
process.BeginErrorReadLine();
}
await Task.Run(() => process.WaitForExit(), cancellationToken).ConfigureAwait(false);
result.ExitCode = process.ExitCode;
}
catch (OperationCanceledException)
{
result.ExitCode = -1;
}
catch (Exception)
{
result.ExitCode = -1;
}
}
lock (outputLines)
{
result.Output = outputLines;
}
return result;
}
}
}

View file

@ -0,0 +1,63 @@
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace TwitchArchive.Core.Services
{
public class ProcessorService : IProcessorService
{
private readonly IProcessRunner _runner;
public ProcessorService(IProcessRunner runner)
{
_runner = runner;
}
public async Task<bool> ProcessRawStreamAsync(string rawPath, string outputPath, string quality, CancellationToken ct = default)
{
if (!File.Exists(rawPath)) return false;
try
{
Directory.CreateDirectory(Path.GetDirectoryName(outputPath) ?? ".");
// If audio-only
if (quality == "audio_only")
{
var args = $"-i \"{rawPath}\" -vn -c:a aac -ar 48000 -ac 2 -b:a 192k \"{outputPath}\"";
var res = await _runner.RunAsync(new ProcessRunOptions { FileName = "ffmpeg", Arguments = args, RedirectOutput = true }, ct).ConfigureAwait(false);
return res.ExitCode == 0;
}
// default: copy streams into mp4
var args2 = $"-y -i \"{rawPath}\" -c:v copy -c:a copy -movflags +faststart \"{outputPath}\"";
var r2 = await _runner.RunAsync(new ProcessRunOptions { FileName = "ffmpeg", Arguments = args2, RedirectOutput = true }, ct).ConfigureAwait(false);
return r2.ExitCode == 0;
}
catch
{
return false;
}
}
public async Task<bool> MergeVideoAndChatAsync(string videoPath, string chatVideoPath, string outputPath, string layout, CancellationToken ct = default)
{
try
{
Directory.CreateDirectory(Path.GetDirectoryName(outputPath) ?? ".");
// Simple side-by-side using ffmpeg hstack (assumes same height)
if (layout == "side-by-side")
{
var args = $"-y -i \"{videoPath}\" -i \"{chatVideoPath}\" -filter_complex \"[0:v]scale=iw:ih[v0];[1:v]scale=iw:ih[v1];[v0][v1]hstack=inputs=2[v]\" -map \"[v]\" -map 0:a? -c:v libx264 -crf 23 -preset veryfast \"{outputPath}\"";
var res = await _runner.RunAsync(new ProcessRunOptions { FileName = "ffmpeg", Arguments = args, RedirectOutput = true }, ct).ConfigureAwait(false);
return res.ExitCode == 0;
}
// overlay or other layouts can be added later
return false;
}
catch
{
return false;
}
}
}
}

View file

@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.3" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.3">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="10.0.3" />
<PackageReference Include="Polly" Version="8.6.5" />
<PackageReference Include="NLog" Version="6.1.0" />
<PackageReference Include="NLog.Extensions.Logging" Version="6.1.1" />
</ItemGroup>
</Project>

View file

@ -0,0 +1,178 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using System.IO;
using TwitchArchive.Core.Monitoring;
using Microsoft.Extensions.DependencyInjection;
using TwitchArchive.Core.Persistence.Models;
using TwitchArchive.Core.Recovery;
using TwitchArchive.Core.Services;
namespace TwitchArchive.Core.Workers
{
/// <summary>
/// Monitors a single streamer using a RecoveryPolicy and invokes the process runner when needed.
/// This is a lightweight, testable worker (not derived from BackgroundService).
/// </summary>
public class StreamWorker
{
private readonly string _username;
private readonly ILiveChecker _liveChecker;
private readonly IProcessRunner _processRunner;
private readonly IProcessOutputStore _outputStore;
private readonly IDownloaderService _downloader;
private readonly IProcessorService _processor;
private readonly FileManagerService _fileManager;
private readonly TwitchArchive.Core.Api.ITwitchApiClient _apiClient;
private readonly TwitchArchive.Core.Persistence.ISessionRepository? _sessionRepo;
private readonly IServiceScope? _scope;
private readonly RecoveryPolicy _policy;
private CancellationTokenSource? _cts;
private Task? _loopTask;
public StreamWorker(string username, ILiveChecker liveChecker, IProcessRunner processRunner, IProcessOutputStore outputStore, IDownloaderService downloader, IProcessorService processor, FileManagerService fileManager, TwitchArchive.Core.Api.ITwitchApiClient apiClient, TwitchArchive.Core.Persistence.ISessionRepository? sessionRepo = null, IServiceScope? scope = null, TimeSpan? refreshInterval = null)
{
_username = username;
_liveChecker = liveChecker;
_processRunner = processRunner;
_outputStore = outputStore;
_downloader = downloader;
_processor = processor;
_fileManager = fileManager;
_apiClient = apiClient;
_sessionRepo = sessionRepo;
_scope = scope;
_policy = new RecoveryPolicy(refreshInterval);
}
public void Start()
{
if (_cts != null) return;
_cts = new CancellationTokenSource();
_loopTask = Task.Run(() => MonitorLoopAsync(_cts.Token));
}
public async Task StopAsync()
{
if (_cts == null) return;
_cts.Cancel();
try { if (_loopTask != null) await _loopTask.ConfigureAwait(false); } catch { }
_cts.Dispose();
_cts = null;
_loopTask = null;
try { _scope?.Dispose(); } catch { }
}
private async Task MonitorLoopAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
bool? isLive = null;
bool networkError = false;
try
{
isLive = await _liveChecker.IsLiveAsync(_username, ct).ConfigureAwait(false);
}
catch (OperationCanceledException) { break; }
catch (Exception)
{
// treat as network error — let policy handle backoff
networkError = true;
}
var decision = _policy.Tick(DateTime.UtcNow, isLive, networkError);
if (decision.Action == RecoveryAction.StartRecording)
{
// Build output path and ensure directories
var paths = _fileManager.GetPaths(_username);
_fileManager.EnsureDirectories(paths);
var timestamp = DateTime.UtcNow.ToString("yyyyMMdd_H\'h\'mm\'m\'ss\'");
var filenameBase = timestamp;
var rawFile = Path.Combine(paths.RawPath, $"LIVE_{filenameBase}.ts");
rawFile = _fileManager.GetUniqueFilePath(rawFile);
var jobId = Guid.NewGuid();
// persist session and recording job (if repository available)
StreamSession? session = null;
ArchiveJob? recJob = null;
if (_sessionRepo != null)
{
try
{
session = await _sessionRepo.CreateSessionAsync(_username, string.Empty, DateTime.UtcNow, ct).ConfigureAwait(false);
recJob = await _sessionRepo.CreateJobAsync(session.Id, "Recording", DateTime.UtcNow, ct).ConfigureAwait(false);
}
catch { }
}
var quality = "best"; // TODO: read from config
var args = $"twitch.tv/{_username} {quality} --hls-live-restart --retry-streams 30 --force -o \"{rawFile}\"";
var options = new ProcessRunOptions { FileName = "streamlink", Arguments = args, Streamer = _username, JobId = jobId };
try
{
var res = await _processRunner.RunAsync(options, ct).ConfigureAwait(false);
if (recJob != null)
{
recJob.Status = res.ExitCode == 0 ? "Completed" : "Failed";
recJob.CompletedAt = DateTime.UtcNow;
recJob.FilePath = rawFile;
try { await _sessionRepo!.UpdateJobAsync(recJob, ct).ConfigureAwait(false); } catch { }
}
// recording finished; now trigger processing
var procOutput = Path.Combine(paths.VideoPath, $"LIVE_{filenameBase}.mp4");
// create processing job
ArchiveJob? procJob = null;
if (_sessionRepo != null && session != null)
{
try { procJob = await _sessionRepo.CreateJobAsync(session.Id, "Processing", DateTime.UtcNow, ct).ConfigureAwait(false); } catch { }
}
await _processor.ProcessRawStreamAsync(rawFile, procOutput, quality, ct).ConfigureAwait(false);
if (procJob != null)
{
procJob.Status = "Completed";
procJob.CompletedAt = DateTime.UtcNow;
procJob.FilePath = procOutput;
try { await _sessionRepo.UpdateJobAsync(procJob, ct).ConfigureAwait(false); } catch { }
}
// after processing, try to find matching VOD and download it
var vod = await _apiClient.GetLatestVodAsync(_username, ct).ConfigureAwait(false);
if (vod != null)
{
var vodPath = Path.Combine(paths.VideoPath, $"VOD_{filenameBase}.mp4");
ArchiveJob? vodJob = null;
if (_sessionRepo != null && session != null)
{
try { vodJob = await _sessionRepo.CreateJobAsync(session.Id, "DownloadVOD", DateTime.UtcNow, ct).ConfigureAwait(false); } catch { }
}
await _downloader.DownloadVodAsync(vod, vodPath, ct).ConfigureAwait(false);
if (vodJob != null)
{
vodJob.Status = "Completed";
vodJob.CompletedAt = DateTime.UtcNow;
vodJob.FilePath = vodPath;
try { await _sessionRepo.UpdateJobAsync(vodJob, ct).ConfigureAwait(false); } catch { }
}
}
}
catch { }
}
else if (decision.Action == RecoveryAction.StartProcessing)
{
// placeholder for processing triggers
}
// sleep for the suggested duration (or break if cancelled)
try
{
await Task.Delay(decision.Sleep, ct).ConfigureAwait(false);
}
catch (TaskCanceledException) { break; }
}
}
}
}

View file

@ -0,0 +1,51 @@
using System;
using Microsoft.Extensions.DependencyInjection;
using System.Collections.Concurrent;
using System.Threading.Tasks;
using TwitchArchive.Core.Monitoring;
using TwitchArchive.Core.Services;
namespace TwitchArchive.Core.Workers
{
public class StreamWorkerManager
{
private readonly IServiceProvider _services;
private readonly ConcurrentDictionary<string, StreamWorker> _workers = new();
public StreamWorkerManager(IServiceProvider services)
{
_services = services;
}
public void StartWorker(string username)
{
var worker = _workers.GetOrAdd(username, u =>
{
var scope = _services.CreateScope();
var liveChecker = scope.ServiceProvider.GetRequiredService<ILiveChecker>();
var processRunner = scope.ServiceProvider.GetRequiredService<IProcessRunner>();
var outputStore = scope.ServiceProvider.GetRequiredService<IProcessOutputStore>();
var downloader = scope.ServiceProvider.GetRequiredService<IDownloaderService>();
var processor = scope.ServiceProvider.GetRequiredService<IProcessorService>();
var fileManager = scope.ServiceProvider.GetRequiredService<FileManagerService>();
var api = scope.ServiceProvider.GetRequiredService<TwitchArchive.Core.Api.ITwitchApiClient>();
var sessionRepo = scope.ServiceProvider.GetService<TwitchArchive.Core.Persistence.ISessionRepository>();
var w = new StreamWorker(u, liveChecker, processRunner, outputStore, downloader, processor, fileManager, api, sessionRepo, scope);
return w;
});
worker.Start();
}
public async Task StopWorkerAsync(string username)
{
if (_workers.TryRemove(username, out var worker))
{
await worker.StopAsync().ConfigureAwait(false);
}
}
public bool IsRunning(string username) => _workers.ContainsKey(username);
}
}

View file

@ -0,0 +1,39 @@
using System;
using System.IO;
using Xunit;
using TwitchArchive.Core.Config;
namespace TwitchArchive.Tests
{
public class ConfigurationServiceTests : IDisposable
{
private readonly string _tmp;
public ConfigurationServiceTests()
{
_tmp = Path.Combine(Path.GetTempPath(), "ta_test_" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(_tmp);
}
[Fact]
public void SaveAndLoadGlobalAndStreamer_Roundtrip()
{
var svc = new ConfigurationService(_tmp);
var g = new GlobalConfig { ArchiveRoot = "x:\\archive", DefaultQuality = "best" };
svc.SaveGlobal(g);
var loaded = svc.LoadGlobal();
Assert.Equal(g.ArchiveRoot, loaded.ArchiveRoot);
var s = new StreamerConfig { Username = "alice", Enabled = true, Quality = "720p" };
svc.SaveStreamer(s);
var loadedS = svc.LoadStreamer("alice");
Assert.NotNull(loadedS);
Assert.Equal("720p", loadedS!.Quality);
}
public void Dispose()
{
try { Directory.Delete(_tmp, true); } catch { }
}
}
}

View file

@ -0,0 +1,55 @@
using System;
using System.IO;
using System.Threading.Tasks;
using Moq;
using Xunit;
using TwitchArchive.Core.Services;
using TwitchArchive.Core.Api;
namespace TwitchArchive.Tests
{
public class DownloaderServiceTests : IDisposable
{
private readonly string _tmp;
public DownloaderServiceTests()
{
_tmp = Path.Combine(Path.GetTempPath(), "ta_dl_" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(_tmp);
}
[Fact]
public async Task DownloadVod_CallsRunner_WithExpectedArgs()
{
var mock = new Mock<IProcessRunner>();
mock.Setup(r => r.RunAsync(It.IsAny<ProcessRunOptions>(), default)).ReturnsAsync(new ProcessRunResult { ExitCode = 0 });
var svc = new DownloaderService(mock.Object);
var vod = new VodInfo("v123", "title", "2020-01-01T00:00:00Z");
var outp = Path.Combine(_tmp, "vod.mp4");
var ok = await svc.DownloadVodAsync(vod, outp);
Assert.True(ok);
mock.Verify(r => r.RunAsync(It.Is<ProcessRunOptions>(o => o.FileName.Contains("TwitchDownloader") && o.Arguments.Contains("-u") && o.Arguments.Contains("-o \"" + outp + "\"")), default), Times.Once);
}
[Fact]
public async Task DownloadChatJson_CreatesFileAndReturnsTrue()
{
var mock = new Mock<IProcessRunner>();
mock.Setup(r => r.RunAsync(It.IsAny<ProcessRunOptions>(), default)).ReturnsAsync(new ProcessRunResult { ExitCode = 0 });
var svc = new DownloaderService(mock.Object);
var jsonPath = Path.Combine(_tmp, "chat.json");
// ensure file exists to satisfy Post-run check
File.WriteAllText(jsonPath, "{}");
var ok = await svc.DownloadChatJsonAsync("v123", jsonPath);
Assert.True(ok);
mock.Verify(r => r.RunAsync(It.Is<ProcessRunOptions>(o => o.FileName.Contains("TwitchDownloader") && o.Arguments.Contains("chatdownload")), default), Times.Once);
}
public void Dispose()
{
try { Directory.Delete(_tmp, true); } catch { }
}
}
}

View file

@ -0,0 +1,38 @@
using Xunit;
using TwitchArchive.Core.Config;
namespace TwitchArchive.Tests
{
public class EffectiveConfigDefaultsTests
{
[Fact]
public void Merge_AppliesStreamerOverridesOverGlobalDefaults()
{
var global = new GlobalConfig
{
DefaultQuality = "best",
Defaults = new DefaultsSection
{
DownloadVOD = true,
MergeVideoChat = false,
VodTimeout = 300
}
};
var streamer = new StreamerConfig
{
Username = "test",
DownloadVOD = false,
MergeVideoChat = true,
VodTimeout = 10
};
var eff = EffectiveConfig.Merge(global, streamer);
Assert.False(eff.Defaults.DownloadVOD);
Assert.True(eff.Defaults.MergeVideoChat);
Assert.Equal(10, eff.Defaults.VodTimeout);
Assert.Equal("best", eff.DefaultQuality);
}
}
}

View file

@ -0,0 +1,27 @@
using Xunit;
using TwitchArchive.Core.Config;
namespace TwitchArchive.Tests
{
public class EffectiveConfigTests
{
[Fact]
public void StreamerOverrideWins_WhenPresent()
{
var global = new GlobalConfig { DefaultQuality = "best", UploadToCloud = false };
var streamer = new StreamerConfig { Username = "bob", Quality = "480p", UploadToCloud = true };
var eff = EffectiveConfig.Merge(global, streamer);
Assert.Equal("480p", eff.DefaultQuality);
Assert.True(eff.UploadToCloud);
}
[Fact]
public void GlobalUsed_WhenStreamerNull()
{
var global = new GlobalConfig { DefaultQuality = "best", UploadToCloud = false };
var eff = EffectiveConfig.Merge(global, null);
Assert.Equal("best", eff.DefaultQuality);
Assert.False(eff.UploadToCloud);
}
}
}

View file

@ -0,0 +1,38 @@
using System;
using System.IO;
using Xunit;
using TwitchArchive.Core.Services;
namespace TwitchArchive.Tests
{
public class FileManagerServiceTests : IDisposable
{
private readonly string _root;
public FileManagerServiceTests()
{
_root = Path.Combine(Path.GetTempPath(), "ta_fm_" + Guid.NewGuid().ToString("N"));
}
[Fact]
public void EnsureDirectories_CreatesPaths_And_GetUniquePath_AppendsSuffix()
{
var svc = new FileManagerService(_root);
var paths = svc.GetPaths("alice");
svc.EnsureDirectories(paths);
Assert.True(Directory.Exists(paths.RawPath));
Assert.True(Directory.Exists(paths.VideoPath));
var file = Path.Combine(paths.RawPath, "file.ts");
File.WriteAllText(file, "x");
var unique = svc.GetUniqueFilePath(file);
Assert.NotEqual(file, unique);
Assert.Contains("file-", Path.GetFileName(unique));
}
public void Dispose()
{
try { Directory.Delete(_root, true); } catch { }
}
}
}

View file

@ -0,0 +1,57 @@
using System;
using System.IO;
using System.Threading.Tasks;
using Moq;
using Xunit;
using TwitchArchive.Core.Services;
namespace TwitchArchive.Tests
{
public class ProcessorServiceTests : IDisposable
{
private readonly string _tmp;
public ProcessorServiceTests()
{
_tmp = Path.Combine(Path.GetTempPath(), "ta_proc_" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(_tmp);
}
[Fact]
public async Task ProcessRawStream_CallsFfmpeg_CopyMode()
{
var mock = new Mock<IProcessRunner>();
mock.Setup(r => r.RunAsync(It.IsAny<ProcessRunOptions>(), default)).ReturnsAsync(new ProcessRunResult { ExitCode = 0 });
var svc = new ProcessorService(mock.Object);
var raw = Path.Combine(_tmp, "in.ts");
var outp = Path.Combine(_tmp, "out.mp4");
File.WriteAllText(raw, "x");
var ok = await svc.ProcessRawStreamAsync(raw, outp, "best");
Assert.True(ok);
mock.Verify(r => r.RunAsync(It.Is<ProcessRunOptions>(o => o.FileName == "ffmpeg" && o.Arguments.Contains("-c:v copy")), default), Times.Once);
}
[Fact]
public async Task ProcessRawStream_AudioOnly_UsesAac()
{
var mock = new Mock<IProcessRunner>();
mock.Setup(r => r.RunAsync(It.IsAny<ProcessRunOptions>(), default)).ReturnsAsync(new ProcessRunResult { ExitCode = 0 });
var svc = new ProcessorService(mock.Object);
var raw = Path.Combine(_tmp, "in.ts");
var outp = Path.Combine(_tmp, "out.mp4");
File.WriteAllText(raw, "x");
var ok = await svc.ProcessRawStreamAsync(raw, outp, "audio_only");
Assert.True(ok);
mock.Verify(r => r.RunAsync(It.Is<ProcessRunOptions>(o => o.FileName == "ffmpeg" && o.Arguments.Contains("-vn") && o.Arguments.Contains("-c:a aac")), default), Times.Once);
}
public void Dispose()
{
try { Directory.Delete(_tmp, true); } catch { }
}
}
}

View file

@ -0,0 +1,86 @@
using System;
using TwitchArchive.Core.Recovery;
using Xunit;
namespace TwitchArchive.Tests
{
public class RecoveryPolicyTests
{
[Fact]
public void Monitoring_WhenNotLive_SleepsRefresh()
{
var policy = new RecoveryPolicy(TimeSpan.FromSeconds(60));
var now = DateTime.UtcNow;
var decision = policy.Tick(now, isLive: false, networkError: false);
Assert.Equal(RecoveryAction.SleepOnly, decision.Action);
Assert.Equal(TimeSpan.FromSeconds(60), decision.Sleep);
Assert.Equal(RecoveryState.Monitoring, policy.CurrentState);
}
[Fact]
public void WhenLive_StartsRecording()
{
var policy = new RecoveryPolicy(TimeSpan.FromSeconds(60));
var now = DateTime.UtcNow;
var decision = policy.Tick(now, isLive: true, networkError: false);
Assert.Equal(RecoveryAction.StartRecording, decision.Action);
Assert.Equal(TimeSpan.Zero, decision.Sleep);
Assert.Equal(RecoveryState.Recording, policy.CurrentState);
}
[Fact]
public void RecordingToFastReconnect_ThenToSlowReconnect()
{
var policy = new RecoveryPolicy(TimeSpan.FromSeconds(60));
var t0 = DateTime.UtcNow;
// Start recording
var d1 = policy.Tick(t0, isLive: true, networkError: false);
Assert.Equal(RecoveryAction.StartRecording, d1.Action);
Assert.Equal(RecoveryState.Recording, policy.CurrentState);
// Recording ended -> fast reconnect begins
var t1 = t0.AddMinutes(10); // time advanced
var d2 = policy.Tick(t1, isLive: false, networkError: false);
Assert.Equal(RecoveryAction.SleepOnly, d2.Action);
Assert.Equal(TimeSpan.FromSeconds(10), d2.Sleep);
Assert.Equal(RecoveryState.FastReconnect, policy.CurrentState);
// Still within fast reconnect window -> continue fast polling
var t2 = t1.AddSeconds(30);
var d3 = policy.Tick(t2, isLive: false, networkError: false);
Assert.Equal(RecoveryAction.SleepOnly, d3.Action);
Assert.Equal(TimeSpan.FromSeconds(10), d3.Sleep);
Assert.Equal(RecoveryState.FastReconnect, policy.CurrentState);
// After fast window expires -> move to slow reconnect and start processing
var t3 = t1.AddMinutes(3); // beyond 2-minute window
var d4 = policy.Tick(t3, isLive: false, networkError: false);
Assert.Equal(RecoveryAction.StartProcessing, d4.Action);
Assert.Equal(TimeSpan.FromSeconds(60), d4.Sleep);
Assert.Equal(RecoveryState.SlowReconnect, policy.CurrentState);
}
[Fact]
public void NetworkFault_Backoff_IncreasesAndRecovers()
{
var policy = new RecoveryPolicy(TimeSpan.FromSeconds(60));
var now = DateTime.UtcNow;
var d1 = policy.Tick(now, isLive: false, networkError: true);
Assert.Equal(RecoveryAction.SleepOnly, d1.Action);
Assert.Equal(TimeSpan.FromSeconds(30), d1.Sleep);
Assert.Equal(RecoveryState.NetworkFault, policy.CurrentState);
var d2 = policy.Tick(now.AddSeconds(1), isLive: false, networkError: true);
Assert.Equal(RecoveryAction.SleepOnly, d2.Action);
Assert.True(d2.Sleep >= TimeSpan.FromSeconds(30));
// Recover
var d3 = policy.Tick(now.AddMinutes(1), isLive: false, networkError: false);
Assert.Equal(RecoveryAction.SleepOnly, d3.Action);
Assert.Equal(TimeSpan.FromSeconds(60), d3.Sleep);
Assert.Equal(RecoveryState.Monitoring, policy.CurrentState);
}
}
}

View file

@ -0,0 +1,49 @@
using System;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using TwitchArchive.Core.Persistence;
using TwitchArchive.Core.Persistence.Models;
using Xunit;
namespace TwitchArchive.Tests
{
public class SessionRepositoryTests
{
private ArchiveDbContext CreateContext()
{
var opts = new DbContextOptionsBuilder<ArchiveDbContext>()
.UseInMemoryDatabase("ta_test_db_" + Guid.NewGuid().ToString("N"))
.Options;
return new ArchiveDbContext(opts);
}
private class InMemoryFactory : IDbContextFactory<ArchiveDbContext>
{
private readonly DbContextOptions<ArchiveDbContext> _opts;
public InMemoryFactory(DbContextOptions<ArchiveDbContext> opts) { _opts = opts; }
public ArchiveDbContext CreateDbContext() => new ArchiveDbContext(_opts);
}
[Fact]
public async Task AddSessionAndJob_AndQuery()
{
var opts = new DbContextOptionsBuilder<ArchiveDbContext>()
.UseInMemoryDatabase("ta_test_db_" + Guid.NewGuid().ToString("N"))
.Options;
var factory = new InMemoryFactory(opts);
var repo = new SessionRepository(factory);
var session = await repo.CreateSessionAsync("charlie", "", DateTime.UtcNow);
Assert.NotNull(session);
var job = await repo.CreateJobAsync(session.Id, "Recording", DateTime.UtcNow);
Assert.NotNull(job);
var recent = await repo.GetRecentSessionsAsync(10);
Assert.Contains(recent, s => s.Id == session.Id);
job.Status = "Completed";
await repo.UpdateJobAsync(job);
var jobs = await repo.GetJobsForSessionAsync(session.Id);
Assert.Contains(jobs, j => j.Status == "Completed");
}
}
}

View file

@ -0,0 +1,67 @@
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Threading;
using System.Threading.Tasks;
using System.Text.Json;
using Xunit;
using TwitchArchive.Core.Api;
namespace TwitchArchive.Tests
{
class FakeHandler : HttpMessageHandler
{
private readonly Func<HttpRequestMessage, HttpResponseMessage> _responder;
public FakeHandler(Func<HttpRequestMessage, HttpResponseMessage> responder) => _responder = responder;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
=> Task.FromResult(_responder(request));
}
public class TwitchApiClientTests
{
[Fact]
public async Task GetOauthToken_CachesAndReturnsToken()
{
Environment.SetEnvironmentVariable("CLIENT-ID", "x");
Environment.SetEnvironmentVariable("CLIENT-SECRET", "y");
var handler = new FakeHandler(req =>
{
var obj = JsonSerializer.Serialize(new { access_token = "tok123", expires_in = 3600 });
return new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(obj) };
});
var client = new HttpClient(handler);
var api = new TwitchApiClient(client);
var t1 = await api.GetOauthTokenAsync();
Assert.Equal("tok123", t1);
var t2 = await api.GetOauthTokenAsync();
Assert.Equal("tok123", t2);
}
[Fact]
public async Task GetStreamStatus_ReturnsInfo_WhenLive()
{
var gqlResp = JsonSerializer.Serialize(new { data = new { user = new { stream = new { title = "hi", createdAt = "2020-01-01T00:00:00Z", archiveVideo = (object?)null } } } });
var handler = new FakeHandler(req => new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(gqlResp) });
var client = new HttpClient(handler);
var api = new TwitchApiClient(client);
var info = await api.GetStreamStatusAsync("alice");
Assert.NotNull(info);
Assert.Equal("hi", info!.Title);
}
[Fact]
public async Task GetLatestVod_ReturnsVod_WhenPresent()
{
var vodJson = JsonSerializer.Serialize(new { data = new { user = new { videos = new { edges = new[] { new { node = new { id = "v123", title = "vod", recordedAt = "2020-01-01T00:00:00Z" } } } } } } });
var handler = new FakeHandler(req => new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(vodJson) });
var client = new HttpClient(handler);
var api = new TwitchApiClient(client);
var vod = await api.GetLatestVodAsync("bob");
Assert.NotNull(vod);
Assert.Equal("v123", vod!.Id);
}
}
}

View file

@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<IsPackable>false</IsPackable>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="10.0.3" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="Moq" Version="4.20.72" />
<PackageReference Include="Microsoft.EntityFrameworkCore.InMemory" Version="8.0.0" />
<PackageReference Include="coverlet.collector" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\TwitchArchive.Core\TwitchArchive.Core.csproj" />
</ItemGroup>
</Project>

View file

@ -0,0 +1,14 @@
@using Microsoft.AspNetCore.Components.Routing
@using Microsoft.AspNetCore.Components.Authorization
@using TwitchArchive.Web.Shared
<CascadingAuthenticationState>
<Router AppAssembly="typeof(Program).Assembly">
<Found Context="routeData">
<AuthorizeRouteView RouteData="routeData" DefaultLayout="typeof(MainLayout)" />
</Found>
<NotFound>
<p>Sorry, there's nothing at this address.</p>
</NotFound>
</Router>
</CascadingAuthenticationState>

View file

@ -0,0 +1,20 @@
using Microsoft.AspNetCore.SignalR;
using System.Threading.Tasks;
namespace TwitchArchive.Web.Hubs
{
public class ProcessOutputHub : Hub
{
// methods are server-driven; clients listen on ReceiveLine
public Task JoinStreamerGroup(string streamer)
{
return Groups.AddToGroupAsync(Context.ConnectionId, streamer);
}
public Task LeaveStreamerGroup(string streamer)
{
return Groups.RemoveFromGroupAsync(Context.ConnectionId, streamer);
}
}
}

View file

@ -0,0 +1,208 @@
@page "/addstreamer"
@inject TwitchArchive.Core.Config.IConfigurationService ConfigService
@inject NavigationManager Nav
<h3>Add Streamer</h3>
<EditForm Model="model" OnValidSubmit="Save">
<DataAnnotationsValidator />
<ValidationSummary />
<div class="card">
<label title="Streamer username (lowercase)">Username</label>
<InputText @bind-Value="model.Username" title="Streamer username (lowercase)" />
</div>
<div class="card">
<label title="Enable/disable monitoring for this streamer">Enabled</label>
<InputCheckbox @bind-Value="model.Enabled" title="Enable/disable monitoring for this streamer" />
</div>
<div class="card">
<label title="Optional quality override (e.g., best, 720p)">Quality (optional)</label>
<InputText @bind-Value="model.Quality" placeholder="leave empty to use default" title="Optional quality override (e.g., best, 720p)" />
</div>
<div class="card">
<label title="Optional cloud upload destination (rclone remote)">Upload Destination (optional)</label>
<InputText @bind-Value="model.UploadDestination" title="Optional cloud upload destination (rclone remote)" />
</div>
<div>
<button class="btn-link" @onclick="ToggleAdvanced">@(showAdvanced ? "Hide advanced" : "Show advanced")</button>
</div>
@if (showAdvanced)
{
<div class="card">
<h4>Advanced per-streamer defaults</h4>
<div>
<label title="If enabled, VODs will be downloaded for this streamer">Download VOD</label>
<InputCheckbox @bind="downloadVOD" title="If enabled, VODs will be downloaded for this streamer" />
</div>
<div>
<label title="If enabled, chat will be downloaded for VODs">Download CHAT</label>
<InputCheckbox @bind="downloadCHAT" title="If enabled, chat will be downloaded for VODs" />
</div>
<div>
<label title="If enabled, live chat will be captured while streaming">Download Live CHAT</label>
<InputCheckbox @bind="downloadLiveCHAT" title="If enabled, live chat will be captured while streaming" />
</div>
<div>
<label title="Combine video and chat into a merged output">Merge Video & Chat</label>
<InputCheckbox @bind="mergeVideoChat" title="Combine video and chat into a merged output" />
</div>
<div>
<label title="Layout used when merging chat with video">Merge Chat Layout</label>
<InputSelect @bind-Value="model.MergeChatLayout" title="Layout used when merging chat with video">
<option value="side-by-side">Side by side</option>
<option value="stacked">Stacked</option>
<option value="overlay">Overlay</option>
</InputSelect>
</div>
<div>
<label title="Time in seconds to wait for VOD completion before timing out">VOD Timeout (sec)</label>
<InputNumber @bind-Value="vodTimeout" title="Time in seconds to wait for VOD completion before timing out" />
</div>
<div>
<label title="Upload the raw pre-merged video to cloud">Upload Pre-Merge Video</label>
<InputCheckbox @bind="uploadPreMergeVideo" title="Upload the raw pre-merged video to cloud" />
</div>
<div>
<label title="Upload the merged video to cloud">Upload Merged Video</label>
<InputCheckbox @bind="uploadMergedVideo" title="Upload the merged video to cloud" />
</div>
<div>
<label title="Upload a video composed from chat only">Upload Chat Video</label>
<InputCheckbox @bind="uploadChatVideo" title="Upload a video composed from chat only" />
</div>
<div>
<label title="Delete local files after successful upload">Delete Files</label>
<InputCheckbox @bind="deleteFiles" title="Delete local files after successful upload" />
</div>
<div>
<label title="Keep only raw recordings and skip processed outputs">Only Raw</label>
<InputCheckbox @bind="onlyRaw" title="Keep only raw recordings and skip processed outputs" />
</div>
<div>
<label title="Remove temporary raw files after processing">Clean Raw</label>
<InputCheckbox @bind="cleanRaw" title="Remove temporary raw files after processing" />
</div>
<div>
<label title="Number of HLS segments to keep for live streams">HLS segments (live)</label>
<InputNumber @bind-Value="hlsSegments" title="Number of HLS segments to keep for live streams" />
</div>
<div>
<label title="Number of HLS segments to use when producing VOD HLS">HLS segments (VOD)</label>
<InputNumber @bind-Value="hlsSegmentsVOD" title="Number of HLS segments to use when producing VOD HLS" />
</div>
<div>
<label title="Enable legacy ttvlol streamlink behavior">Streamlink ttvlol</label>
<InputCheckbox @bind="streamlinkTtvlol" title="Enable legacy ttvlol streamlink behavior" />
</div>
<div>
<label title="Hardware acceleration setting for FFmpeg">FFmpeg HW Accel</label>
<InputSelect @bind-Value="model.FfmpegHwaccel" title="Hardware acceleration setting for FFmpeg">
<option value="auto">Auto</option>
<option value="none">None</option>
<option value="vaapi">VAAPI</option>
<option value="dxva2">DXVA2</option>
<option value="qsv">QSV</option>
<option value="cuda">CUDA</option>
</InputSelect>
</div>
<div>
<label title="Threads supplied to ffmpeg (0 = auto)">FFmpeg Threads</label>
<InputNumber @bind-Value="ffmpegThreads" title="Threads supplied to ffmpeg (0 = auto)" />
</div>
<div>
<label title="Audio codec used by FFmpeg (e.g., aac)">FFmpeg Audio Codec</label>
<InputText @bind-Value="ffmpegAudioCodec" title="Audio codec used by FFmpeg (e.g., aac)" />
</div>
<div>
<label title="Audio sample rate for FFmpeg">FFmpeg Audio Sample Rate</label>
<InputNumber @bind-Value="ffmpegAudioSamplerate" title="Audio sample rate for FFmpeg" />
</div>
<div>
<label title="Audio bitrate (e.g., 192k)">FFmpeg Audio Bitrate</label>
<InputText @bind-Value="ffmpegAudioBitrate" title="Audio bitrate (e.g., 192k)" />
</div>
<div>
<label title="Enable FFmpeg error recovery options">FFmpeg Error Recovery</label>
<InputCheckbox @bind="ffmpegErrorRecovery" title="Enable FFmpeg error recovery options" />
</div>
<div>
<label title="Enable faststart flag for MP4 to allow streaming">FFmpeg Faststart</label>
<InputCheckbox @bind="ffmpegFaststart" title="Enable faststart flag for MP4 to allow streaming" />
</div>
<div>
<label title="Emit FFmpeg progress updates">FFmpeg Progress</label>
<InputCheckbox @bind="ffmpegProgress" title="Emit FFmpeg progress updates" />
</div>
</div>
}
<div class="mt-2">
<button type="submit">Create</button>
</div>
</EditForm>
@code {
private TwitchArchive.Core.Config.StreamerConfig model = new() { Enabled = true };
private bool showAdvanced = false;
// local fields for binding nullable/global override values
private bool downloadVOD = true;
private bool downloadCHAT = true;
private bool downloadLiveCHAT = true;
private bool mergeVideoChat = false;
private int? vodTimeout;
private bool uploadPreMergeVideo = true;
private bool uploadMergedVideo = true;
private bool uploadChatVideo = false;
private bool deleteFiles = false;
private bool onlyRaw = false;
private bool cleanRaw = true;
private int? hlsSegments;
private int? hlsSegmentsVOD;
private bool streamlinkTtvlol = false;
private string? ffmpegHwaccel;
private int? ffmpegThreads;
private string? ffmpegAudioCodec;
private int? ffmpegAudioSamplerate;
private string? ffmpegAudioBitrate;
private bool ffmpegErrorRecovery = true;
private bool ffmpegFaststart = true;
private bool ffmpegProgress = false;
private void ToggleAdvanced() => showAdvanced = !showAdvanced;
private void Save()
{
model.Username = model.Username?.Trim().ToLowerInvariant() ?? string.Empty;
if (string.IsNullOrWhiteSpace(model.Username)) return;
// map local fields into nullable model properties
model.DownloadVOD = downloadVOD;
model.DownloadCHAT = downloadCHAT;
model.DownloadLiveCHAT = downloadLiveCHAT;
model.MergeVideoChat = mergeVideoChat;
model.VodTimeout = vodTimeout;
model.UploadPreMergeVideo = uploadPreMergeVideo;
model.UploadMergedVideo = uploadMergedVideo;
model.UploadChatVideo = uploadChatVideo;
model.DeleteFiles = deleteFiles;
model.OnlyRaw = onlyRaw;
model.CleanRaw = cleanRaw;
model.HlsSegments = hlsSegments;
model.HlsSegmentsVOD = hlsSegmentsVOD;
model.StreamlinkTtvlol = streamlinkTtvlol;
model.FfmpegHwaccel = ffmpegHwaccel;
model.FfmpegThreads = ffmpegThreads;
model.FfmpegAudioCodec = ffmpegAudioCodec;
model.FfmpegAudioSamplerate = ffmpegAudioSamplerate;
model.FfmpegAudioBitrate = ffmpegAudioBitrate;
model.FfmpegErrorRecovery = ffmpegErrorRecovery;
model.FfmpegFaststart = ffmpegFaststart;
model.FfmpegProgress = ffmpegProgress;
ConfigService.SaveStreamer(model);
Nav.NavigateTo($"/config/{model.Username}");
}
}

View file

@ -0,0 +1,230 @@
@page "/settings"
@using System.Text.Json
@inject TwitchArchive.Core.Config.IConfigurationService ConfigService
@inject TwitchArchive.Web.Services.IAuthService Auth
<h3>Settings</h3>
@if (saved)
{
<div class="alert">Saved.</div>
}
<EditForm Model="globalModel" OnValidSubmit="SaveGlobal">
<div>
<label>Archive Root</label>
<InputText @bind-value="globalModel.ArchiveRoot" />
</div>
<div>
<label>Streamlink Path</label>
<InputText @bind-value="globalModel.StreamlinkPath" />
</div>
<div>
<label>FFmpeg Path</label>
<InputText @bind-value="globalModel.FfmpegPath" />
</div>
<div>
<label>TwitchDownloader Path</label>
<InputText @bind-value="globalModel.TwitchDownloaderPath" />
</div>
<div>
<label>Rclone Path</label>
<InputText @bind-value="globalModel.RclonePath" />
</div>
<div>
<label>Default Quality</label>
<InputText @bind-value="globalModel.DefaultQuality" />
</div>
<div>
<label>Upload To Cloud</label>
<InputCheckbox @bind-Value="globalModel.UploadToCloud" />
</div>
<div>
<label>Upload Destination</label>
<InputText @bind-value="globalModel.UploadDestination" />
</div>
<div>
<label>Refresh Interval (seconds)</label>
<InputNumber @bind-Value="globalModel.RefreshIntervalSeconds" />
</div>
<div>
<label>Stream Segment Threads</label>
<InputNumber @bind-Value="globalModel.StreamSegmentThreads" />
</div>
</EditForm>
<div class="card">
<button class="btn-link" @onclick="ToggleDefaults">@(showDefaults ? "Hide Defaults" : "Show Defaults")</button>
@if (showDefaults)
{
<EditForm Model="globalModel.Defaults" OnValidSubmit="SaveGlobal">
<DataAnnotationsValidator />
<div class="card">
<h4>Defaults</h4>
<div>
<label>Download VOD</label>
<InputCheckbox @bind-Value="globalModel.Defaults.DownloadVOD" />
</div>
<div>
<label>Download CHAT</label>
<InputCheckbox @bind-Value="globalModel.Defaults.DownloadCHAT" />
</div>
<div>
<label>Download Live CHAT</label>
<InputCheckbox @bind-Value="globalModel.Defaults.DownloadLiveCHAT" />
</div>
<div>
<label>Merge Video & Chat</label>
<InputCheckbox @bind-Value="globalModel.Defaults.MergeVideoChat" />
</div>
<div>
<label>Merge Chat Layout</label>
<InputSelect @bind-Value="globalModel.Defaults.MergeChatLayout">
<option value="side-by-side">Side by side</option>
<option value="stacked">Stacked</option>
<option value="overlay">Overlay</option>
</InputSelect>
</div>
<div>
<label>VOD Timeout (sec)</label>
<InputNumber @bind-Value="globalModel.Defaults.VodTimeout" />
</div>
<div>
<label>Upload to Cloud</label>
<InputCheckbox @bind-Value="globalModel.Defaults.UploadCloud" />
</div>
<div>
<label>Delete Files After Upload</label>
<InputCheckbox @bind-Value="globalModel.Defaults.DeleteFiles" />
</div>
<div>
<label>Only Raw</label>
<InputCheckbox @bind-Value="globalModel.Defaults.OnlyRaw" />
</div>
<div>
<label>Clean Raw</label>
<InputCheckbox @bind-Value="globalModel.Defaults.CleanRaw" />
</div>
<div>
<label>HLS segments (live)</label>
<InputNumber @bind-Value="globalModel.Defaults.HlsSegments" />
</div>
<div>
<label>HLS segments (VOD)</label>
<InputNumber @bind-Value="globalModel.Defaults.HlsSegmentsVOD" />
</div>
<div>
<label>Streamlink ttvlol</label>
<InputCheckbox @bind-Value="globalModel.Defaults.StreamlinkTtvlol" />
</div>
<div>
<label>FFmpeg HW Accel</label>
<InputSelect @bind-Value="globalModel.Defaults.FfmpegHwaccel">
<option value="auto">Auto</option>
<option value="none">None</option>
<option value="vaapi">VAAPI</option>
<option value="dxva2">DXVA2</option>
<option value="qsv">QSV</option>
<option value="cuda">CUDA</option>
</InputSelect>
</div>
<div>
<label>FFmpeg Threads</label>
<InputNumber @bind-Value="globalModel.Defaults.FfmpegThreads" />
</div>
<div>
<label>FFmpeg Audio Codec</label>
<InputText @bind-Value="globalModel.Defaults.FfmpegAudioCodec" />
</div>
<div>
<label>FFmpeg Audio Sample Rate</label>
<InputNumber @bind-Value="globalModel.Defaults.FfmpegAudioSamplerate" />
</div>
<div>
<label>FFmpeg Audio Bitrate</label>
<InputText @bind-Value="globalModel.Defaults.FfmpegAudioBitrate" />
</div>
<div>
<label>FFmpeg Error Recovery</label>
<InputCheckbox @bind-Value="globalModel.Defaults.FfmpegErrorRecovery" />
</div>
<div>
<label>FFmpeg Faststart</label>
<InputCheckbox @bind-Value="globalModel.Defaults.FfmpegFaststart" />
</div>
<div>
<label>FFmpeg Progress</label>
<InputCheckbox @bind-Value="globalModel.Defaults.FfmpegProgress" />
</div>
<div class="mt-2">
<button type="submit">Save Defaults</button>
</div>
</div>
</EditForm>
}
</div>
<div class="mt-2">
<button type="submit" @onclick="SaveGlobal">Save All</button>
</div>
<h4 class="mt-3">Change Password</h4>
@if (!string.IsNullOrEmpty(pwError))
{
<div class="alert">@pwError</div>
}
<div>
<input type="password" @bind="currentPw" placeholder="Current password" />
<input type="password" @bind="newPw" placeholder="New password" />
<input type="password" @bind="confirmPw" placeholder="Confirm" />
<button @onclick="ChangePassword">Change</button>
</div>
@code {
private TwitchArchive.Core.Config.GlobalConfig globalModel = new();
private bool saved = false;
private bool showDefaults = false;
private string currentPw = string.Empty;
private string newPw = string.Empty;
private string confirmPw = string.Empty;
private string pwError = string.Empty;
protected override void OnInitialized()
{
Load();
}
private void Load()
{
try
{
globalModel = ConfigService.LoadGlobal() ?? new TwitchArchive.Core.Config.GlobalConfig();
}
catch { globalModel = new TwitchArchive.Core.Config.GlobalConfig(); }
}
private void SaveGlobal()
{
try
{
ConfigService.SaveGlobal(globalModel);
saved = true;
// optionally notify auth or other services
}
catch { }
}
private void ToggleDefaults() => showDefaults = !showDefaults;
private void ChangePassword()
{
pwError = string.Empty;
if (!Auth.ValidatePassword(currentPw)) { pwError = "Current password incorrect"; return; }
if (string.IsNullOrWhiteSpace(newPw)) { pwError = "New password required"; return; }
if (newPw != confirmPw) { pwError = "Passwords do not match"; return; }
var hash = BCrypt.Net.BCrypt.HashPassword(newPw);
Auth.SetPasswordHash(hash);
Auth.Refresh();
saved = true;
}
}

View file

@ -0,0 +1,10 @@
@page "/config/global"
@inject NavigationManager Nav
@code {
protected override void OnInitialized()
{
// Consolidated settings are now at /settings
Nav.NavigateTo("/settings", true);
}
}

View file

@ -0,0 +1,96 @@
@page "/"
@inject TwitchArchive.Core.Workers.StreamWorkerManager WorkerManager
@inject TwitchArchive.Web.Services.SessionCacheService SessionCache
<h2>Dashboard</h2>
@if (streamers.Count == 0)
{
<div class="alert alert-info">No streamers configured. Add one on the <a href="/addstreamer">Add Streamer</a> page.</div>
}
<div class="cards">
@foreach (var s in streamers)
{
<div class="card">
<div class="card-header">
<a href="/streamer/@s">@s</a>
<a class="btn-link" href="/config/@s">Edit</a>
<span class="badge">@(WorkerManager.IsRunning(s) ? "Live" : "Offline")</span>
</div>
<div class="card-body">
<div>Last session: @(lastStarts.ContainsKey(s) ? lastStarts[s].ToLocalTime().ToString() : "-")</div>
<div class="actions">
<button @onclick="() => Start(s)">Start</button>
<button @onclick="() => Stop(s)">Stop</button>
</div>
</div>
</div>
}
</div>
@* Show global feedback when there are streamers but no recent sessions *@
@if (streamers.Count > 0 && (lastStarts == null || lastStarts.Count == 0))
{
<div class="alert alert-warning mt-3">No recent sessions found for configured streamers.</div>
}
@code {
private List<string> streamers = new();
private Dictionary<string, DateTime> lastStarts = new();
private void OnCacheUpdatedHandler()
{
_ = InvokeAsync(() => {
lastStarts = SessionCache.GetSnapshot();
StateHasChanged();
});
}
protected override async Task OnInitializedAsync()
{
LoadStreamers();
lastStarts = SessionCache.GetSnapshot();
SessionCache.Updated += OnCacheUpdatedHandler;
}
private void LoadStreamers()
{
// Try to find the config/streamers folder from the app content root and parent folders.
string? cfgDir = FindConfigStreamersFolder();
if (!string.IsNullOrEmpty(cfgDir) && Directory.Exists(cfgDir))
{
streamers = Directory.GetFiles(cfgDir, "*.json").Select(f => Path.GetFileNameWithoutExtension(f)).ToList();
}
}
private string? FindConfigStreamersFolder()
{
// Prefer ContentRoot if available, fall back to Environment.CurrentDirectory.
var start = AppContext.BaseDirectory ?? Environment.CurrentDirectory;
var dir = new DirectoryInfo(start);
for (int i = 0; i < 6 && dir != null; i++)
{
var candidate = Path.Combine(dir.FullName, "config", "streamers");
if (Directory.Exists(candidate)) return candidate;
dir = dir.Parent;
}
// final attempt: repo-root relative (use project parent heuristics)
var alt = Path.Combine(Environment.CurrentDirectory, "..", "..", "..", "config", "streamers");
try { alt = Path.GetFullPath(alt); } catch { }
if (Directory.Exists(alt)) return alt;
return null;
}
// Index reads from the singleton SessionCacheService; updates are pushed via the Updated event.
private void Start(string u) { WorkerManager.StartWorker(u); }
private async Task Stop(string u) { await WorkerManager.StopWorkerAsync(u); }
public async ValueTask DisposeAsync()
{
SessionCache.Updated -= OnCacheUpdatedHandler;
await Task.CompletedTask;
}
}

View file

@ -0,0 +1,20 @@
@page "/login"
@attribute [AllowAnonymous]
@using Microsoft.AspNetCore.Components
<h3>Login</h3>
@if (error)
{
<div class="alert">Invalid password</div>
}
<form method="post" action="/auth/login">
<input type="password" name="password" placeholder="Password" />
<button type="submit">Login</button>
</form>
@code {
[Parameter]
public bool error { get; set; }
}

View file

@ -0,0 +1,59 @@
@page "/media"
@inject TwitchArchive.Core.Config.IConfigurationService ConfigService
<h3>Media Library</h3>
@if (string.IsNullOrWhiteSpace(archiveRoot))
{
<div class="alert">Archive root is not configured. Set it on the <a href="/settings">Settings</a> page.</div>
}
else if (!Directory.Exists(archiveRoot))
{
<div class="alert">Archive root '@archiveRoot' does not exist on disk.</div>
}
else
{
@if (entries.Count == 0)
{
<div class="alert">No media files found in '@archiveRoot'.</div>
}
else
{
@foreach (var kv in entries)
{
<div class="card">
<h4>@kv.Key</h4>
<ul>
@foreach (var f in kv.Value)
{
<li>@f.Name - @((f.Length/1024.0/1024.0).ToString("0.00")) MB - @f.LastWriteTime.ToLocalTime()</li>
}
</ul>
</div>
}
}
}
@code {
private string? archiveRoot;
private Dictionary<string, List<FileInfo>> entries = new();
protected override void OnInitialized()
{
var g = ConfigService.LoadGlobal();
archiveRoot = g?.ArchiveRoot;
if (!string.IsNullOrWhiteSpace(archiveRoot) && Directory.Exists(archiveRoot))
{
var di = new DirectoryInfo(archiveRoot);
foreach (var dir in di.GetDirectories())
{
try
{
var files = dir.GetFiles("*.mp4").Concat(dir.GetFiles("*.mkv")).Concat(dir.GetFiles("*.ts")).Concat(dir.GetFiles("*.flv")).OrderByDescending(f => f.LastWriteTime).ToList();
if (files.Count > 0) entries[dir.Name] = files;
}
catch { }
}
}
}
}

View file

@ -0,0 +1,37 @@
@page "/monitor"
@inject TwitchArchive.Core.Workers.StreamWorkerManager WorkerManager
@inject TwitchArchive.Core.Services.IProcessOutputStore OutputStore
@using TwitchArchive.Web.Shared
<h2>Streamer Monitor</h2>
<div>
<input @bind="username" placeholder="username" />
<button @onclick="Start">Start</button>
<button @onclick="Stop">Stop</button>
</div>
<div style="margin-top:1rem">
<strong>Status:</strong> @status
</div>
<ProcessConsole Streamer="username" />
@code {
private string username = "hackerling";
private string status = "idle";
private void Start()
{
if (string.IsNullOrWhiteSpace(username)) return;
WorkerManager.StartWorker(username);
status = WorkerManager.IsRunning(username) ? "running" : "starting";
}
private async Task Stop()
{
if (string.IsNullOrWhiteSpace(username)) return;
await WorkerManager.StopWorkerAsync(username);
status = WorkerManager.IsRunning(username) ? "running" : "stopped";
}
}

View file

@ -0,0 +1,73 @@
@page "/sessions"
@using TwitchArchive.Core.Persistence.Models
@inject TwitchArchive.Core.Persistence.ISessionRepository SessionRepo
<h3>Sessions</h3>
<button @onclick="Refresh">Refresh</button>
<table class="table">
<thead>
<tr><th>Id</th><th>Streamer</th><th>Started</th><th>Ended</th><th>Status</th><th>Jobs</th></tr>
</thead>
<tbody>
@foreach (var s in sessions)
{
<tr>
<td>@s.Id</td>
<td>@s.StreamerUsername</td>
<td>@s.StartedAt.ToLocalTime()</td>
<td>@(s.EndedAt?.ToLocalTime().ToString() ?? "-")</td>
<td>@s.Status</td>
<td><button @onclick="() => ToggleJobs(s.Id)">Toggle</button></td>
</tr>
@if (expandedSession == s.Id)
{
<tr><td colspan="6">
<ul>
@if (jobs?.Any() ?? false)
{
@foreach (var j in jobs)
{
<li>@j.Id - @j.JobType - @j.Status - @j.StartedAt.ToLocalTime() - @(j.FilePath ?? "")</li>
}
}
else
{
<li>No jobs</li>
}
</ul>
</td></tr>
}
}
</tbody>
</table>
@code {
private List<StreamSession> sessions = new();
private List<ArchiveJob>? jobs;
private long? expandedSession;
protected override async Task OnInitializedAsync()
{
await Refresh();
}
private async Task Refresh()
{
sessions = await SessionRepo.GetRecentSessionsAsync(50);
StateHasChanged();
}
private async Task ToggleJobs(long sessionId)
{
if (expandedSession == sessionId)
{
expandedSession = null;
jobs = null;
return;
}
jobs = await SessionRepo.GetJobsForSessionAsync(sessionId);
expandedSession = sessionId;
}
}

View file

@ -0,0 +1,228 @@
@page "/config/{Username}"
@inject TwitchArchive.Core.Config.IConfigurationService ConfigService
@inject NavigationManager Nav
<h3>Streamer Config: @Username</h3>
<EditForm Model="model" OnValidSubmit="Save">
<DataAnnotationsValidator />
<ValidationSummary />
@if (saved)
{
<div class="alert">Saved.</div>
}
<div>
<label title="Toggle whether this streamer is active">Enabled</label>
<InputCheckbox Value="model.Enabled" ValueChanged="@( (bool v) => model.Enabled = v )" ValueExpression="() => model.Enabled" title="Toggle whether this streamer is active" />
</div>
<div class="card">
<label title="Quality for this streamer">Quality</label>
<InputText @bind-Value="model.Quality" placeholder="@(global?.DefaultQuality ?? "")" title="Enter a quality string (e.g., best, 720p)" />
</div>
<div>
<label title="Toggle cloud upload for this streamer">Upload to Cloud</label>
<InputCheckbox Value="uploadToCloudVal" ValueChanged="@( (bool v) => uploadToCloudVal = v )" ValueExpression="() => uploadToCloudVal" title="Upload to configured cloud destination" />
</div>
<div class="card">
<label title="Cloud destination (e.g., rclone remote) for this streamer">Upload Destination</label>
<InputText @bind-Value="model.UploadDestination" title="Cloud destination (e.g., rclone remote)" />
</div>
@* Streamlink path is global-only; not configurable per-streamer *@
<div class="card">
<h4>Per-streamer settings</h4>
<div>
<label>Download VOD</label>
<InputCheckbox Value="downloadVODVal" ValueChanged="@( (bool v) => downloadVODVal = v )" ValueExpression="() => downloadVODVal" />
</div>
<div>
<label>Download CHAT</label>
<InputCheckbox Value="downloadCHATVal" ValueChanged="@( (bool v) => downloadCHATVal = v )" ValueExpression="() => downloadCHATVal" />
</div>
<div>
<label>Merge Video & Chat</label>
<InputCheckbox Value="mergeVideoChatVal" ValueChanged="@( (bool v) => mergeVideoChatVal = v )" ValueExpression="() => mergeVideoChatVal" />
</div>
<div>
<label>Merge Chat Layout</label>
<InputSelect @bind-Value="mergeChatLayoutVal">
<option value="side-by-side">Side by side</option>
<option value="stacked">Stacked</option>
<option value="overlay">Overlay</option>
</InputSelect>
</div>
<div>
<label>VOD Timeout (sec)</label>
<InputNumber @bind-Value="vodTimeoutVal" />
</div>
<div>
<label>Delete Files</label>
<InputCheckbox Value="deleteFilesVal" ValueChanged="@( (bool v) => deleteFilesVal = v )" ValueExpression="() => deleteFilesVal" />
</div>
<div>
<label>Download Live CHAT</label>
<InputCheckbox Value="downloadLiveCHATVal" ValueChanged="@( (bool v) => downloadLiveCHATVal = v )" ValueExpression="() => downloadLiveCHATVal" />
</div>
<div>
<label>Upload Pre-Merge Video</label>
<InputCheckbox Value="uploadPreMergeVideoVal" ValueChanged="@( (bool v) => uploadPreMergeVideoVal = v )" ValueExpression="() => uploadPreMergeVideoVal" />
</div>
<div>
<label>Upload Merged Video</label>
<InputCheckbox Value="uploadMergedVideoVal" ValueChanged="@( (bool v) => uploadMergedVideoVal = v )" ValueExpression="() => uploadMergedVideoVal" />
</div>
<div>
<label>Upload Chat Video</label>
<InputCheckbox Value="uploadChatVideoVal" ValueChanged="@( (bool v) => uploadChatVideoVal = v )" ValueExpression="() => uploadChatVideoVal" />
</div>
<div>
<label>Only Raw</label>
<InputCheckbox Value="onlyRawVal" ValueChanged="@( (bool v) => onlyRawVal = v )" ValueExpression="() => onlyRawVal" />
</div>
<div>
<label>Clean Raw</label>
<InputCheckbox Value="cleanRawVal" ValueChanged="@( (bool v) => cleanRawVal = v )" ValueExpression="() => cleanRawVal" />
</div>
</div>
<button type="submit" title="Save streamer configuration">Save</button>
<button type="button" @onclick="() => showConfirm = true" title="Delete this streamer and its configuration">Delete</button>
</EditForm>
@* Confirmation modal *@
@if (showConfirm)
{
<div class="modal-backdrop">
<div class="modal card">
<div class="card-body">
<h4>Confirm delete</h4>
<p>Delete streamer '@Username'? This will remove its config file.</p>
<div class="actions">
<button class="btn-link" @onclick="ConfirmDelete">Yes, delete</button>
<button class="btn-link" @onclick="() => showConfirm = false">Cancel</button>
</div>
</div>
</div>
</div>
}
@code {
[Parameter]
public string Username { get; set; } = string.Empty;
private TwitchArchive.Core.Config.StreamerConfig model = new();
private TwitchArchive.Core.Config.GlobalConfig? global;
private bool saved = false;
private bool showConfirm = false;
// local values for nullable per-streamer settings (bind safely)
private bool downloadVODVal;
private bool downloadCHATVal;
private bool downloadLiveCHATVal;
private bool mergeVideoChatVal;
private string mergeChatLayoutVal = "side-by-side";
private int? vodTimeoutVal;
private bool deleteFilesVal;
private bool uploadPreMergeVideoVal;
private bool uploadMergedVideoVal;
private bool uploadChatVideoVal;
private bool onlyRawVal;
private bool cleanRawVal;
private bool uploadToCloudVal;
protected override void OnInitialized()
{
global = ConfigService.LoadGlobal();
var s = ConfigService.LoadStreamer(Username);
var isNew = s == null;
if (s != null) model = s;
// initialize local values from model or global defaults
downloadVODVal = model.DownloadVOD ?? global?.Defaults.DownloadVOD ?? true;
downloadCHATVal = model.DownloadCHAT ?? global?.Defaults.DownloadCHAT ?? true;
downloadLiveCHATVal = model.DownloadLiveCHAT ?? global?.Defaults.DownloadLiveCHAT ?? true;
mergeVideoChatVal = model.MergeVideoChat ?? global?.Defaults.MergeVideoChat ?? false;
mergeChatLayoutVal = model.MergeChatLayout ?? global?.Defaults.MergeChatLayout ?? "side-by-side";
vodTimeoutVal = model.VodTimeout ?? global?.Defaults.VodTimeout ?? 300;
deleteFilesVal = model.DeleteFiles ?? global?.Defaults.DeleteFiles ?? false;
uploadPreMergeVideoVal = model.UploadPreMergeVideo ?? global?.Defaults.UploadPreMergeVideo ?? true;
uploadMergedVideoVal = model.UploadMergedVideo ?? global?.Defaults.UploadMergedVideo ?? true;
uploadChatVideoVal = model.UploadChatVideo ?? global?.Defaults.UploadChatVideo ?? false;
onlyRawVal = model.OnlyRaw ?? global?.Defaults.OnlyRaw ?? false;
cleanRawVal = model.CleanRaw ?? global?.Defaults.CleanRaw ?? true;
uploadToCloudVal = model.UploadToCloud ?? global?.UploadToCloud ?? false;
// when creating a new streamer config, populate model with global defaults so
// the streamer config stores initial values and subsequent edits use streamer values
if (isNew)
{
model.Quality = model.Quality ?? global?.DefaultQuality;
model.UploadToCloud = uploadToCloudVal;
model.UploadDestination = model.UploadDestination ?? global?.UploadDestination;
model.DownloadVOD = downloadVODVal;
model.DownloadCHAT = downloadCHATVal;
model.DownloadLiveCHAT = downloadLiveCHATVal;
model.MergeVideoChat = mergeVideoChatVal;
model.MergeChatLayout = mergeChatLayoutVal;
model.VodTimeout = vodTimeoutVal;
model.DeleteFiles = deleteFilesVal;
model.UploadPreMergeVideo = uploadPreMergeVideoVal;
model.UploadMergedVideo = uploadMergedVideoVal;
model.UploadChatVideo = uploadChatVideoVal;
model.OnlyRaw = onlyRawVal;
model.CleanRaw = cleanRawVal;
}
}
private void Save()
{
model.Username = Username;
if (string.IsNullOrWhiteSpace(model.Quality)) model.Quality = null;
// Upload to cloud
model.UploadToCloud = uploadToCloudVal;
// Ensure global-only settings are not stored per-streamer
model.StreamlinkPath = null;
model.HlsSegments = null;
model.HlsSegmentsVOD = null;
model.StreamlinkTtvlol = null;
model.FfmpegHwaccel = null;
model.FfmpegThreads = null;
model.FfmpegAudioBitrate = null;
model.FfmpegAudioCodec = null;
model.FfmpegAudioSamplerate = null;
model.FfmpegErrorRecovery = null;
model.FfmpegFaststart = null;
model.FfmpegProgress = null;
// Per-streamer values: always map local values into the model
model.DownloadVOD = downloadVODVal;
model.DownloadCHAT = downloadCHATVal;
model.DownloadLiveCHAT = downloadLiveCHATVal;
model.MergeVideoChat = mergeVideoChatVal;
model.MergeChatLayout = mergeChatLayoutVal;
model.VodTimeout = vodTimeoutVal;
model.DeleteFiles = deleteFilesVal;
model.UploadPreMergeVideo = uploadPreMergeVideoVal;
model.UploadMergedVideo = uploadMergedVideoVal;
model.UploadChatVideo = uploadChatVideoVal;
model.OnlyRaw = onlyRawVal;
model.CleanRaw = cleanRawVal;
ConfigService.SaveStreamer(model);
saved = true;
}
private void ConfirmDelete()
{
try
{
ConfigService.DeleteStreamer(Username);
showConfirm = false;
Nav.NavigateTo("/");
}
catch
{
// ignore
}
}
}

View file

@ -0,0 +1,23 @@
@page "/streamer/{Username}"
@using TwitchArchive.Core.Workers
@inject StreamWorkerManager WorkerManager
<h2>Streamer: @Username</h2>
<div class="streamer-header">
<span class="status">Status: <strong>@(WorkerManager.IsRunning(Username) ? "Live" : "Offline")</strong></span>
<span class="actions"><a class="btn-link" href="/config/@Username">Edit Settings</a></span>
</div>
<div class="pipeline">
<div class="step done">Record</div>
<div class="step">Process</div>
<div class="step">Upload</div>
</div>
<ProcessConsole Streamer="Username" />
@code {
[Parameter]
public string Username { get; set; } = string.Empty;
}

View file

@ -0,0 +1,19 @@
@page "/"
@namespace TwitchArchive.Web.Pages
@addTagHelper *, Microsoft.AspNetCore.Mvc.TagHelpers
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Twitch Archive</title>
<base href="~/" />
<link href="css/app.css" rel="stylesheet" />
</head>
<body>
<app>
<component type="typeof(App)" render-mode="ServerPrerendered" />
</app>
<script src="_framework/blazor.server.js"></script>
</body>
</html>

View file

@ -0,0 +1,105 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Authentication;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.EntityFrameworkCore;
using TwitchArchive.Core.Services;
using TwitchArchive.Core.Config;
using TwitchArchive.Core.Persistence;
using TwitchArchive.Web.Hubs;
using TwitchArchive.Web.Services;
var builder = WebApplication.CreateBuilder(args);
// Add services
builder.Services.AddRazorPages();
builder.Services.AddServerSideBlazor();
builder.Services.AddSignalR();
// Authentication
builder.Services.AddSingleton<TwitchArchive.Web.Services.IAuthService, TwitchArchive.Web.Services.AuthService>();
builder.Services.AddAuthentication(options =>
{
options.DefaultScheme = Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationDefaults.AuthenticationScheme;
})
.AddCookie(options => { options.LoginPath = "/login"; });
// Register Core services
builder.Services.AddSingleton<IProcessRunner, ProcessRunner>();
builder.Services.AddSingleton<IProcessOutputStore, ProcessOutputStore>();
builder.Services.AddSingleton<FileManagerService>();
builder.Services.AddHttpClient<TwitchArchive.Core.Api.ITwitchApiClient, TwitchArchive.Core.Api.TwitchApiClient>(client => { /* base config if needed */ });
builder.Services.AddSingleton<TwitchArchive.Core.Monitoring.ILiveChecker, TwitchArchive.Core.Monitoring.TwitchLiveChecker>();
builder.Services.AddSingleton<TwitchArchive.Core.Workers.StreamWorkerManager>();
builder.Services.AddSingleton<IDownloaderService, DownloaderService>();
builder.Services.AddSingleton<IProcessorService, ProcessorService>();
// Configuration service for global + per-streamer JSON files in ./config
builder.Services.AddScoped<IConfigurationService, ConfigurationService>();
// Broadcaster forwards output store events to the SignalR hub
builder.Services.AddSingleton<ProcessOutputBroadcaster>();
// SQLite DB (file in app folder)
var conn = "Data Source=archive.db";
// Provide a factory for creating DbContext instances for background work
builder.Services.AddDbContextFactory<ArchiveDbContext>(opt => opt.UseSqlite(conn));
// persistence
builder.Services.AddScoped<TwitchArchive.Core.Persistence.ISessionRepository, TwitchArchive.Core.Persistence.SessionRepository>();
// Session cache and background refresh
builder.Services.AddSingleton<TwitchArchive.Web.Services.SessionCacheService>();
builder.Services.AddHostedService<TwitchArchive.Web.Services.SessionRefreshHostedService>();
var app = builder.Build();
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
}
// Ensure DB schema exists (creates DB when missing)
using (var scope = app.Services.CreateScope())
{
var factory = scope.ServiceProvider.GetRequiredService<IDbContextFactory<ArchiveDbContext>>();
try
{
using var db = factory.CreateDbContext();
// Apply any pending EF migrations (creates/updates schema as needed)
db.Database.Migrate();
}
catch { }
}
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
// login endpoint
app.MapPost("/auth/login", async (Microsoft.AspNetCore.Http.HttpContext http, TwitchArchive.Web.Services.IAuthService auth) =>
{
try
{
var form = await http.Request.ReadFormAsync();
var pwd = form["password"].ToString();
if (auth.ValidatePassword(pwd))
{
var claims = new[] { new System.Security.Claims.Claim(System.Security.Claims.ClaimTypes.Name, "admin") };
var id = new System.Security.Claims.ClaimsIdentity(claims, Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationDefaults.AuthenticationScheme);
var principal = new System.Security.Claims.ClaimsPrincipal(id);
await http.SignInAsync(Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationDefaults.AuthenticationScheme, principal);
http.Response.Redirect("/");
return;
}
}
catch { }
http.Response.Redirect("/login?error=1");
});
app.MapBlazorHub();
app.MapHub<ProcessOutputHub>("/processOutputHub");
app.MapFallbackToPage("/_Host");
app.Run();

View file

@ -0,0 +1,12 @@
{
"profiles": {
"TwitchArchive.Web": {
"commandName": "Project",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "https://localhost:64466;http://localhost:64467"
}
}
}

View file

@ -0,0 +1,75 @@
using System;
using System.Linq;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using TwitchArchive.Core.Persistence;
using TwitchArchive.Core.Persistence.Models;
namespace TwitchArchive.Web.Services
{
public class AuthService : IAuthService
{
private readonly IDbContextFactory<ArchiveDbContext> _dbFactory;
private readonly ILogger<AuthService> _log;
private string? _cachedHash;
public AuthService(IDbContextFactory<ArchiveDbContext> dbFactory, ILogger<AuthService> log)
{
_dbFactory = dbFactory ?? throw new ArgumentNullException(nameof(dbFactory));
_log = log;
Refresh();
}
public void Refresh()
{
try
{
using var ctx = _dbFactory.CreateDbContext();
var u = ctx.UserCredentials.AsNoTracking().FirstOrDefault();
_cachedHash = u?.PasswordHash;
}
catch (Exception ex)
{
_log?.LogWarning(ex, "Failed to read password from database");
_cachedHash = null;
}
}
public bool ValidatePassword(string plain)
{
// If no password configured, allow access (default open)
if (string.IsNullOrWhiteSpace(_cachedHash)) return true;
try { return BCrypt.Net.BCrypt.Verify(plain ?? string.Empty, _cachedHash); }
catch (Exception ex)
{
_log?.LogWarning(ex, "Password verification failed");
return false;
}
}
public void SetPasswordHash(string hash)
{
try
{
using var ctx = _dbFactory.CreateDbContext();
var existing = ctx.UserCredentials.FirstOrDefault();
if (existing == null)
{
existing = new UserCredential { PasswordHash = hash };
ctx.UserCredentials.Add(existing);
}
else
{
existing.PasswordHash = hash;
ctx.UserCredentials.Update(existing);
}
ctx.SaveChanges();
_cachedHash = hash;
}
catch (Exception ex)
{
_log?.LogError(ex, "Failed to save password to database");
}
}
}
}

View file

@ -0,0 +1,11 @@
using System.Threading.Tasks;
namespace TwitchArchive.Web.Services
{
public interface IAuthService
{
bool ValidatePassword(string plain);
void Refresh();
void SetPasswordHash(string hash);
}
}

View file

@ -0,0 +1,35 @@
using Microsoft.AspNetCore.SignalR;
using System;
using TwitchArchive.Core.Services;
using TwitchArchive.Web.Hubs;
namespace TwitchArchive.Web.Services
{
public class ProcessOutputBroadcaster : IDisposable
{
private readonly IProcessOutputStore _store;
private readonly IHubContext<ProcessOutputHub> _hub;
public ProcessOutputBroadcaster(IProcessOutputStore store, IHubContext<ProcessOutputHub> hub)
{
_store = store;
_hub = hub;
_store.LineAppended += OnLineAppended;
}
private void OnLineAppended(string streamer, OutputLine line)
{
try
{
// send to clients; clients should listen on "ReceiveLine"
_hub.Clients.Group(streamer).SendAsync("ReceiveLine", streamer, line);
}
catch { }
}
public void Dispose()
{
_store.LineAppended -= OnLineAppended;
}
}
}

View file

@ -0,0 +1,38 @@
using System;
using System.Collections.Generic;
using TwitchArchive.Core.Persistence.Models;
namespace TwitchArchive.Web.Services
{
public class SessionCacheService
{
private readonly object _lock = new();
private Dictionary<string, DateTime> _snapshot = new();
public event Action? Updated;
public void Update(IEnumerable<StreamSession> sessions)
{
var next = new Dictionary<string, DateTime>();
foreach (var s in sessions)
{
if (!next.ContainsKey(s.StreamerUsername)) next[s.StreamerUsername] = s.StartedAt;
}
lock (_lock)
{
_snapshot = next;
}
Updated?.Invoke();
}
public Dictionary<string, DateTime> GetSnapshot()
{
lock (_lock)
{
return new Dictionary<string, DateTime>(_snapshot);
}
}
}
}

View file

@ -0,0 +1,50 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using TwitchArchive.Core.Persistence;
namespace TwitchArchive.Web.Services
{
public class SessionRefreshHostedService : BackgroundService
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly SessionCacheService _cache;
private readonly ILogger<SessionRefreshHostedService> _logger;
private readonly TimeSpan _interval;
public SessionRefreshHostedService(IServiceScopeFactory scopeFactory, SessionCacheService cache, ILogger<SessionRefreshHostedService> logger)
{
_scopeFactory = scopeFactory ?? throw new ArgumentNullException(nameof(scopeFactory));
_cache = cache ?? throw new ArgumentNullException(nameof(cache));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_interval = TimeSpan.FromSeconds(10);
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using var timer = new PeriodicTimer(_interval);
try
{
while (await timer.WaitForNextTickAsync(stoppingToken))
{
try
{
using var scope = _scopeFactory.CreateScope();
var repo = scope.ServiceProvider.GetRequiredService<ISessionRepository>();
var sessions = await repo.GetRecentSessionsAsync(200, stoppingToken).ConfigureAwait(false);
_cache.Update(sessions);
}
catch (OperationCanceledException) { break; }
catch (Exception ex)
{
_logger.LogError(ex, "Error while refreshing sessions");
}
}
}
catch (OperationCanceledException) { }
}
}
}

View file

@ -0,0 +1,31 @@
@inherits LayoutComponentBase
<header class="topbar">
<div class="topbar-inner">
<button class="hamburger" @onclick="ToggleSidebar">☰</button>
<h1 class="title">Twitch Archive</h1>
<nav class="top-actions">
<NavLink href="/login" class="action">Login</NavLink>
</nav>
</div>
</header>
<div class="page">
<nav class="sidebar @(sidebarCollapsed ? "collapsed" : "")">
<div class="brand">Twitch Archive</div>
<ul class="nav-list">
<li><NavLink href="/" class="nav-link">Dashboard</NavLink></li>
<li><NavLink href="/" class="nav-link">Streamers</NavLink></li>
<li><NavLink href="/addstreamer" class="nav-link">Add Streamer</NavLink></li>
<li><NavLink href="/sessions" class="nav-link">Sessions</NavLink></li>
<li><NavLink href="/monitor" class="nav-link">Monitor</NavLink></li>
<li><NavLink href="/settings" class="nav-link">Settings</NavLink></li>
</ul>
</nav>
<main class="main">
<div class="container">@Body</div>
</main>
</div>
@code {
bool sidebarCollapsed;
void ToggleSidebar() => sidebarCollapsed = !sidebarCollapsed;
}

View file

@ -0,0 +1,93 @@
@using TwitchArchive.Core.Services
@using Microsoft.AspNetCore.Components
@using Microsoft.AspNetCore.SignalR.Client
@using System.Linq
@inject IProcessOutputStore OutputStore
@inject NavigationManager Navigation
@inherits ComponentBase
<div class="console" style="background:#111;color:#eee;padding:8px;height:400px;overflow:auto;font-family:monospace;white-space:pre-wrap;">
@foreach (var line in lines)
{
<div style="color:@(line.IsError ? "#ff8888" : "#ddd")">@line.TimestampUtc.ToLocalTime().ToString("HH:mm:ss") - @line.Line</div>
}
</div>
@code {
[Parameter]
public string Streamer { get; set; } = string.Empty;
private List<OutputLine> lines = new();
private HubConnection? hubConnection;
private string? _currentGroup;
protected override async Task OnInitializedAsync()
{
OutputStore.LineAppended += OnLineAppended;
hubConnection = new HubConnectionBuilder()
.WithUrl(Navigation.ToAbsoluteUri("/processOutputHub"))
.WithAutomaticReconnect()
.Build();
hubConnection.On<string, OutputLine>("ReceiveLine", (streamer, line) =>
{
if (!string.Equals(streamer, Streamer, StringComparison.OrdinalIgnoreCase)) return;
// avoid duplicates
if (lines.Any(l => l.TimestampUtc == line.TimestampUtc && l.Line == line.Line)) return;
lines.Add(line);
if (lines.Count > 1000) lines.RemoveAt(0);
InvokeAsync(StateHasChanged);
});
await hubConnection.StartAsync();
if (!string.IsNullOrWhiteSpace(Streamer))
{
await hubConnection.SendAsync("JoinStreamerGroup", Streamer);
_currentGroup = Streamer;
}
await base.OnInitializedAsync();
}
protected override async Task OnParametersSetAsync()
{
if (hubConnection == null) return;
if (!string.IsNullOrWhiteSpace(_currentGroup) && !string.Equals(_currentGroup, Streamer, StringComparison.OrdinalIgnoreCase))
{
try { await hubConnection.SendAsync("LeaveStreamerGroup", _currentGroup); } catch { }
_currentGroup = null;
}
if (!string.IsNullOrWhiteSpace(Streamer) && !string.Equals(_currentGroup, Streamer, StringComparison.OrdinalIgnoreCase))
{
try { await hubConnection.SendAsync("JoinStreamerGroup", Streamer); _currentGroup = Streamer; } catch { }
}
await base.OnParametersSetAsync();
}
private void OnLineAppended(string streamer, OutputLine line)
{
if (!string.Equals(streamer, Streamer, StringComparison.OrdinalIgnoreCase)) return;
lines.Add(line);
if (lines.Count > 1000) lines.RemoveAt(0);
InvokeAsync(StateHasChanged);
}
public async ValueTask DisposeAsync()
{
OutputStore.LineAppended -= OnLineAppended;
if (hubConnection != null)
{
try
{
if (!string.IsNullOrWhiteSpace(_currentGroup)) await hubConnection.SendAsync("LeaveStreamerGroup", _currentGroup);
}
catch { }
try { await hubConnection.StopAsync(); } catch { }
try { await hubConnection.DisposeAsync(); } catch { }
hubConnection = null;
}
}
}

View file

@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\TwitchArchive.Core\TwitchArchive.Core.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.Server" Version="10.0.3" />
<PackageReference Include="Microsoft.AspNetCore.SignalR.Client" Version="10.0.3" />
<PackageReference Include="NLog.Web.AspNetCore" Version="6.1.1" />
<PackageReference Include="BCrypt.Net-Next" Version="4.*" />
</ItemGroup>
</Project>

View file

@ -0,0 +1,10 @@
@using System
@using System.Net.Http
@using Microsoft.AspNetCore.Components
@using Microsoft.AspNetCore.Components.Web
@using Microsoft.AspNetCore.Components.Routing
@using Microsoft.AspNetCore.Components.Forms
@using Microsoft.AspNetCore.Authorization
@using TwitchArchive.Core
@using TwitchArchive.Core.Services
@using TwitchArchive.Web.Shared

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -0,0 +1,34 @@
/* App layout styles for Twitch Archive */
:root{
--bg:#0f1720; --panel:#111322; --muted:#9aa4c5; --accent:#7dd3fc; --accent-2:#89b4fa; --card:#0b1220;
}
html,body { height:100%; margin:0; font-family:Segoe UI, Roboto, -apple-system, sans-serif; background:var(--bg); color:#e6eef8; }
.page { display:flex; min-height:100vh; }
.sidebar { width:220px; flex-shrink:0; background:var(--panel); color:var(--muted); overflow-y:auto; padding:1rem 0; box-shadow:2px 0 8px rgba(6,6,10,0.6); }
.sidebar.collapsed { width:66px; }
.brand { font-weight:600; padding:0 1rem 0.8rem 1rem; color:#fff; }
.nav-list { list-style:none; margin:0; padding:0; }
.nav-list li { margin:0.2rem 0; }
.nav-link { display:block; padding:0.6rem 1rem; color:var(--muted); text-decoration:none; border-left:4px solid transparent; }
.nav-link.active, .nav-link:hover { background:linear-gradient(90deg, rgba(255,255,255,0.02), transparent); color:#fff; border-left-color:var(--accent-2); }
.topbar { display:flex; align-items:center; background:var(--panel); padding:0.6rem 1rem; box-shadow:0 2px 6px rgba(0,0,0,0.4); }
.topbar .topbar-inner { display:flex; align-items:center; width:100%; }
.hamburger { font-size:1.2rem; margin-right:1rem; background:transparent; border:none; color:inherit; cursor:pointer; }
.title { margin:0; font-size:1.05rem; flex:1; }
.top-actions .action { color:var(--muted); text-decoration:none; margin-left:1rem; }
.main { flex:1; overflow-y:auto; padding:1.5rem; }
.container { max-width:1100px; margin:0 auto; }
/* Cards and forms */
.card { background:var(--card); padding:1rem; border-radius:8px; box-shadow:0 2px 8px rgba(2,6,23,0.6); margin-bottom:1rem; }
.card-header { display:flex; justify-content:space-between; align-items:center; font-weight:600; }
.card-body { margin-top:0.5rem; color:var(--muted); }
button { background:var(--accent); border:none; color:#002; padding:0.45rem 0.8rem; border-radius:6px; cursor:pointer; }
button:hover { opacity:0.95; }
input[type=password], input[type=text], input[type=number], .input, InputText { padding:0.5rem; border-radius:6px; border:1px solid rgba(255,255,255,0.06); background:transparent; color:inherit; }
@media(max-width:768px) {
.sidebar { display:none; }
.topbar { display:flex; }
}

View file

@ -32,6 +32,9 @@ DEFAULT_CONFIG = {
'mergeChatLayout': 'side-by-side', # Layout: 'side-by-side' or 'overlay' 'mergeChatLayout': 'side-by-side', # Layout: 'side-by-side' or 'overlay'
'vodTimeout': 300, 'vodTimeout': 300,
'uploadCloud': True, 'uploadCloud': True,
'uploadPreMergeVideo': True, # Upload original videos before merging
'uploadMergedVideo': True, # Upload merged videos (video + chat)
'uploadChatVideo': False, # Upload standalone chat video
'deleteFiles': False, 'deleteFiles': False,
'onlyRaw': False, 'onlyRaw': False,
'cleanRaw': True, 'cleanRaw': True,

View file

@ -4,10 +4,13 @@ Includes fallback support for chat_downloader when VOD-based methods fail.
""" """
import os import os
import sys
import subprocess import subprocess
import json import json
import threading import threading
import time import time
import socket
import re
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
from colorama import Fore, Style from colorama import Fore, Style
@ -38,11 +41,15 @@ class ContentDownloader:
self.ffmpeg_path = ffmpeg_path self.ffmpeg_path = ffmpeg_path
self.quality = config.get('quality', 'best') self.quality = config.get('quality', 'best')
self.hls_segments_vod = config.get('hls_segmentsVOD', 10) self.hls_segments_vod = config.get('hls_segmentsVOD', 10)
self.download_vod = config.get('downloadVOD', True) self.download_vod_enabled = config.get('downloadVOD', True)
self.download_chat = config.get('downloadCHAT', True) self.download_chat_enabled = config.get('downloadCHAT', True)
self.download_live_chat = config.get('downloadLiveCHAT', True) self.download_live_chat_enabled = config.get('downloadLiveCHAT', True)
self.use_chat_downloader_primary = config.get('useChatDownloaderPrimary', False) self.use_chat_downloader_primary = config.get('useChatDownloaderPrimary', False)
self.use_chat_downloader_fallback = config.get('useChatDownloaderFallback', True) self.use_chat_downloader_fallback = config.get('useChatDownloaderFallback', True)
default_chat_font = 'Arial' if sys.platform.startswith('win') else 'DejaVu Sans'
self.chat_render_font = config.get('chat_render_font', default_chat_font)
self.last_chat_render_attempted = False
self.last_chat_render_succeeded = False
# Initialize chat_downloader if available # Initialize chat_downloader if available
self.chat_downloader = None self.chat_downloader = None
@ -60,6 +67,11 @@ class ContentDownloader:
self.chat_thread_success = False self.chat_thread_success = False
self.chat_thread_error = None self.chat_thread_error = None
def reset_chat_render_status(self) -> None:
"""Reset chat render tracking before a processing pass."""
self.last_chat_render_attempted = False
self.last_chat_render_succeeded = False
def download_vod(self, vod_info: Dict[str, Any], output_path: str) -> bool: def download_vod(self, vod_info: Dict[str, Any], output_path: str) -> bool:
""" """
Download VOD using TwitchDownloaderCLI. Download VOD using TwitchDownloaderCLI.
@ -71,7 +83,7 @@ class ContentDownloader:
Returns: Returns:
bool: True if download succeeded, False otherwise bool: True if download succeeded, False otherwise
""" """
if not self.download_vod: if not self.download_vod_enabled:
return False return False
print(f'\n{Fore.CYAN}Downloading VOD: {vod_info["title"]}{Style.RESET_ALL}') print(f'\n{Fore.CYAN}Downloading VOD: {vod_info["title"]}{Style.RESET_ALL}')
@ -188,7 +200,7 @@ class ContentDownloader:
'-h', '1080', '-h', '1080',
'--framerate', '30', '--framerate', '30',
'--outline', '--outline',
'-f', 'Arial', '-f', self.chat_render_font,
'--font-size', '22', '--font-size', '22',
'--update-rate', '1.0', '--update-rate', '1.0',
'--offline', '--offline',
@ -213,6 +225,9 @@ class ContentDownloader:
try: try:
print(f'{Fore.YELLOW}Rendering chat video...{Style.RESET_ALL}') print(f'{Fore.YELLOW}Rendering chat video...{Style.RESET_ALL}')
print(f'{Fore.CYAN}Using chat font: {self.chat_render_font}{Style.RESET_ALL}')
self.last_chat_render_attempted = True
self.last_chat_render_succeeded = False
# Build complete command # Build complete command
full_cmd = [self.twitch_downloader_path, 'chatrender', '-i', json_path, '-o', video_path] + chat_settings full_cmd = [self.twitch_downloader_path, 'chatrender', '-i', json_path, '-o', video_path] + chat_settings
@ -247,6 +262,7 @@ class ContentDownloader:
print(f'{Fore.RED}✗ Chat video file is too small ({file_size} bytes){Style.RESET_ALL}') print(f'{Fore.RED}✗ Chat video file is too small ({file_size} bytes){Style.RESET_ALL}')
return False return False
self.last_chat_render_succeeded = True
print(f'{Fore.GREEN}✓ Chat rendered ({file_size:,} bytes){Style.RESET_ALL}') print(f'{Fore.GREEN}✓ Chat rendered ({file_size:,} bytes){Style.RESET_ALL}')
return True return True
@ -270,7 +286,7 @@ class ContentDownloader:
Returns: Returns:
bool: True if succeeded, False otherwise bool: True if succeeded, False otherwise
""" """
if not self.download_chat: if not self.download_chat_enabled:
return False return False
print(f'\n{Fore.CYAN}Downloading chat: {vod_info["title"]}{Style.RESET_ALL}') print(f'\n{Fore.CYAN}Downloading chat: {vod_info["title"]}{Style.RESET_ALL}')
@ -296,7 +312,7 @@ class ContentDownloader:
Returns: Returns:
subprocess.Popen: The process handle, or None if failed to start subprocess.Popen: The process handle, or None if failed to start
""" """
if not self.download_live_chat: if not self.download_live_chat_enabled:
return None return None
print(f'\n{Fore.CYAN}Starting live chat download...{Style.RESET_ALL}') print(f'\n{Fore.CYAN}Starting live chat download...{Style.RESET_ALL}')
@ -326,6 +342,118 @@ class ContentDownloader:
print(f'{Fore.RED}✗ Failed to start live chat download: {str(e)}{Style.RESET_ALL}') print(f'{Fore.RED}✗ Failed to start live chat download: {str(e)}{Style.RESET_ALL}')
return None return None
def _download_live_chat_via_irc(self, username: str, json_path: str,
max_messages: Optional[int] = None,
timeout: Optional[float] = None,
shutdown_check: Optional[callable] = None,
stream_monitor = None,
verbose: bool = False) -> bool:
"""
Simple IRC-based fallback to capture Twitch chat when GraphQL methods fail.
This writes newline-delimited JSON objects with at least: timestamp (ms),
author (dict with `name`), and `message`.
"""
try:
sock = socket.socket()
sock.connect(('irc.chat.twitch.tv', 6667))
sock.settimeout(1.0)
# Request tags & capabilities
sock.sendall(b'CAP REQ :twitch.tv/tags twitch.tv/commands twitch.tv/membership\r\n')
sock.sendall(b'PASS SCHMOOPIIE\r\n')
sock.sendall(b'NICK justinfan67420\r\n')
sock.sendall(f'JOIN #{username}\r\n'.encode('utf-8'))
messages_written = 0
start_time = time.time()
# Open file for streaming newline-delimited JSON
os.makedirs(os.path.dirname(json_path), exist_ok=True)
with open(json_path, 'w', encoding='utf-8') as out_f:
buffer = ''
while True:
# Shutdown/timeouts
if shutdown_check and shutdown_check():
break
if timeout and (time.time() - start_time) > timeout:
break
if stream_monitor:
try:
if not stream_monitor.is_user_live():
break
except Exception:
pass
try:
data = sock.recv(4096).decode('utf-8', 'ignore')
except socket.timeout:
continue
except Exception as e:
print(f'{Fore.YELLOW}⚠ IRC recv error: {e}{Style.RESET_ALL}')
break
if not data:
continue
buffer += data
lines = buffer.split('\r\n')
buffer = lines.pop() # remainder
for line in lines:
if not line:
continue
# Respond to PINGs
if line.startswith('PING'):
try:
sock.sendall(b'PONG :tmi.twitch.tv\r\n')
except Exception:
pass
continue
# Extract PRIVMSG lines
m = re.match(r'(?:@[^ ]+ )?:([^!]+)!.* PRIVMSG #[^ ]+ :(.+)', line)
if not m:
continue
author = m.group(1)
msg_text = m.group(2)
timestamp_ms = int(time.time() * 1000)
item = {
'timestamp': timestamp_ms,
'author': {'name': author},
'message': msg_text
}
out_f.write(json.dumps(item, ensure_ascii=False) + '\n')
out_f.flush()
messages_written += 1
if verbose and (messages_written % 10 == 0):
print(f'\n{Fore.GREEN}💬 {author}: {Fore.WHITE}{msg_text}{Style.RESET_ALL}')
if max_messages and messages_written >= max_messages:
break
if max_messages and messages_written >= max_messages:
break
sock.close()
if messages_written > 0:
print(f'\n{Fore.GREEN}✓ IRC fallback captured {messages_written} messages{Style.RESET_ALL}')
return True
else:
print(f'\n{Fore.RED}✗ IRC fallback captured no messages{Style.RESET_ALL}')
return False
except Exception as e:
print(f'{Fore.RED}✗ IRC fallback failed: {e}{Style.RESET_ALL}')
import traceback
traceback.print_exc()
return False
def wait_for_chat_download(self, process: Optional[subprocess.Popen], def wait_for_chat_download(self, process: Optional[subprocess.Popen],
json_path: str, timeout: int = 300) -> bool: json_path: str, timeout: int = 300) -> bool:
""" """
@ -365,6 +493,7 @@ class ContentDownloader:
max_messages: Optional[int] = None, max_messages: Optional[int] = None,
timeout: Optional[float] = None, timeout: Optional[float] = None,
shutdown_check: Optional[callable] = None, shutdown_check: Optional[callable] = None,
stream_monitor = None,
verbose: bool = False) -> bool: verbose: bool = False) -> bool:
""" """
Download live chat using chat_downloader library as fallback. Download live chat using chat_downloader library as fallback.
@ -376,6 +505,7 @@ class ContentDownloader:
max_messages: Maximum messages to download (None = unlimited) max_messages: Maximum messages to download (None = unlimited)
timeout: Stop after this many seconds (None = until stream ends) timeout: Stop after this many seconds (None = until stream ends)
shutdown_check: Optional callback function that returns True when shutdown requested shutdown_check: Optional callback function that returns True when shutdown requested
stream_monitor: Optional stream monitor to check if stream is still live
verbose: Show chat message previews verbose: Show chat message previews
Returns: Returns:
@ -386,10 +516,20 @@ class ContentDownloader:
print(f'{Fore.YELLOW} Install with: pip install chat-downloader{Style.RESET_ALL}') print(f'{Fore.YELLOW} Install with: pip install chat-downloader{Style.RESET_ALL}')
return False return False
if not self.download_live_chat: if not self.download_live_chat_enabled:
print(f'{Fore.YELLOW}⚠ downloadLiveCHAT is disabled in config{Style.RESET_ALL}') print(f'{Fore.YELLOW}⚠ downloadLiveCHAT is disabled in config{Style.RESET_ALL}')
return False return False
# If a stream monitor was provided, check that the user is currently live
if stream_monitor is not None:
try:
if not stream_monitor.is_user_live():
print(f'{Fore.YELLOW}⚠ Stream is not live; skipping chat download{Style.RESET_ALL}')
return False
except Exception as e:
# If we couldn't determine live status, continue and let chat_downloader handle it
print(f'{Fore.YELLOW}⚠ Could not determine live status: {e} - proceeding with chat download{Style.RESET_ALL}')
print(f'\n{Fore.CYAN}Starting live chat download (chat_downloader)...{Style.RESET_ALL}') print(f'\n{Fore.CYAN}Starting live chat download (chat_downloader)...{Style.RESET_ALL}')
print(f'{Fore.MAGENTA}[VERBOSE] chat_downloader library version: {ChatDownloader.__module__}{Style.RESET_ALL}') print(f'{Fore.MAGENTA}[VERBOSE] chat_downloader library version: {ChatDownloader.__module__}{Style.RESET_ALL}')
@ -401,8 +541,12 @@ class ContentDownloader:
print(f'{Fore.MAGENTA}[VERBOSE] Timeout: {timeout}s (None = unlimited){Style.RESET_ALL}') print(f'{Fore.MAGENTA}[VERBOSE] Timeout: {timeout}s (None = unlimited){Style.RESET_ALL}')
print(f'{Fore.MAGENTA}[VERBOSE] Max messages: {max_messages} (None = unlimited){Style.RESET_ALL}') print(f'{Fore.MAGENTA}[VERBOSE] Max messages: {max_messages} (None = unlimited){Style.RESET_ALL}')
# Get chat messages # Get chat messages with a small retry loop to handle transient GQL/network issues
print(f'{Fore.CYAN}Connecting to Twitch chat...{Style.RESET_ALL}') print(f'{Fore.CYAN}Connecting to Twitch chat...{Style.RESET_ALL}')
chat = None
max_attempts = 3
for attempt in range(1, max_attempts + 1):
try:
chat = self.chat_downloader.get_chat( chat = self.chat_downloader.get_chat(
stream_url, stream_url,
message_types=['text_message'], # Basic text messages message_types=['text_message'], # Basic text messages
@ -410,11 +554,40 @@ class ContentDownloader:
timeout=timeout, timeout=timeout,
max_messages=max_messages max_messages=max_messages
) )
break
except Exception as e:
# Provide a clearer, user-facing message for common failures
print(f"{Fore.YELLOW}⚠ chat_downloader attempt {attempt}/{max_attempts} failed: {str(e)}{Style.RESET_ALL}")
# On final attempt, dump traceback to help diagnose library internals
if attempt >= max_attempts:
print(f"{Fore.RED}✗ chat_downloader failed after {max_attempts} attempts. This may be caused by Twitch GraphQL changes or rate-limiting.{Style.RESET_ALL}")
print(f"{Fore.YELLOW} Try upgrading the chat-downloader package: pip install -U chat-downloader{Style.RESET_ALL}")
import traceback
traceback.print_exc()
# Try IRC fallback before giving up
print(f"{Fore.MAGENTA}[VERBOSE] Attempting IRC fallback for chat capture...{Style.RESET_ALL}")
try:
return self._download_live_chat_via_irc(username, json_path,
max_messages=max_messages,
timeout=timeout,
shutdown_check=shutdown_check,
stream_monitor=stream_monitor,
verbose=verbose)
except Exception as fallback_err:
print(f"{Fore.RED}✗ IRC fallback failed: {fallback_err}{Style.RESET_ALL}")
traceback.print_exc()
return False
else:
time.sleep(1)
continue
# The get_chat with output parameter writes to file automatically # The get_chat with output parameter writes to file automatically
# We just need to iterate to trigger the download # We just need to iterate to trigger the download
message_count = 0 message_count = 0
print(f'{Fore.CYAN}Receiving chat messages (press Ctrl+C to stop)...{Style.RESET_ALL}') last_check_time = time.time()
check_interval = 10.0 # Check if stream is still live every 10 seconds
print(f'{Fore.CYAN}Receiving chat messages (will stop when stream ends)...{Style.RESET_ALL}')
try: try:
for message in chat: for message in chat:
# Check for shutdown request # Check for shutdown request
@ -422,6 +595,19 @@ class ContentDownloader:
print(f'\n{Fore.YELLOW}⚠ Chat download stopped by shutdown request{Style.RESET_ALL}') print(f'\n{Fore.YELLOW}⚠ Chat download stopped by shutdown request{Style.RESET_ALL}')
break break
# Periodically check if stream is still live
current_time = time.time()
if stream_monitor and (current_time - last_check_time) >= check_interval:
last_check_time = current_time
try:
is_live = stream_monitor.is_user_live()
if not is_live:
print(f'\n{Fore.YELLOW}⚠ Stream ended, stopping chat download{Style.RESET_ALL}')
break
except Exception as check_error:
print(f'\n{Fore.YELLOW}⚠ Could not check stream status: {check_error}{Style.RESET_ALL}')
# Continue downloading to avoid false positives from API errors
message_count += 1 message_count += 1
# Show progress every 100 messages # Show progress every 100 messages
@ -467,6 +653,7 @@ class ContentDownloader:
def start_chat_downloader_thread(self, username: str, json_path: str, def start_chat_downloader_thread(self, username: str, json_path: str,
shutdown_check: Optional[callable] = None, shutdown_check: Optional[callable] = None,
stream_monitor = None,
verbose: bool = False) -> threading.Thread: verbose: bool = False) -> threading.Thread:
""" """
Start chat_downloader in a background thread. Start chat_downloader in a background thread.
@ -475,6 +662,7 @@ class ContentDownloader:
username: Twitch username username: Twitch username
json_path: Path to save chat JSON json_path: Path to save chat JSON
shutdown_check: Callback to check for shutdown shutdown_check: Callback to check for shutdown
stream_monitor: Optional stream monitor to check if stream is still live
verbose: Show chat previews verbose: Show chat previews
Returns: Returns:
@ -485,6 +673,7 @@ class ContentDownloader:
self.chat_thread_success = self.download_live_chat_with_chat_downloader( self.chat_thread_success = self.download_live_chat_with_chat_downloader(
username, json_path, username, json_path,
shutdown_check=shutdown_check, shutdown_check=shutdown_check,
stream_monitor=stream_monitor,
verbose=verbose verbose=verbose
) )
except Exception as e: except Exception as e:

View file

@ -9,7 +9,7 @@ import subprocess
from typing import List from typing import List
from colorama import Fore, Style from colorama import Fore, Style
from .constants import PREFIX_LIVE, PREFIX_VOD, PREFIX_CHAT, PREFIX_METADATA from .constants import PREFIX_LIVE, PREFIX_VOD, PREFIX_CHAT, PREFIX_METADATA, PREFIX_MERGED
from .utils import get_bin_path from .utils import get_bin_path
@ -28,6 +28,9 @@ class FileManager:
self.root_path = pathlib.Path(root_path) self.root_path = pathlib.Path(root_path)
self.username = username self.username = username
self.upload_cloud = config.get('uploadCloud', True) self.upload_cloud = config.get('uploadCloud', True)
self.upload_pre_merge_video = config.get('uploadPreMergeVideo', True)
self.upload_merged_video = config.get('uploadMergedVideo', True)
self.upload_chat_video = config.get('uploadChatVideo', True)
self.delete_files = config.get('deleteFiles', False) self.delete_files = config.get('deleteFiles', False)
self.clean_raw = config.get('cleanRaw', True) self.clean_raw = config.get('cleanRaw', True)
self.download_vod = config.get('downloadVOD', True) self.download_vod = config.get('downloadVOD', True)
@ -43,6 +46,128 @@ class FileManager:
self.metadata_path = self.root_path / username / "metadata" self.metadata_path = self.root_path / username / "metadata"
self.log_file = self.root_path / ".log" self.log_file = self.root_path / ".log"
def _to_rclone_relative_path(self, *parts: str) -> str:
"""Build a POSIX-style relative path for rclone --files-from."""
return pathlib.PurePosixPath(*parts).as_posix()
def _build_upload_relative_paths(self, filename_base: str) -> List[str]:
"""Build the candidate upload list relative to root_path for rclone."""
files_to_upload: List[str] = [
self._to_rclone_relative_path(self.username, 'metadata', f"{PREFIX_METADATA}{filename_base}.json"),
self._to_rclone_relative_path(self.username, 'chat', 'json', f"{PREFIX_CHAT}{filename_base}.json")
]
if self.upload_pre_merge_video:
files_to_upload.extend([
self._to_rclone_relative_path(self.username, 'video', 'raw', f"{PREFIX_LIVE}{filename_base}.ts"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_LIVE}{filename_base}.mp4"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_LIVE}{filename_base}.mp3"),
self._to_rclone_relative_path(self.username, 'video', 'raw', f"{PREFIX_VOD}{filename_base}.ts"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_VOD}{filename_base}.mp4"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_VOD}{filename_base}.mp3")
])
if self.upload_merged_video:
files_to_upload.extend([
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_MERGED}{filename_base}.mp4"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_MERGED}{filename_base}.mp3"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_MERGED}{PREFIX_VOD}{filename_base}.mp4"),
self._to_rclone_relative_path(self.username, 'video', f"{PREFIX_MERGED}{PREFIX_VOD}{filename_base}.mp3")
])
if self.upload_chat_video:
files_to_upload.append(self._to_rclone_relative_path(self.username, 'chat', f"{PREFIX_CHAT}{filename_base}.mp4"))
return files_to_upload
def _get_existing_upload_relative_paths(self, relative_paths: List[str]) -> List[str]:
"""Filter candidate upload paths to the files that actually exist."""
existing_paths: List[str] = []
for relative_path in relative_paths:
if (self.root_path / pathlib.PurePosixPath(relative_path)).exists():
existing_paths.append(relative_path)
return existing_paths
def _run_rclone_copy(self, relative_paths: List[str], description: str) -> bool:
"""Run rclone copy for a set of paths relative to root_path."""
existing_paths = self._get_existing_upload_relative_paths(relative_paths)
missing_paths = [path for path in relative_paths if path not in existing_paths]
if not existing_paths:
print(f'{Fore.RED}✗ Upload skipped: no matching files found for {description}{Style.RESET_ALL}')
for missing_path in missing_paths:
print(f'{Fore.YELLOW} Missing: {missing_path}{Style.RESET_ALL}')
return False
if missing_paths:
print(f'{Fore.YELLOW}⚠ Some configured upload files were not found and will be skipped{Style.RESET_ALL}')
for missing_path in missing_paths:
print(f'{Fore.YELLOW} Missing: {missing_path}{Style.RESET_ALL}')
print(f'{Fore.CYAN}rclone source: {self.root_path.resolve()}{Style.RESET_ALL}')
print(f'{Fore.CYAN}rclone destination: {self.rclone_path}{Style.RESET_ALL}')
print(f'{Fore.CYAN}Files queued for upload: {len(existing_paths)}{Style.RESET_ALL}')
bin_path = get_bin_path()
upload_list_path = os.path.join(bin_path, 'temp', 'upload.txt')
os.makedirs(os.path.dirname(upload_list_path), exist_ok=True)
with open(upload_list_path, 'w', encoding='utf-8', newline='\n') as f:
f.write('\n'.join(existing_paths))
f.write('\n')
try:
cmd = [
'rclone', 'copy',
str(self.root_path.resolve()),
self.rclone_path,
'--files-from', upload_list_path,
'--progress'
]
print(f'{Fore.CYAN}Running: {' '.join(cmd)}{Style.RESET_ALL}')
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
if proc.stdout:
for line in proc.stdout:
print(line, end='')
proc.wait()
return proc.returncode == 0
finally:
if os.path.exists(upload_list_path):
os.remove(upload_list_path)
def run_rclone_smoke_test(self) -> bool:
"""Create and upload a tiny metadata file to verify rclone output and configuration."""
smoke_name = 'RCLONE_SMOKE_TEST'
smoke_relative_path = self._to_rclone_relative_path(
self.username,
'metadata',
f"{PREFIX_METADATA}{smoke_name}.json"
)
smoke_file_path = self.root_path / pathlib.PurePosixPath(smoke_relative_path)
smoke_payload = {
'type': 'rclone_smoke_test',
'username': self.username
}
smoke_file_path.parent.mkdir(parents=True, exist_ok=True)
with open(smoke_file_path, 'w', encoding='utf-8') as f:
json.dump(smoke_payload, f, indent=2)
print(f'{Fore.CYAN}Created smoke-test file: {smoke_file_path}{Style.RESET_ALL}')
try:
result = self._run_rclone_copy([smoke_relative_path], 'rclone smoke test')
if result:
print(f'{Fore.GREEN}✓ Rclone smoke test completed{Style.RESET_ALL}')
else:
print(f'{Fore.RED}✗ Rclone smoke test failed{Style.RESET_ALL}')
return result
finally:
if smoke_file_path.exists():
smoke_file_path.unlink()
def initialize_directories(self) -> None: def initialize_directories(self) -> None:
"""Create all necessary directory structures.""" """Create all necessary directory structures."""
for path in [self.raw_path, self.video_path, self.chat_json_path, for path in [self.raw_path, self.video_path, self.chat_json_path,
@ -118,52 +243,22 @@ class FileManager:
if notification_callback: if notification_callback:
notification_callback(f'☁ Uploading - {filename_base}', 'Uploading files to cloud storage') notification_callback(f'☁ Uploading - {filename_base}', 'Uploading files to cloud storage')
# Create list of files to upload files_to_upload = self._build_upload_relative_paths(filename_base)
bin_path = get_bin_path()
upload_list_path = os.path.join(bin_path, 'temp', 'upload.txt')
# Ensure temp directory exists
os.makedirs(os.path.dirname(upload_list_path), exist_ok=True)
files_to_upload = [
f"{PREFIX_LIVE}{filename_base}.ts",
f"{PREFIX_LIVE}{filename_base}.mp4",
f"{PREFIX_LIVE}{filename_base}.mp3",
f"{PREFIX_VOD}{filename_base}.ts",
f"{PREFIX_VOD}{filename_base}.mp4",
f"{PREFIX_VOD}{filename_base}.mp3",
f"{PREFIX_METADATA}{filename_base}.json",
f"{PREFIX_CHAT}{filename_base}.json",
f"{PREFIX_CHAT}{filename_base}.mp4"
]
with open(upload_list_path, 'w') as f:
f.write('\n'.join(files_to_upload))
# Run rclone
try: try:
result = subprocess.call([ result = self._run_rclone_copy(files_to_upload, f'archive batch {filename_base}')
'rclone', 'copy',
str(self.root_path.resolve()),
self.rclone_path,
'--include-from', upload_list_path
])
# Clean up upload list if result:
if os.path.exists(upload_list_path):
os.remove(upload_list_path)
if result == 0:
print(f'{Fore.GREEN}✓ Upload complete{Style.RESET_ALL}') print(f'{Fore.GREEN}✓ Upload complete{Style.RESET_ALL}')
if notification_callback: if notification_callback:
notification_callback(f'✓ Upload Success - {filename_base}', 'All files uploaded successfully') notification_callback(f'✓ Upload Success - {filename_base}', 'All files uploaded successfully')
return True return True
else:
print(f'{Fore.RED}✗ Upload failed (exit code: {result}){Style.RESET_ALL}') print(f'{Fore.RED}✗ Upload failed{Style.RESET_ALL}')
print(f'{Fore.YELLOW}Files preserved locally due to upload failure{Style.RESET_ALL}') print(f'{Fore.YELLOW}Files preserved locally due to upload failure{Style.RESET_ALL}')
if notification_callback: if notification_callback:
notification_callback(f'✗ Upload Failed - {filename_base}', notification_callback(f'✗ Upload Failed - {filename_base}',
f'Upload failed with code {result}. Files preserved locally.') 'Upload failed. Files preserved locally. Check rclone output above.')
return False return False
except Exception as e: except Exception as e:
@ -175,6 +270,8 @@ class FileManager:
""" """
Delete local archive files after successful upload. Delete local archive files after successful upload.
Only deletes files that were configured to be uploaded.
Args: Args:
filename_base: Base filename (without prefixes/extensions) filename_base: Base filename (without prefixes/extensions)
live_raw_path: Path to live raw file live_raw_path: Path to live raw file
@ -191,14 +288,15 @@ class FileManager:
files_to_delete: List[str] = [] files_to_delete: List[str] = []
# Live files # Live files (only if pre-merge videos are uploaded)
if self.upload_pre_merge_video:
if not self.clean_raw and os.path.exists(live_raw_path): if not self.clean_raw and os.path.exists(live_raw_path):
files_to_delete.append(live_raw_path) files_to_delete.append(live_raw_path)
if os.path.exists(live_proc_path): if os.path.exists(live_proc_path):
files_to_delete.append(live_proc_path) files_to_delete.append(live_proc_path)
# VOD files # VOD files (only if pre-merge videos are uploaded)
if self.download_vod: if self.download_vod and self.upload_pre_merge_video:
vod_raw = self.raw_path / f"{PREFIX_VOD}{filename_base}.ts" vod_raw = self.raw_path / f"{PREFIX_VOD}{filename_base}.ts"
vod_mp4 = self.video_path / f"{PREFIX_VOD}{filename_base}.mp4" vod_mp4 = self.video_path / f"{PREFIX_VOD}{filename_base}.mp4"
vod_mp3 = self.video_path / f"{PREFIX_VOD}{filename_base}.mp3" vod_mp3 = self.video_path / f"{PREFIX_VOD}{filename_base}.mp3"
@ -210,17 +308,37 @@ class FileManager:
if vod_mp3.exists(): if vod_mp3.exists():
files_to_delete.append(str(vod_mp3)) files_to_delete.append(str(vod_mp3))
# Merged video files (only if merged videos are uploaded)
if self.upload_merged_video:
merged_live_mp4 = self.video_path / f"{PREFIX_MERGED}{filename_base}.mp4"
merged_live_mp3 = self.video_path / f"{PREFIX_MERGED}{filename_base}.mp3"
merged_vod_mp4 = self.video_path / f"{PREFIX_MERGED}{PREFIX_VOD}{filename_base}.mp4"
merged_vod_mp3 = self.video_path / f"{PREFIX_MERGED}{PREFIX_VOD}{filename_base}.mp3"
if merged_live_mp4.exists():
files_to_delete.append(str(merged_live_mp4))
if merged_live_mp3.exists():
files_to_delete.append(str(merged_live_mp3))
if merged_vod_mp4.exists():
files_to_delete.append(str(merged_vod_mp4))
if merged_vod_mp3.exists():
files_to_delete.append(str(merged_vod_mp3))
# Chat files # Chat files
if self.download_chat: if self.download_chat:
chat_json = self.chat_json_path / f"{PREFIX_CHAT}{filename_base}.json" chat_json = self.chat_json_path / f"{PREFIX_CHAT}{filename_base}.json"
chat_mp4 = self.chat_mp4_path / f"{PREFIX_CHAT}{filename_base}.mp4"
# Always delete JSON (it's always uploaded)
if chat_json.exists(): if chat_json.exists():
files_to_delete.append(str(chat_json)) files_to_delete.append(str(chat_json))
# Only delete chat MP4 if chat videos are uploaded
if self.upload_chat_video:
chat_mp4 = self.chat_mp4_path / f"{PREFIX_CHAT}{filename_base}.mp4"
if chat_mp4.exists(): if chat_mp4.exists():
files_to_delete.append(str(chat_mp4)) files_to_delete.append(str(chat_mp4))
# Metadata files # Metadata files (always uploaded)
if self.download_metadata: if self.download_metadata:
metadata = self.metadata_path / f"{PREFIX_METADATA}{filename_base}.json" metadata = self.metadata_path / f"{PREFIX_METADATA}{filename_base}.json"
if metadata.exists(): if metadata.exists():

View file

@ -6,7 +6,7 @@ import os
import subprocess import subprocess
from colorama import Fore, Style from colorama import Fore, Style
from .utils import detect_hardware_acceleration, get_hwaccel_encoder from .utils import detect_hardware_acceleration, get_hwaccel_encoder, resolve_hwaccel_type
class StreamProcessor: class StreamProcessor:
@ -36,38 +36,80 @@ class StreamProcessor:
config.get('ffmpeg_hwaccel', 'auto'), config.get('ffmpeg_hwaccel', 'auto'),
os_type os_type
) )
self.hwaccel_type = resolve_hwaccel_type(self.hwaccel_type, os_type)
def process_raw_stream(self, raw_path: str, output_path: str) -> None: def process_raw_stream(self, raw_path: str, output_path: str) -> bool:
""" """
Process raw .ts file into mp4/mp3 using ffmpeg. Process raw .ts file into mp4/mp3 using ffmpeg.
Args: Args:
raw_path: Path to the raw .ts file raw_path: Path to the raw .ts file
output_path: Path for the processed output file output_path: Path for the processed output file
Returns:
bool: True when conversion succeeded, False otherwise
""" """
if not os.path.exists(raw_path): if not os.path.exists(raw_path):
print(f'{Fore.YELLOW}⚠ Raw file not found, skipping processing{Style.RESET_ALL}') print(f'{Fore.YELLOW}⚠ Raw file not found, skipping processing{Style.RESET_ALL}')
return return False
if self.only_raw: if self.only_raw:
print(f'{Fore.CYAN}Keeping raw .ts file (onlyRaw mode){Style.RESET_ALL}') print(f'{Fore.CYAN}Keeping raw .ts file (onlyRaw mode){Style.RESET_ALL}')
return return False
print(f'{Fore.YELLOW}Processing raw stream file...{Style.RESET_ALL}') print(f'{Fore.YELLOW}Processing raw stream file...{Style.RESET_ALL}')
# Build ffmpeg command based on quality # Build ffmpeg command based on quality
if self.quality == 'audio_only': if self.quality == 'audio_only':
self._process_audio(raw_path, output_path) result = self._process_audio(raw_path, output_path)
else: else:
self._process_video(raw_path, output_path) result = self._process_video(raw_path, output_path)
if result:
print(f'{Fore.GREEN}✓ Stream processed successfully{Style.RESET_ALL}') print(f'{Fore.GREEN}✓ Stream processed successfully{Style.RESET_ALL}')
else:
print(f'{Fore.RED}✗ Stream processing failed{Style.RESET_ALL}')
def _process_audio(self, raw_path: str, output_path: str) -> None: return result
def _run_ffmpeg_command(self, cmd: list, output_path: str) -> bool:
"""Run FFmpeg while streaming its output to the terminal."""
print(f'{Fore.CYAN}Running FFmpeg: {' '.join(cmd)}{Style.RESET_ALL}')
process = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding='utf-8',
errors='replace'
)
if process.stdout:
for line in process.stdout:
print(line, end='')
result = process.wait()
if result != 0:
print(f'{Fore.RED}✗ FFmpeg exited with code: {result}{Style.RESET_ALL}')
return False
if not os.path.exists(output_path):
print(f'{Fore.RED}✗ FFmpeg did not create output: {output_path}{Style.RESET_ALL}')
return False
if os.path.getsize(output_path) == 0:
print(f'{Fore.RED}✗ FFmpeg created an empty output file: {output_path}{Style.RESET_ALL}')
return False
return True
def _process_audio(self, raw_path: str, output_path: str) -> bool:
"""Process audio-only stream.""" """Process audio-only stream."""
# Audio-only conversion with modern AAC encoding # Audio-only conversion with modern AAC encoding
cmd = [ cmd = [
self.ffmpeg_path, self.ffmpeg_path,
'-y',
'-i', raw_path, '-i', raw_path,
'-vn', # No video '-vn', # No video
'-c:a', self.ffmpeg_audio_codec, '-c:a', self.ffmpeg_audio_codec,
@ -85,14 +127,9 @@ class StreamProcessor:
cmd.extend(['-movflags', '+faststart']) cmd.extend(['-movflags', '+faststart'])
cmd.append(output_path) cmd.append(output_path)
return self._run_ffmpeg_command(cmd, output_path)
# Run FFmpeg def _process_video(self, raw_path: str, output_path: str) -> bool:
if self.ffmpeg_progress:
subprocess.call(cmd)
else:
subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
def _process_video(self, raw_path: str, output_path: str) -> None:
"""Process video stream.""" """Process video stream."""
cmd = [ cmd = [
self.ffmpeg_path, self.ffmpeg_path,
@ -135,12 +172,7 @@ class StreamProcessor:
cmd.extend(['-movflags', '+faststart']) cmd.extend(['-movflags', '+faststart'])
cmd.append(output_path) cmd.append(output_path)
return self._run_ffmpeg_command(cmd, output_path)
# Run FFmpeg
if self.ffmpeg_progress:
subprocess.call(cmd)
else:
subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
def build_chat_output_args(self) -> str: def build_chat_output_args(self) -> str:
""" """

View file

@ -7,6 +7,8 @@ import subprocess
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
from colorama import Fore, Style from colorama import Fore, Style
from .utils import get_env_value
class StreamRecorder: class StreamRecorder:
"""Handles live stream recording using streamlink.""" """Handles live stream recording using streamlink."""
@ -68,7 +70,7 @@ class StreamRecorder:
print(f'{Fore.YELLOW} Consider disabling streamlink_ttvlol in config or using alternative methods{Style.RESET_ALL}') print(f'{Fore.YELLOW} Consider disabling streamlink_ttvlol in config or using alternative methods{Style.RESET_ALL}')
# Add authentication if available # Add authentication if available
oauth_token = os.getenv("OAUTH-PRIVATE-TOKEN", "") oauth_token = get_env_value("OAUTH-PRIVATE-TOKEN", "OAUTH_PRIVATE_TOKEN", default="")
if oauth_token and oauth_token != "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx": if oauth_token and oauth_token != "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx":
cmd.extend(['--twitch-api-header', f'Authorization=OAuth {oauth_token}']) cmd.extend(['--twitch-api-header', f'Authorization=OAuth {oauth_token}'])

Some files were not shown because too many files have changed in this diff Show more