Refactor downloader and file manager for improved rclone integration and add healthcheck and smoke test options
- Renamed download flags in ContentDownloader for clarity. - Enhanced FileManager with methods to build upload paths and verify existing files for rclone uploads. - Updated StreamProcessor to return success status for stream processing. - Added rclone smoke test and healthcheck functions to validate configuration and tool availability. - Improved environment variable handling with a utility function. - Updated TwitchArchive to incorporate new rclone verification and processing logic. - Added unit tests for new functionality and refactored existing tests for clarity and coverage. Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
parent
e92f36474a
commit
f97e0200d6
23 changed files with 1013 additions and 289 deletions
67
README.md
67
README.md
|
|
@ -26,6 +26,73 @@ Notes:
|
|||
|
||||
Python script to check, download live stream, VOD, chat and upload them to any cloud storage supported by rclone.
|
||||
|
||||
## Docker
|
||||
|
||||
This repository now includes a Python-only container setup for the archiver. The dotnet subapp is not part of this container flow.
|
||||
|
||||
Files:
|
||||
|
||||
- `docker/python.Dockerfile`: production image for the Python archiver
|
||||
- `docker-compose.yml`: deployment-oriented compose file
|
||||
- `docker-compose.override.yml`: local development and testing override
|
||||
- `.env.production`: production container and app environment template
|
||||
- `.env.development`: development container and app environment template
|
||||
- `dockerstart.bat`: Windows helper to run the container like the old batch launcher
|
||||
|
||||
### Container layout
|
||||
|
||||
- Mount your external archive folder to `/app/archive`
|
||||
- Mount your external config folder to `/app/config`
|
||||
- Put your `rclone.conf` file at `/app/config/rclone.conf` on the mounted host path
|
||||
- The container exports `RCLONE_CONFIG=/app/config/rclone.conf`, so rclone will use that file automatically
|
||||
|
||||
### Production deployment
|
||||
|
||||
1. Edit `.env.production` with your image name, Twitch credentials, bind paths, and default arguments.
|
||||
2. Place your streamer JSON files and `rclone.conf` in the mounted config folder.
|
||||
3. Start the container:
|
||||
|
||||
```powershell
|
||||
docker compose --env-file .env.production up -d
|
||||
```
|
||||
|
||||
4. Follow logs:
|
||||
|
||||
```powershell
|
||||
docker compose --env-file .env.production logs -f twitch-archive
|
||||
```
|
||||
|
||||
### Development and local testing
|
||||
|
||||
The override compose file builds the image locally and mounts the repository for faster iteration.
|
||||
|
||||
Start it with:
|
||||
|
||||
```powershell
|
||||
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml up --build
|
||||
```
|
||||
|
||||
Run a one-off manual test for another streamer:
|
||||
|
||||
```powershell
|
||||
docker compose --env-file .env.development -f docker-compose.yml -f docker-compose.override.yml run --rm twitch-archive python twitch-archive.py -u hackerling --verbose
|
||||
```
|
||||
|
||||
Or use the Windows helper:
|
||||
|
||||
```powershell
|
||||
.\dockerstart.bat vinesauce --verbose
|
||||
```
|
||||
|
||||
That batch launcher mirrors the old pattern and expands to a compose `run` command, so you can test any streamer manually.
|
||||
|
||||
### Healthcheck and smoke tests
|
||||
|
||||
- Container healthcheck command: `python twitch-archive.py --healthcheck -u vinesauce`
|
||||
- Rclone smoke test command: `python twitch-archive.py -u vinesauce --rclone-smoke-test`
|
||||
|
||||
The healthcheck verifies config loading plus `streamlink`, `ffmpeg`, `TwitchDownloaderCLI`, and `rclone` availability. The smoke test writes a tiny file, uploads it with the configured rclone remote, and prints the live rclone output into the container logs.
|
||||
|
||||
## ⚡ FFmpeg 8.0 Enhanced
|
||||
Now with FFmpeg 8.0+ support featuring hardware acceleration and performance improvements!
|
||||
- **5-10x faster encoding** with NVIDIA, Intel, or AMD GPUs
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue