diff --git a/README.md b/README.md index f38cd83..52feb2d 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Twitch Archive Inspired by https://github.com/EnterGin/Auto-Stream-Recording-Twitch -Python script to monitor a twitch channel: +Python script to check, download live stream, VOD, chat and upload them to any cloud service supported by rclone. ## Requirements - [Python 3](https://www.python.org/downloads/) - [Streamlink](https://github.com/streamlink/streamlink) ## Getting started 1. Install Python 3 2. Install Streamlink -3. If you want to upload to a remote service using rclone, [configure it](https://rclone.org/docs/#configure) (Doesnt need to download, the `rclone.exe` is avalible in [tools/rclone.exe](https://github.com/piero0920/Twitch-Archive/blob/main/tools/rclone.exe)). +3. If you want to upload to any cloud service using rclone, [configure rclone](https://rclone.org/docs/#configure) (Doesnt need to download, the `rclone.exe` is avalible in [bin/rclone.exe](https://github.com/piero0920/Twitch-Archive/blob/main/bin/rclone.exe)). 4. `git clone https://github.com/piero0920/Twitch-Archive.git` 5. `cd Twitch-Archive` 6. `pip install -r requirements.txt` @@ -18,7 +18,7 @@ CLIENT-ID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx CLIENT-SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OAUTH-PRIVATE-TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` -8. if you want to enable/disable more available options, open `twitch-archive.py` +8. if you want to enable/disable more available options, edit `twitch-archive.py` 9. run `Python twitch-archive.py` or for multiple streamers `Python twitch-archive.py -u streamer` ## Features - Auto records the live stream @@ -27,70 +27,3 @@ OAUTH-PRIVATE-TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Downloads the metadata of the VOD - Uploads them to the Cloud - Notifies you through Gmail of the progress - -## Explanation -### Record live stream: -Using [Streamlink](https://streamlink.github.io/) downloads the `.ts` file from the live stream since the script starred running, if the script was running since before the beginning of the stream, it will record everything and save it to: `/root_path/streamer_username/video/recorded/LIVE_yyyymmdd_hhmmss.ts` - -This option will need a oauth-token to record without ADS. -After the stream ended, the `.ts` file will be processed to `.mp4` file. using [ffmpeg](https://ffmpeg.org/) and save it to: `/root_path/streamer_username/video/processed/LIVE_yyyymmdd_hhmmss.mp4` -> [Here's](https://youtu.be/1MBsUoFGuls) a tutorial that shows you how to get the oauth-token. -### Download VOD -Using [Streamlink](https://streamlink.github.io/) downloads the `.ts` file from the VOD, the download is faster and is available to get the unmuted segments before twitch mutes the entire VOD. it will be save it to: `/root_path/streamer_username/video/recorded/VOD_yyyymmdd_hhmmss.ts` - -Then the `.ts` file will be processed to `.mp4` file. using [ffmpeg](https://ffmpeg.org/) and save it to: `/root_path/streamer_username/video/processed/VOD_yyyymmdd_hhmmss.mp4` -This option will Download the latest public VOD, if the streamer hasn't published or has hide the VOD, the current VOD will no be downloaded instead the previous VOD. -### Download and render chat - Using [TwitchDownloaderCLI](https://github.com/lay295/TwitchDownloader) downloads the `.json` file of the chat logs from the VOD, ands saves it to: `/root_path/streamer_username/chat/json/CHAT_yyyymmdd_hhmmss.json` - - Then after the `.json` file is downloaded using again [TwitchDownloaderCLI](https://github.com/lay295/TwitchDownloader) renders it to a viewable `.mp4` file and saves it to: - `/root_path/streamer_username/chat/rendered/CHAT_yyyymmdd_hhmmss.mp4` - - If you want to change the rendered settings go to [chat.bat](https://github.com/piero0920/Twitch-Archive/blob/main/tools/chat.bat) file and change the parameters: - `--background-color #FF111111 -w 500 -h 1080 --outline true -f Arial --font-size 22 --update-rate 1.0` -### Download metadata -Using a simple api request downloads the `.json` metadata of the latest VOD and saves it to: -`/root_path/streamer_username/metadata/metada_yyyymmdd_hhmmss.json` -### Upload to the cloud -Using [rclone](https://rclone.org/) after everything being downloaded and rendered , it will upload every file from the `root_path/streamer` folder to any cloud service supported by rclone such as [Google Drive, Mega, One Drive, etc.](https://rclone.org/overview/#features) - The destination path to where it will be uploaded it has to be stated in the `.env` file. - Example: - ```env -remote=GoogleDrive:Archive - ``` - ### Gmail notification - Using the python library [smtplib](https://docs.python.org/3/library/smtplib.html) sends gmail messages to desire gmail. To be run correctly the emails have to be stated in the `.env` file. - Example: - ```.env - SENDER=example@gmail.com - PWD=xxxxxxxxxxxxxxxx - RECEIVER=example@gmail.com - ``` - the `PWD` env is NOT your normal password is a 16 character password generated by google. - > [Here's](https://stackoverflow.com/a/73214197) a well documented way of how to get your PWD. - -### Seccion finished -After every option if enabled is done it will go back to check again. -Here's an example of how a regular PATH will look like. -``` -root_path -└───username - ├───chat - │ ├───json - │ │ CHAT_20221203_06h40m41s.json - │ │ - │ └───rendered - │ CHAT_20221203_06h40m41s.mp4 - │ - ├───metadata - │ metadata_20221203_06h40m41s.json - │ - └───video - ├───processed - │ LIVE_20221203_06h40m41s.mp4 - │ VOD_20221203_06h40m41s.mp4 - │ - └───recorded - LIVE_20221203_06h40m41s.ts - VOD_20221203_06h40m41s.ts -``` diff --git a/bin/TwitchDownloaderCLI b/bin/TwitchDownloaderCLI new file mode 100644 index 0000000..28bab2f --- /dev/null +++ b/bin/TwitchDownloaderCLI @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9028d9e24f43062e1466c2240dceee557219bc65f30f94c8fdcb6a7f1a0aac0c +size 51824214 diff --git a/bin/TwitchDownloaderCLI.exe b/bin/TwitchDownloaderCLI.exe new file mode 100644 index 0000000..5a699de --- /dev/null +++ b/bin/TwitchDownloaderCLI.exe @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab6817895a1b8a4f35f3165b94f2d724fd5a07422a533bd0d6256708fe773766 +size 51457162 diff --git a/tools/chat.bat b/bin/chat.bat similarity index 100% rename from tools/chat.bat rename to bin/chat.bat diff --git a/tools-ubuntu/chat.sh b/bin/chat.sh old mode 100755 new mode 100644 similarity index 66% rename from tools-ubuntu/chat.sh rename to bin/chat.sh index 0b52df2..d88b0bd --- a/tools-ubuntu/chat.sh +++ b/bin/chat.sh @@ -1,2 +1,2 @@ ./tools-ubuntu/TwitchDownloaderCLI -m ChatDownload --id $1 -o $2 --embed-emotes -./tools-ubuntu/TwitchDownloaderCLI -m ChatRender -i $2 -o $3 --background-color "#FF111111" -w 500 -h 1080 --outline true -f Arial --font-size 22 --update-rate 1.0 --ffmpeg-path ./tools-ubuntu/ffmpeg --temp-path ./tools-ubuntu/temp \ No newline at end of file +./tools-ubuntu/TwitchDownloaderCLI -m ChatRender -i $2 -o $3 --background-color "#FF111111" -w 500 -h 1080 --outline true -f Arial --font-size 22 --update-rate 1.0 --ffmpeg-path ./bin/ffmpeg --temp-path ./bin/temp \ No newline at end of file diff --git a/bin/ffmpeg b/bin/ffmpeg new file mode 100644 index 0000000..fb69ebc --- /dev/null +++ b/bin/ffmpeg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ea58083710f63bf920b16c7d5d24ae081e7d731f57a656fed11af0410d4eb48 +size 77173696 diff --git a/bin/ffmpeg.exe b/bin/ffmpeg.exe new file mode 100644 index 0000000..afc749c --- /dev/null +++ b/bin/ffmpeg.exe @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98094b9736637916bd98cc8ee7546d9d3638763545f153c6daa9dcf87b43d10e +size 78925824 diff --git a/tools/rclone.exe b/bin/rclone.exe similarity index 100% rename from tools/rclone.exe rename to bin/rclone.exe diff --git a/tools/upload.bat b/bin/upload.bat similarity index 82% rename from tools/upload.bat rename to bin/upload.bat index ec792c2..6561dcc 100644 --- a/tools/upload.bat +++ b/bin/upload.bat @@ -3,4 +3,4 @@ set root_path=%1 set user=%2 FOR /F "eol=# tokens=*" %%i in (%~dp0\..\.env) do SET %%i CD %~dp0 -rclone.exe copy ../%root_path%/%user% %remote%/%root_path%/%user% --progress --drive-chunk-size 512M \ No newline at end of file +rclone.exe copy ../%root_path%/%user% %remote%/%root_path%/%user% --progress \ No newline at end of file diff --git a/tools-ubuntu/upload.sh b/bin/upload.sh old mode 100755 new mode 100644 similarity index 100% rename from tools-ubuntu/upload.sh rename to bin/upload.sh diff --git a/extra.md b/extra.md new file mode 100644 index 0000000..d98477a --- /dev/null +++ b/extra.md @@ -0,0 +1,74 @@ +- install and configure rclone +``` +cd tools-ubuntu +sudo chmod +x chat.sh +sudo chmod +x upload.sh +sudo chmod +x TwitchDownloaderCLI +sudo chmod +x ffmpeg +``` +## Explanation +### Record live stream: +Using [Streamlink](https://streamlink.github.io/) downloads the `.ts` file from the live stream since the script starred running, if the script was running since before the beginning of the stream, it will record everything and save it to: `/root_path/streamer_username/video/recorded/LIVE_yyyymmdd_hhmmss.ts` + +This option will need a oauth-token to record without ADS. +After the stream ended, the `.ts` file will be processed to `.mp4` file. using [ffmpeg](https://ffmpeg.org/) and save it to: `/root_path/streamer_username/video/processed/LIVE_yyyymmdd_hhmmss.mp4` +> [Here's](https://youtu.be/1MBsUoFGuls) a tutorial that shows you how to get the oauth-token. +### Download VOD +Using [Streamlink](https://streamlink.github.io/) downloads the `.ts` file from the VOD, the download is faster and is available to get the unmuted segments before twitch mutes the entire VOD. it will be save it to: `/root_path/streamer_username/video/recorded/VOD_yyyymmdd_hhmmss.ts` + +Then the `.ts` file will be processed to `.mp4` file. using [ffmpeg](https://ffmpeg.org/) and save it to: `/root_path/streamer_username/video/processed/VOD_yyyymmdd_hhmmss.mp4` +This option will Download the latest public VOD, if the streamer hasn't published or has hide the VOD, the current VOD will no be downloaded instead the previous VOD. +### Download and render chat + Using [TwitchDownloaderCLI](https://github.com/lay295/TwitchDownloader) downloads the `.json` file of the chat logs from the VOD, ands saves it to: `/root_path/streamer_username/chat/json/CHAT_yyyymmdd_hhmmss.json` + + Then after the `.json` file is downloaded using again [TwitchDownloaderCLI](https://github.com/lay295/TwitchDownloader) renders it to a viewable `.mp4` file and saves it to: + `/root_path/streamer_username/chat/rendered/CHAT_yyyymmdd_hhmmss.mp4` + + If you want to change the rendered settings go to [chat.bat](https://github.com/piero0920/Twitch-Archive/blob/main/main/chat.bat) file and change the parameters: + `--background-color #FF111111 -w 500 -h 1080 --outline true -f Arial --font-size 22 --update-rate 1.0` +### Download metadata +Using a simple api request downloads the `.json` metadata of the latest VOD and saves it to: +`/root_path/streamer_username/metadata/metada_yyyymmdd_hhmmss.json` +### Upload to the cloud +Using [rclone](https://rclone.org/) after everything being downloaded and rendered , it will upload every file from the `root_path/streamer` folder to any cloud service supported by rclone such as [Google Drive, Mega, One Drive, etc.](https://rclone.org/overview/#features) + The destination path to where it will be uploaded it has to be stated in the `.env` file. + Example: + ```env +remote=GoogleDrive:Archive + ``` + ### Gmail notification + Using the python library [smtplib](https://docs.python.org/3/library/smtplib.html) sends gmail messages to desire gmail. To be run correctly the emails have to be stated in the `.env` file. + Example: + ```.env + SENDER=example@gmail.com + PWD=xxxxxxxxxxxxxxxx + RECEIVER=example@gmail.com + ``` + the `PWD` env is NOT your normal password is a 16 character password generated by google. + > [Here's](https://stackoverflow.com/a/73214197) a well documented way of how to get your PWD. + +### Seccion finished +After every option if enabled is done it will go back to check again. +Here's an example of how a regular PATH will look like. +``` +root_path +└───username + ├───chat + │ ├───json + │ │ CHAT_20221203_06h40m41s.json + │ │ + │ └───rendered + │ CHAT_20221203_06h40m41s.mp4 + │ + ├───metadata + │ metadata_20221203_06h40m41s.json + │ + └───video + ├───processed + │ LIVE_20221203_06h40m41s.mp4 + │ VOD_20221203_06h40m41s.mp4 + │ + └───recorded + LIVE_20221203_06h40m41s.ts + VOD_20221203_06h40m41s.ts +``` diff --git a/linux.md b/linux.md deleted file mode 100644 index b09a55b..0000000 --- a/linux.md +++ /dev/null @@ -1,8 +0,0 @@ -- install and configure rclone -``` -cd tools-ubuntu -sudo chmod +x chat.sh -sudo chmod +x upload.sh -sudo chmod +x TwitchDownloaderCLI -sudo chmod +x ffmpeg -``` \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index beb33cc..6d5fe5d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,5 @@ colorama==0.4.6 python-dotenv==0.21.0 +python_dateutil==2.8.2 pytz==2022.6 -requests==2.28.1 \ No newline at end of file +requests==2.28.1 diff --git a/tools-ubuntu/TwitchDownloaderCLI b/tools-ubuntu/TwitchDownloaderCLI deleted file mode 100755 index d347610..0000000 Binary files a/tools-ubuntu/TwitchDownloaderCLI and /dev/null differ diff --git a/tools-ubuntu/ffmpeg b/tools-ubuntu/ffmpeg deleted file mode 100755 index 13e56c8..0000000 Binary files a/tools-ubuntu/ffmpeg and /dev/null differ diff --git a/tools/TwitchDownloaderCLI.exe b/tools/TwitchDownloaderCLI.exe deleted file mode 100644 index 50f0653..0000000 Binary files a/tools/TwitchDownloaderCLI.exe and /dev/null differ diff --git a/tools/ffmpeg.exe b/tools/ffmpeg.exe deleted file mode 100644 index af99a49..0000000 Binary files a/tools/ffmpeg.exe and /dev/null differ diff --git a/linux-twitch-archive.py b/twitch-archive similarity index 79% rename from linux-twitch-archive.py rename to twitch-archive index a21b693..6b8e720 100644 --- a/linux-twitch-archive.py +++ b/twitch-archive @@ -1,4 +1,4 @@ -import requests, os, time, json, sys, subprocess, getopt, smtplib +import requests, os, time, json, sys, subprocess, getopt, smtplib, pathlib from colorama import Fore, Style from datetime import datetime, timedelta from pytz import timezone @@ -29,7 +29,7 @@ class TwitchArchive: def run(self): print('Twitch-Archive') print('Configuration:') - print(f'Root path: {Fore.GREEN}{self.root_path}{Style.RESET_ALL}') + print(f'Root path: {Fore.GREEN}' + str(pathlib.Path(self.root_path).resolve()) + f'{Style.RESET_ALL}') print(f'Timezone: {Fore.GREEN}{self.timezone}{Style.RESET_ALL}') print(f'Refresh rate: {Fore.GREEN} {str(self.refresh)}{Style.RESET_ALL}') if self.notifications == 1: print(f'Email notifications: {Fore.GREEN}Enabled{Style.RESET_ALL}') @@ -41,8 +41,9 @@ class TwitchArchive: if self.downloadCHAT == 1: print(f'Chat downloading {Fore.GREEN}Enabled{Style.RESET_ALL}') else: print(f'Chat downloading: {Fore.RED}Disabled{Style.RESET_ALL}') if self.uploadCloud == 1: print(f'Upload to Google Drive: {Fore.GREEN}Enabled{Style.RESET_ALL}') - else: print(f'Upload to Google Drive: {Fore.RED}Disabled{Style.RESET_ALL}') + else: print(f'Upload to cloud service: {Fore.RED}Disabled{Style.RESET_ALL}') if self.deleteFiles == 1: print(f'{Fore.RED}'+'\033[1m'+f'CAREFUL FILES ARE CONFIGURATED TO BE DELETED{Style.RESET_ALL}') + else: print(f'{Fore.GREEN}'+'\033[1m'+f'Files will NOT be deleted{Style.RESET_ALL}') if self.uploadCloud == 0 and self.deleteFiles == 1: print(f'{Fore.RED}'+'\033[1m'+f'FILES WILL BE DELETED AND NO UPLOADED {Style.RESET_ALL}{Fore.GREEN}\n"CTRL + C"{Style.RESET_ALL}{Fore.RED}'+'\033[1m'+f' TO STOP AND CHANGED CONFIGURATION{Style.RESET_ALL}') self.oauth_token = self.get_oauth_token() @@ -73,13 +74,13 @@ class TwitchArchive: os.makedirs(stream_dir_path) print('Fixing ' + recorded_filename + '.') try: - subprocess.call(['./tools-ubuntu/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) + subprocess.call(['./bin/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) except Exception as e: print(e) elif(os.path.exists(os.path.join(stream_dir_path, f)) is False): print('Fixing ' + recorded_filename + '.') try: - subprocess.call(['./tools-ubuntu/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) + subprocess.call(['./bin/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) except Exception as e: print(e) except Exception as e: @@ -160,7 +161,7 @@ class TwitchArchive: msg['From'] = sender msg['To'] = receiver msg['Subject'] = self.username + " _ " + subject - body = "Current seccion is for " + self.username + "\n\n" + content + body = "Current seccion is for " + self.username + "\n\n\n\n" + content msg.attach(MIMEText((body), 'plain')) server = smtplib.SMTP('smtp.gmail.com', 587) server.starttls() @@ -187,7 +188,8 @@ class TwitchArchive: live_date_plus = live_date + timedelta(minutes=2) present_datetime = datetime.now(timezone(self.timezone)).strftime("%Y%m%d_%Hh%Mm%Ss") raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename recorded_filename = os.path.join(self.recorded_path, live_filename) # start streamlink process subprocess.call(["streamlink", '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segments), "--hls-live-restart", "--twitch-disable-hosting", "twitch.tv/" + self.username, self.quality, "--retry-streams", str(self.refresh)] + self.debug_cmd + ["-o", recorded_filename]) @@ -204,85 +206,81 @@ class TwitchArchive: vod_id = vodsinfodic["data"][0]["id"] created_at = vodsinfodic["data"][0]["created_at"] created_at = self.toTZ(created_at) - print(created_at + ' date formated') raw_filename = created_at + ".ts" - live_filename = "LIVE_" + created_at + ".ts" + live_filename = "LIVE_" + raw_filename raw_vod_filename = "VOD_" + raw_filename - if self.downloadMETADATA == 1: - metadata_filename = "metadata_" + created_at + ".json" - with open(os.path.join(self.metadata_path, metadata_filename), 'w', encoding='utf-8') as f: - json.dump(vodsinfodic["data"][0], f, ensure_ascii=False, indent=4) try: os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) except Exception as e: raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) - print(e) - print('first exception as e\nAn error has occurred. VOD and chat will not be downloaded. Please check them manually.') - self.sendNotif('ERROR', 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n ' + e) + print('first exception as e\nAn error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) + self.sendNotif('ERROR - '+ present_datetime, 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n ' + e) + if self.downloadMETADATA == 1: + metadata_filename = "metadata_" + created_at + ".json" + self.sendNotif('Metadata - ' + created_at,'Downloading and saving metadata:\n' + json.dumps(vodsinfodic["data"][0], indent=4)) + with open(os.path.join(self.metadata_path, metadata_filename), 'w', encoding='utf-8') as f: + json.dump(vodsinfodic["data"][0], f, ensure_ascii=False, indent=4) if self.downloadVOD == 1: print('Downloading VOD: ' + vodsinfodic["data"][0]["title"]) - self.sendNotif('Downloading VOD', vodsinfodic["data"][0]["title"]) - subprocess.call(['streamlink', '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segmentsVOD), "twitch.tv/videos/" + vod_id, self.quality] + self.debug_cmd + ["-o", os.path.join(self.recorded_path, raw_vod_filename)]) + self.sendNotif('VOD - ' + created_at,'Downloading VOD: ' + vodsinfodic["data"][0]["title"]) + try: + subprocess.call(['streamlink', '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segmentsVOD), "twitch.tv/videos/" + vod_id, self.quality] + self.debug_cmd + ["-o", os.path.join(self.recorded_path, raw_vod_filename)]) + except Exception as e: + print('Error', 'A ERROR has ocurred and the VOD will not be downloaded.\n' + e) + self.sendNotif('ERROR - ' + created_at, 'A ERROR has ocurred and the VOD will not be downloaded. \nerror:\n' + e) if self.downloadCHAT == 1: print('Downloading and rendering CHAT: ' + vodsinfodic["data"][0]["title"]) - self.sendNotif('Downloading and rendering CHAT', vodsinfodic["data"][0]["title"]) + self.sendNotif('CHAT - ' + created_at,'Downloading JSON and rendering chat logs from VOD:\n' + vodsinfodic["data"][0]["title"]) chat_filename = "CHAT_" + raw_filename[:-2] + "json" render_filename = "CHAT_" + raw_filename[:-2] + "mp4" outputJSON = os.path.join(self.chatJSON_path, chat_filename) outputMP4 = os.path.join(self.chatMP4_path, render_filename) try: - subprocess.call(["bash","./tools-ubuntu/chat.sh", vod_id, outputJSON, outputMP4]) + subprocess.call(["bash","./bin/chat.sh", vod_id, outputJSON, outputMP4]) except Exception as e: - self.sendNotif('ERROR', "A ERROR has ocurred and chat will need to be downloaded and rendered manually.\n " + e) - print("A ERROR has ocurred and chat will need to be downloaded and rendered manually") - print(e) + self.sendNotif('ERROR - ' + created_at, "A ERROR has ocurred and chat will need to be downloaded and rendered manually.\n" + e) + print("A ERROR has ocurred and chat will need to be downloaded and rendered manually\n" + e) else: - print('A ERROR has ocurred, the latest VOD doesnt match with the livestream') - self.sendNotif('ERROR', 'A ERROR has ocurred, the latest VOD doesnt match with the livestream') + print('A ERROR has ocurred, the latest VOD doesnt match with the livestream, the VOD is not published') + self.sendNotif('ERROR - ' + present_datetime, 'A ERROR has ocurred, the latest VOD doesnt match with the livestream, the VOD is not published') else: raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) except Exception as e: raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) - print(e) - print('An error has occurred. VOD and chat will not be downloaded. Please check them manually.') - self.sendNotif('ERROR', 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) + print('An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) + self.sendNotif('ERROR - ' + present_datetime, 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) print("Recording stream is done. Fixing video file.") - self.sendNotif("STREAM DONE", "Recording stream is done. Fixing video file.") + self.sendNotif("STREAM DONE - " + present_datetime, "Recording stream is done. Fixing video file.") if(os.path.exists(recorded_filename) is True): file_mp4 = live_filename[:-2] + "mp4" vod_filename = raw_vod_filename[:-2] + "mp4" processed_filename = os.path.join(self.processed_path, file_mp4) - try: - subprocess.call(['./tools-ubuntu/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', processed_filename], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) - except Exception as e: - print(e) + subprocess.call(['./bin/ffmpeg', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', processed_filename], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) if(os.path.exists(os.path.join(self.recorded_path, raw_vod_filename)) is True): - try: - subprocess.call(['./tools-ubuntu/ffmpeg', '-y', '-i', os.path.join(self.recorded_path, raw_vod_filename), '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(self.processed_path, vod_filename)], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) - except Exception as e: - print(e) + subprocess.call(['./bin/ffmpeg', '-y', '-i', os.path.join(self.recorded_path, raw_vod_filename), '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(self.processed_path, vod_filename)], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) else: print("Skip fixing. File not found.") - self.sendNotif("ERROR", "Skip fixing. File not found.") print("Fixing is done.") if self.uploadCloud == 1: - tree = subprocess.check_output(["tree", self.root_path+"/"+self.username]).decode(sys.stdout.encoding)[:-25] - print('Uploading to cloud the Following files') - print(tree) - self.sendNotif("UPLOADING TO CLOUD", 'the following files: \n' + tree) - subprocess.call(["bash","./tools-ubuntu/upload.sh", self.root_path,self.username]) + tree = subprocess.check_output(["tree", self.root_path+"/"+self.username]).decode(sys.stdout.encoding).split("\n",1)[1] + print('Uploading the following files:\n' + tree) + self.sendNotif("UPLOADING - " + present_datetime, 'Uploading the following files: \n' + tree) + subprocess.call(["bash","./bin/upload.sh", self.root_path,self.username]) if self.deleteFiles == 1: - self.sendNotif("DELETING", "the files were deleted") + self.sendNotif("DELETING - " + present_datetime, "Deleting the files from current seccion.") print(f'{Fore.RED}DELETING FILES{Style.RESET_ALL}') print(f'{Fore.RED}Deleting ' + recorded_filename + f'{Style.RESET_ALL}') os.remove(recorded_filename) @@ -307,7 +305,7 @@ class TwitchArchive: print(f'{Fore.RED}Deleting ' + os.path.join(self.metadata_path, metadata_filename) + f'{Style.RESET_ALL}') os.remove(os.path.join(self.metadata_path, metadata_filename)) print('CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') - self.sendNotif("SECCION DONE", 'CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') + self.sendNotif("SECCION DONE - " + present_datetime, 'CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') time.sleep(self.refresh) def main(argv): twitch_recorder = TwitchArchive() diff --git a/twitch-archive.py b/twitch-archive.py index 3c6e2c4..bfa9477 100644 --- a/twitch-archive.py +++ b/twitch-archive.py @@ -1,9 +1,11 @@ -import requests, os, time, json, sys, subprocess, getopt, smtplib -from colorama import Fore, Style -from datetime import datetime +import requests, os, time, json, sys, subprocess, getopt, smtplib, pathlib +from colorama import Fore, Style +from datetime import datetime, timedelta from pytz import timezone +from dateutil import parser from dotenv import load_dotenv, find_dotenv -from email.message import EmailMessage +from email.mime.multipart import MIMEMultipart +from email.mime.text import MIMEText load_dotenv(find_dotenv()) class TwitchArchive: def __init__(self): @@ -14,11 +16,11 @@ class TwitchArchive: self.root_path = r"archive" # Path where this script saves everything (livestream,VODs,chat,metadata) self.timezone = "US/Eastern" # Timezones, you can see a list of the format timezone here: https://gist.github.com/heyalexej/8bf688fd67d7199be4a1682b3eec7568 self.refresh = 5.0 # Time between checking (5.0 is recommended) - self.notifications = 0 # 0 - disable email notification of current seccion, 1 - enable email notification of current seccion + self.notifications = 1 # 0 - disable email notification of current seccion, 1 - enable email notification of current seccion self.downloadMETADATA = 1 # 0 - disable metadata downloading, 1 - enable metadata downloading self.downloadVOD = 1 # 0 - disable VOD downloading after stream finished, 1 - enable VOD downloading after stream finished (this option downloads the latest public vod) self.downloadCHAT = 1 # 0 - disable chat downloading and rendering, 1 - enable chat downloading and rendering - self.uploadCloud = 0 # 0 - disable upload to remote cloud, 1 - enable upload to remote cloud + self.uploadCloud = 1 # 0 - disable upload to remote cloud, 1 - enable upload to remote cloud self.deleteFiles = 1 # 0 - disable the deleting of files from current seccion after being uploaded to the cloud, 1 - enable the deleting files of files from current seccion after being uploaded to the cloud (BE CAREFUL WITH THIS OPTION) self.streamlink_debug = 0 # 0 - disable streamlink debug display, 1 - enable streamlink debug display self.hls_segments = 3 # 1-10 for live stream, it's possible to use multiple threads to potentially increase the throughput. 2-3 is enough @@ -27,7 +29,7 @@ class TwitchArchive: def run(self): print('Twitch-Archive') print('Configuration:') - print(f'Root path: {Fore.GREEN}{self.root_path}{Style.RESET_ALL}') + print(f'Root path: {Fore.GREEN}' + str(pathlib.Path(self.root_path).resolve()) + f'{Style.RESET_ALL}') print(f'Timezone: {Fore.GREEN}{self.timezone}{Style.RESET_ALL}') print(f'Refresh rate: {Fore.GREEN} {str(self.refresh)}{Style.RESET_ALL}') if self.notifications == 1: print(f'Email notifications: {Fore.GREEN}Enabled{Style.RESET_ALL}') @@ -39,21 +41,16 @@ class TwitchArchive: if self.downloadCHAT == 1: print(f'Chat downloading {Fore.GREEN}Enabled{Style.RESET_ALL}') else: print(f'Chat downloading: {Fore.RED}Disabled{Style.RESET_ALL}') if self.uploadCloud == 1: print(f'Upload to Google Drive: {Fore.GREEN}Enabled{Style.RESET_ALL}') - else: print(f'Upload to Google Drive: {Fore.RED}Disabled{Style.RESET_ALL}') + else: print(f'Upload to cloud service: {Fore.RED}Disabled{Style.RESET_ALL}') if self.deleteFiles == 1: print(f'{Fore.RED}'+'\033[1m'+f'CAREFUL FILES ARE CONFIGURATED TO BE DELETED{Style.RESET_ALL}') + else: print(f'{Fore.GREEN}'+'\033[1m'+f'Files will NOT be deleted{Style.RESET_ALL}') if self.uploadCloud == 0 and self.deleteFiles == 1: print(f'{Fore.RED}'+'\033[1m'+f'FILES WILL BE DELETED AND NO UPLOADED {Style.RESET_ALL}{Fore.GREEN}\n"CTRL + C"{Style.RESET_ALL}{Fore.RED}'+'\033[1m'+f' TO STOP AND CHANGED CONFIGURATION{Style.RESET_ALL}') self.oauth_token = self.get_oauth_token() self.get_channel_id() if self.streamlink_debug == 1: self.debug_cmd = "--loglevel trace".split() else: self.debug_cmd = "".split() - if self.notifications == 1: - self.gmail = smtplib.SMTP_SSL('smtp.gmail.com', 465) - try: - self.gmail.login(os.environ.get('SENDER'), os.environ.get('PWD')) - except: - print('Your email or password are incorrect, check before running again') - + self.recorded_path = os.path.join(self.root_path,self.username,"video", "recorded") self.processed_path = os.path.join(self.root_path, self.username, "video", "processed") self.chatJSON_path = os.path.join(self.root_path, self.username, "chat", "json") @@ -77,13 +74,13 @@ class TwitchArchive: os.makedirs(stream_dir_path) print('Fixing ' + recorded_filename + '.') try: - subprocess.call(['./tools/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) + subprocess.call(["powershell.exe",'./bin/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) except Exception as e: print(e) elif(os.path.exists(os.path.join(stream_dir_path, f)) is False): print('Fixing ' + recorded_filename + '.') try: - subprocess.call(['./tools/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) + subprocess.call(["powershell.exe",'./bin/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(stream_dir_path, f[:-2]+"mp4")], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) except Exception as e: print(e) except Exception as e: @@ -157,14 +154,22 @@ class TwitchArchive: return date_formated def sendNotif(self, subject, content): - msg = EmailMessage() - msg['Subject'] = self.username + " _ " + subject - msg['From'] = os.environ.get('SENDER') - msg['To'] = os.environ.get('RECEIVER') - msg.set_content("Channel: " + self.username + "\n" +"Quality: " + self.quality + "\n" + content) if self.notifications == 1: - self.gmail.send_message(msg) - + sender = os.environ.get("SENDER") + receiver = os.environ.get("RECEIVER") + msg = MIMEMultipart() + msg['From'] = sender + msg['To'] = receiver + msg['Subject'] = self.username + " _ " + subject + body = "Current seccion is for " + self.username + "\n\n\n\n" + content + msg.attach(MIMEText((body), 'plain')) + server = smtplib.SMTP('smtp.gmail.com', 587) + server.starttls() + server.login(sender, os.environ.get("PASSWD")) + txt = msg.as_string() + server.sendmail(sender, receiver, txt) + server.quit() + def loopcheck(self): while True: status, info = self.check_user() @@ -178,9 +183,13 @@ class TwitchArchive: print(datetime.now(timezone(self.timezone)).strftime("%Y%m%d_%Hh%Mm%Ss")," ","Error with channel id or oauth token. Try to check internet connection or client-id and client-secret. Will try again in", self.refresh, "seconds.") time.sleep(self.refresh) elif status == 0: + live_date = datetime.now(timezone('UTC')) + live_date_min = live_date - timedelta(minutes=2) + live_date_plus = live_date + timedelta(minutes=2) present_datetime = datetime.now(timezone(self.timezone)).strftime("%Y%m%d_%Hh%Mm%Ss") raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename recorded_filename = os.path.join(self.recorded_path, live_filename) # start streamlink process subprocess.call(["streamlink", '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segments), "--hls-live-restart", "--twitch-disable-hosting", "twitch.tv/" + self.username, self.quality, "--retry-streams", str(self.refresh)] + self.debug_cmd + ["-o", recorded_filename]) @@ -190,87 +199,88 @@ class TwitchArchive: info = info_tmp try: vodurl = 'https://api.twitch.tv/helix/videos?user_id=' + str(self.channel_id) + '&type=archive' - vods = requests.get(vodurl, headers = {"Authorization" : "Bearer " + self.oauth_token, "Client-ID": os.environ.get('CLIENT-ID')}, timeout = 5) + vods = requests.get(vodurl, headers = {"Authorization" : "Bearer " + self.oauth_token, "Client-ID": os.environ.get('CLIENT-ID')}, timeout = 30) vodsinfodic = json.loads(vods.text) - if vodsinfodic["data"] != []: - vod_id = vodsinfodic["data"][0]["id"] - created_at = vodsinfodic["data"][0]["created_at"] - created_at = self.toTZ(created_at) - raw_filename = created_at + ".ts" - live_filename = "LIVE_" + created_at + ".ts" - raw_vod_filename = "VOD_" + raw_filename - if self.downloadMETADATA == 1: - metadata_filename = "metadata_" + created_at + ".json" - with open(os.path.join(self.metadata_path, metadata_filename), 'w', encoding='utf-8') as f: - json.dump(vodsinfodic["data"][0], f, ensure_ascii=False, indent=4) - try: - os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) - recorded_filename = os.path.join(self.recorded_path, live_filename) - except Exception as e: - raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" - os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) - recorded_filename = os.path.join(self.recorded_path, live_filename) - print(e) - print('first exception as e\nAn error has occurred. VOD and chat will not be downloaded. Please check them manually.') - self.sendNotif('ERROR', 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.') - if self.downloadVOD == 1: - print('Downloading VOD: ' + vodsinfodic["data"][0]["title"]) - self.sendNotif('Downloading VOD', vodsinfodic["data"][0]["title"]) - subprocess.call(['streamlink', '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segmentsVOD), "twitch.tv/videos/" + vod_id, self.quality] + self.debug_cmd + ["-o", os.path.join(self.recorded_path, raw_vod_filename)]) - if self.downloadCHAT == 1: - print('Downloading and rendering CHAT: ' + vodsinfodic["data"][0]["title"]) - self.sendNotif('Downloading and rendering CHAT', vodsinfodic["data"][0]["title"]) - chat_filename = "CHAT_" + raw_filename[:-2] + "json" - render_filename = "CHAT_" + raw_filename[:-2] + "mp4" - outputJSON = os.path.join(self.chatJSON_path, chat_filename) - outputMP4 = os.path.join(self.chatMP4_path, render_filename) + if vodsinfodic["data"][0] != []: + if live_date_min <= parser.parse(vodsinfodic["data"][0]["created_at"]) <= live_date_plus: + vod_id = vodsinfodic["data"][0]["id"] + created_at = vodsinfodic["data"][0]["created_at"] + created_at = self.toTZ(created_at) + raw_filename = created_at + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename try: - subprocess.call(["powershell.exe",f'./tools/chat.bat {vod_id} ../{outputJSON} ../{outputMP4}']) + os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) + recorded_filename = os.path.join(self.recorded_path, live_filename) except Exception as e: - self.sendNotif('ERROR', "A ERROR has ocurred and chat will need to be downloaded and rendered manually") - print("A ERROR has ocurred and chat will need to be downloaded and rendered manually") - print(e) + raw_filename = present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename + os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) + recorded_filename = os.path.join(self.recorded_path, live_filename) + print('first exception as e\nAn error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) + self.sendNotif('ERROR - '+ present_datetime, 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n ' + e) + if self.downloadMETADATA == 1: + metadata_filename = "metadata_" + created_at + ".json" + self.sendNotif('Metadata - ' + created_at,'Downloading and saving metadata:\n' + json.dumps(vodsinfodic["data"][0], indent=4)) + with open(os.path.join(self.metadata_path, metadata_filename), 'w', encoding='utf-8') as f: + json.dump(vodsinfodic["data"][0], f, ensure_ascii=False, indent=4) + if self.downloadVOD == 1: + print('Downloading VOD: ' + vodsinfodic["data"][0]["title"]) + self.sendNotif('VOD - ' + created_at,'Downloading VOD: ' + vodsinfodic["data"][0]["title"]) + try: + subprocess.call(['streamlink', '--http-header', 'Authorization=OAuth ' + os.environ.get('OAUTH-PRIVATE-TOKEN'), "--hls-segment-threads", str(self.hls_segmentsVOD), "twitch.tv/videos/" + vod_id, self.quality] + self.debug_cmd + ["-o", os.path.join(self.recorded_path, raw_vod_filename)]) + except Exception as e: + print('Error', 'A ERROR has ocurred and the VOD will not be downloaded.\n' + e) + self.sendNotif('ERROR - ' + created_at, 'A ERROR has ocurred and the VOD will not be downloaded. \nerror:\n' + e) + if self.downloadCHAT == 1: + print('Downloading and rendering CHAT: ' + vodsinfodic["data"][0]["title"]) + self.sendNotif('CHAT - ' + created_at,'Downloading JSON and rendering chat logs from VOD:\n' + vodsinfodic["data"][0]["title"]) + chat_filename = "CHAT_" + raw_filename[:-2] + "json" + render_filename = "CHAT_" + raw_filename[:-2] + "mp4" + outputJSON = os.path.join(self.chatJSON_path, chat_filename) + outputMP4 = os.path.join(self.chatMP4_path, render_filename) + try: + subprocess.call(["powershell.exe","./bin/chat.bat", vod_id, '../'+outputJSON, '../'+outputMP4]) + except Exception as e: + self.sendNotif('ERROR - ' + created_at, "A ERROR has ocurred and chat will need to be downloaded and rendered manually.\n" + e) + print("A ERROR has ocurred and chat will need to be downloaded and rendered manually\n" + e) + else: + print('A ERROR has ocurred, the latest VOD doesnt match with the livestream, the VOD is not published\nThe VOD and chat will not be downloaded and rendered.') + self.sendNotif('ERROR - ' + present_datetime, 'A ERROR has ocurred, the latest VOD doesnt match with the livestream, the VOD is not published\nThe VOD and chat will not be downloaded and rendered.') else: raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) except Exception as e: raw_filename = present_datetime + ".ts" - live_filename = "LIVE_" + present_datetime + ".ts" + live_filename = "LIVE_" + raw_filename + raw_vod_filename = "VOD_" + raw_filename os.rename(recorded_filename,os.path.join(self.recorded_path, live_filename)) recorded_filename = os.path.join(self.recorded_path, live_filename) - print(e) - print('An error has occurred. VOD and chat will not be downloaded. Please check them manually.') - self.sendNotif('ERROR', 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.') + print('An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) + self.sendNotif('ERROR - ' + present_datetime, 'An error has occurred. VOD and chat will not be downloaded. Please check them manually.\n' + e) print("Recording stream is done. Fixing video file.") - self.sendNotif("STREAM DONE", "Recording stream is done. Fixing video file.") + self.sendNotif("STREAM DONE - " + present_datetime, "Recording stream is done. Fixing video file.") if(os.path.exists(recorded_filename) is True): file_mp4 = live_filename[:-2] + "mp4" + vod_filename = raw_vod_filename[:-2] + "mp4" processed_filename = os.path.join(self.processed_path, file_mp4) - try: - subprocess.call(['./tools/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', processed_filename], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) - except Exception as e: - print(e) + subprocess.call(['./bin/ffmpeg.exe', '-y', '-i', recorded_filename, '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', processed_filename], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) if(os.path.exists(os.path.join(self.recorded_path, raw_vod_filename)) is True): - vod_filename = raw_vod_filename[:-2] + "mp4" - try: - subprocess.call(['./tools/ffmpeg.exe', '-y', '-i', os.path.join(self.recorded_path, raw_vod_filename), '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(self.processed_path, vod_filename)], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) - except Exception as e: - print(e) + subprocess.call(['./bin/ffmpeg.exe', '-y', '-i', os.path.join(self.recorded_path, raw_vod_filename), '-analyzeduration', '2147483647', '-probesize', '2147483647', '-c:v', 'copy', '-c:a', 'copy', '-start_at_zero', '-copyts', os.path.join(self.processed_path, vod_filename)], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT) else: print("Skip fixing. File not found.") - self.sendNotif("ERROR", "Skip fixing. File not found.") print("Fixing is done.") if self.uploadCloud == 1: - tree = subprocess.run(["powershell.exe", f"tree {self.root_path}/{self.username} /f"],text=True,capture_output=True).stdout.strip()[106:] - print('Uploading to cloud the Following files') - print(tree) - self.sendNotif("UPLOADING TO CLOUD", 'the following files: \n' + tree) - subprocess.call(["powershell.exe", f"./tools/upload.bat {self.root_path} {self.username}"]) + tree = subprocess.run(["powershell.exe","tree", f"'{self.root_path}/{self.username}'", "/f"], capture_output=True, text=True).stdout.split("\n",2)[2] + print('Uploading the following files:\n' + tree) + self.sendNotif("UPLOADING - " + present_datetime, 'Uploading the following files: \n' + tree) + subprocess.call(["powershell.exe","./bin/upload.bat", self.root_path,self.username]) if self.deleteFiles == 1: - self.sendNotif("DELETING", "the files were deleted") + self.sendNotif("DELETING - " + present_datetime, "Deleting the files from current seccion.") print(f'{Fore.RED}DELETING FILES{Style.RESET_ALL}') print(f'{Fore.RED}Deleting ' + recorded_filename + f'{Style.RESET_ALL}') os.remove(recorded_filename) @@ -295,7 +305,7 @@ class TwitchArchive: print(f'{Fore.RED}Deleting ' + os.path.join(self.metadata_path, metadata_filename) + f'{Style.RESET_ALL}') os.remove(os.path.join(self.metadata_path, metadata_filename)) print('CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') - self.sendNotif("SECCION DONE", 'CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') + self.sendNotif("SECCION DONE - " + present_datetime, 'CURRENT SECCION HAVE FINISHED GOING BACK TO CHECKING') time.sleep(self.refresh) def main(argv): twitch_recorder = TwitchArchive() @@ -315,4 +325,4 @@ def main(argv): twitch_recorder.quality = arg twitch_recorder.run() if __name__ == "__main__": - main(sys.argv[1:]) \ No newline at end of file + main(sys.argv[1:])