Initial version

Author: rebane2001

README.md

@@ -0,0 +1,73 @@
+# Sensible yt-dlp Archive Scripts
+
+I often get asked what yt-dlp scripts I use for my archives. I've not been satisfied with other scripts I've found online, so I decided to make this repo to share scripts that are the best *in my opinion*.
+
+Let me know if something doesn't work correctly or you run into any issues :).
+
+Also check out the [official README](https://github.com/yt-dlp/yt-dlp) if you have any questions about yt-dlp itself.
+
+
+## yt-dlp installation
+
+These are the installation methods I recommend, **however you can install yt-dlp any way you want**. After installation, you should be able to run yt-dlp by just typing `yt-dlp` into your terminal.
+
+These commands also install ffmpeg and aria2, which yt-dlp uses for merging files and downloading faster.
+
+For more information about installation, please read the [official README](https://github.com/yt-dlp/yt-dlp#installation).
+
+### Windows
+
+In PowerShell (run as administrator):
+
+```
+# Install Chocolatey for package management
+Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
+
+choco install -y python3
+choco install -y ffmpeg
+choco install -y aria2
+
+# Run a new PowerShell window here
+
+python -m pip install -U yt-dlp
+# If module pip is not found, run this first: 
+# python -m ensurepip
+```
+
+### Mac OS
+
+```
+# install Homebrew for package management
+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
+
+brew install python3
+brew install ffmpeg
+brew install aria2
+
+python3 -m pip install -U yt-dlp
+# If module pip is not found, run this first: 
+# python3 -m ensurepip
+```
+
+### Linux (debian)
+
+```
+sudo apt-get install -y python3 ffmpeg aria2
+python3 -m pip install -U yt-dlp
+# If module pip is not found, run this first: 
+# python3 -m ensurepip
+```
+
+
+## Updating yt-dlp
+
+Updating yt-dlp is as simple as running `python3 -m pip install -U yt-dlp` (Linux/MacOS) or `python -m pip install -U yt-dlp` (Windows). You can also install the latest master (development version) by running `python3 -m pip install --force-reinstall https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz`.
+
+
+## Using scripts
+
+1. Clone this repo using git, or download it as a zip from [here](https://github.com/rebane2001/sensible-yt-dlp-archive-scripts/archive/refs/heads/mane.zip).
+2. Open the included yt-dlp.conf file from the scripts folder for your OS with any text editor, read through it and make changes if you wish. Commented lines start with #, which you must remove for configuration options you wish to enable.
+3. Open the script file you want to use with any text editor, read through it and make changes if you wish.
+4. Put your urls in the the script's .txt file.
+5. Run the script (`bash script_name.sh` in Linux/MacOS, double-click the bat file in Windows).

scripts/linux-osx/archive_channels.sh

@@ -0,0 +1,13 @@
+# This script downloads every channel linked in channels.txt.
+# Videos will be saved in "channels/channel_name (channel_id)/YYYY-MM-DD - video_title - video_id.ext".
+# Channel and video names will be limited to max 200 characters to prevent issues.
+# List of downloaded video IDs will be kept in channels_downloaded.txt.
+# Uncomment the loop if you want to keep this running constantly.
+
+# while true; do
+yt-dlp --config-locations "./yt-dlp.conf" \
+       --download-archive "channels_downloaded.txt" \
+       --batch-file="channels_input.txt" \
+       --output "channels/%(uploader).200s (%(uploader_id)s)/%(upload_date)s - %(title).200s - %(id)s.%(ext)s"
+# sleep 300
+# done

scripts/linux-osx/archive_playlists.sh

@@ -0,0 +1,13 @@
+# This script downloads every playlist linked in playlists.txt.
+# Videos will be saved in "playlists/playlist_id - playlist_title/YYYY-MM-DD - video_title - video_id.ext".
+# Playlist and video names will be limited to max 200 characters to prevent issues.
+# List of downloaded video IDs will be kept in playlists_downloaded.txt.
+# Uncomment the loop if you want to keep this running constantly.
+
+# while true; do
+yt-dlp --config-locations "./yt-dlp.conf" \
+       --download-archive "playlists_downloaded.txt" \
+       --batch-file="playlists_input.txt" \
+       --output "playlists/%(playlist_id)s - %(playlist_title)s.200s/%(upload_date)s - %(title).200s - %(id)s.%(ext)s"
+# sleep 300
+# done

scripts/linux-osx/archive_videos.sh

@@ -0,0 +1,14 @@
+# This script downloads every channel, playlist, and video linked in videos.txt.
+# Note that this script is pretty much the same as the channels script, just slightly modified to fit this use case better.
+# Videos will be saved in "videos/channel_name (channel_id)/YYYY-MM-DD - video_title - video_id.ext".
+# Channel and video names will be limited to max 200 characters to prevent issues.
+# List of downloaded video IDs will be kept in videos_downloaded.txt.
+# Uncomment the loop if you want to keep this running constantly.
+
+# while true; do
+yt-dlp --config-locations "./yt-dlp.conf" \
+       --download-archive "videos_downloaded.txt" \
+       --batch-file="videos_input.txt" \
+       --output "videos/%(uploader).200s (%(uploader_id)s)/%(upload_date)s - %(title).200s - %(id)s.%(ext)s"
+# sleep 300
+# done

scripts/linux-osx/channels_input.txt

@@ -0,0 +1,68 @@
+# This is the base configuration used for the other scripts.
+# Look through it and comment/uncomment lines according to how you want them.
+
+# Use cookies to bypass things such as age-restrictions and paid/member content.
+#--cookies cookies.txt
+
+# Not using ipv6 can sometimes prevent 429 bans.
+#--force-ipv4
+
+# Sleeping between requests prevents 429 bans.
+--sleep-requests 1.5
+
+# Verbose output
+--verbose
+
+# If errors are not ignored, downloading playlists/channels
+#  stops on an error, which means the rest won't be downloaded
+#  if one video is deleted/privated, or otherwise broken.
+--ignore-errors
+
+# Write useful metadata
+--write-info-json
+--write-description
+--write-thumbnail
+--write-sub
+
+# Embed metadata into file, I personally don't recommend these.
+#--embed-subs
+#--embed-thumbnail
+#--embed-metadata
+#--embed-chapters
+#--embed-info-json
+
+# Write all normal subtitles and live chat.
+--sub-langs all
+
+# For no live chat, use this instead.
+#--sub-langs all,-live_chat
+
+# To also download auto-generated and auto-translated subtitles.
+# Not recommended because these downloads are pretty slow.
+--write-auto-subs
+
+
+# The default format selection/sorting of yt-dlp is excellent,
+#  so use these flags only if you need them.
+
+# Download only as mp4.
+#--format bestvideo[ext=mp4]+bestaudio[ext=m4a]
+
+# Download only as mp4 if possible, otherwise download the best available.
+#--format bestvideo[ext=mp4]+bestaudio[ext=m4a]/bv*+ba/b
+
+# Download only the audio.
+#--format bestaudio
+
+# Download only the audio as mp3.
+#--format bestaudio
+#--extract-audio
+#--audio-format mp3
+
+
+# Always save output as mkv.
+#--merge-output-format mkv
+
+# Use aria2c for downloads. Comment if you don't have aria2 installed.
+--external-downloader aria2c
+       

scripts/linux-osx/playlists_input.txt

@@ -0,0 +1,12 @@
+:: This script downloads every channel linked in channels.txt.
+:: Videos will be saved in "channels/channel_name (channel_id)/YYYY-MM-DD - video_title - video_id.ext".
+:: Channel and video names will be limited to max 200 characters to prevent issues.
+:: List of downloaded video IDs will be kept in channels_downloaded.txt.
+:: Uncomment the loop if you want to keep this running constantly.
+
+:: :loop
+yt-dlp --config-locations ".\yt-dlp.conf" ^
+       --download-archive "channels_downloaded.txt" ^
+       --batch-file="channels_input.txt" ^
+       --output "channels/%%(uploader).200s (%%(uploader_id)s)/%%(upload_date)s - %%(title).200s - %%(id)s.%%(ext)s"
+:: goto loop

scripts/linux-osx/videos_input.txt

@@ -0,0 +1,12 @@
+:: This script downloads every playlist linked in playlists.txt.
+:: Videos will be saved in "playlists/playlist_id - playlist_title/YYYY-MM-DD - video_title - video_id.ext".
+:: Playlist and video names will be limited to max 200 characters to prevent issues.
+:: List of downloaded video IDs will be kept in playlists_downloaded.txt.
+:: Uncomment the loop if you want to keep this running constantly.
+
+:: :loop
+yt-dlp --config-locations ".\yt-dlp.conf" ^
+       --download-archive "playlists_downloaded.txt" ^
+       --batch-file="playlists_input.txt" ^
+       --output "playlists/%%(playlist_id)s - %%(playlist_title)s.200s/%%(upload_date)s - %%(title).200s - %%(id)s.%%(ext)s"
+:: goto loop

scripts/linux-osx/yt-dlp.conf

@@ -0,0 +1,13 @@
+:: This script downloads every channel, playlist, and video linked in videos.txt.
+:: Note that this script is pretty much the same as the channels script, just slightly modified to fit this use case better.
+:: Videos will be saved in "videos/channel_name (channel_id)/YYYY-MM-DD - video_title - video_id.ext".
+:: Channel and video names will be limited to max 200 characters to prevent issues.
+:: List of downloaded video IDs will be kept in videos_downloaded.txt.
+:: Uncomment the loop if you want to keep this running constantly.
+
+:: :loop
+yt-dlp --config-locations ".\yt-dlp.conf" ^
+       --download-archive "videos_downloaded.txt" ^
+       --batch-file="videos_input.txt" ^
+       --output "videos/%%(uploader).200s (%%(uploader_id)s)/%%(upload_date)s - %%(title).200s - %%(id)s.%%(ext)s"
+:: goto loop

scripts/windows/archive_channels.bat

@@ -0,0 +1,67 @@
+# This is the base configuration used for the other scripts.
+# Look through it and comment/uncomment lines according to how you want them.
+
+# Use cookies to bypass things such as age-restrictions and paid/member content.
+#--cookies cookies.txt
+
+# Not using ipv6 can sometimes prevent 429 bans.
+#--force-ipv4
+
+# Sleeping between requests prevents 429 bans.
+--sleep-requests 1.5
+
+# Verbose output
+--verbose
+
+# If errors are not ignored, downloading playlists/channels
+#  stops on an error, which means the rest won't be downloaded
+#  if one video is deleted/privated, or otherwise broken.
+--ignore-errors
+
+# Write useful metadata
+--write-info-json
+--write-description
+--write-thumbnail
+--write-sub
+
+# Embed metadata into file, I personally don't recommend these.
+#--embed-subs
+#--embed-thumbnail
+#--embed-metadata
+#--embed-chapters
+#--embed-info-json
+
+# Write all normal subtitles and live chat.
+--sub-langs all
+
+# For no live chat, use this instead.
+#--sub-langs all,-live_chat
+
+# To also download auto-generated and auto-translated subtitles.
+# Not recommended because these downloads are pretty slow.
+--write-auto-subs
+
+
+# The default format selection/sorting of yt-dlp is excellent,
+#  so use these flags only if you need them.
+
+# Download only as mp4.
+#--format bestvideo[ext=mp4]+bestaudio[ext=m4a]
+
+# Download only as mp4 if possible, otherwise download the best available.
+#--format bestvideo[ext=mp4]+bestaudio[ext=m4a]/bv*+ba/b
+
+# Download only the audio.
+#--format bestaudio
+
+# Download only the audio as mp3.
+#--format bestaudio
+#--extract-audio
+#--audio-format mp3
+
+
+# Always save output as mkv.
+#--merge-output-format mkv
+
+# Use aria2c for downloads. Comment if you don't have aria2 installed.
+--external-downloader aria2c