• Setup
  • Downloading the Project
  • Installing Dependencies

• Usage
  • Running the Project
  • Executable Options
  • Updating the Project

• The Youtube Downloader
  • Configuring the Youtube Downloader
    • Youtube Downloader Settings
    • Youtube Downloader Configuration (Sample)

• The Youtube Channel Downloader
  • Getting an API Key
    • Setting up your API Key
    • Recovering your API Key
  • Configuring the Youtube Channel Downloader
    • Youtube Channel Downloader Settings
    • Youtube Channel Downloader Configuration (Sample)
  • Adding Channels
    • Channel Settings
    • Channel Configuration (Sample)
  • Finding Youtube Playlist IDs
    • Getting the Playlist ID of a Youtube Channel
    • Getting the Playlist ID of a Youtube Playlist
  • Grouping Channels
    • Channel Group Settings
    • Channel Group Configuration (Sample)
    • Channel Group Inheritance
    • Channel Group Inheritance Configuration (Sample)
  • Special Channel Processing

• Configuration
  • Configuration Overview
    • Configuration Sections
    • Configuration (Sample)
  • Process Configuration
    • Process Settings
    • Process Configuration (Sample)
  • SponsorBlock Configuration
    • SponsorBlock Settings
    • SponsorBlock Configuration (Sample)
  • Color Configuration
    • Supported Colors
    • Color Settings
    • Color Configuration (Sample)
  • Logging Configuration
    • Logging Settings
    • Logging Configuration (Sample)

To get started, you will need to download the Project, and install the required Dependencies.

Simply download the latest project files from Github.
Then extract the files to a location of your choice.

Alternatively you can clone the repository with Git using:

git clone "https://github.com/ZGorlock/YoutubeDownloader.git"

To run this project you will need to have the following dependencies installed:

If you do not already have these dependencies installed then download the packages for your operating system.
Then extract the packages to a location of your choice.

Add the /bin folder of each dependency to your Path environment variable so they are accessible on the system path.

You can verify that they are installed properly by executing the following commands and checking the versions:

java --version

mvn -v

ffmpeg -version

git --version

This project contains two programs:

• The Youtube Downloader
• The Youtube Channel Downloader

The project can be run by using the scripts provided in the project directory:

• YoutubeDownloader.bat^🗋
• YoutubeChannelDownloader.bat^🗋
• YoutubeDownloader.sh^🗋
• YoutubeChannelDownloader.sh^🗋

Depending on your operating system use either the .bat scripts (Windows) or the .sh scripts (Linux):

On Windows:

YoutubeDownloader.bat

YoutubeChannelDownloader.bat

On Linux:

./YoutubeDownloader.sh

./YoutubeChannelDownloader.sh

Alternatively you can run the project with Maven using:

mvn compile && mvn exec:java -Dexec.mainClass="youtube.YoutubeDownloader"

mvn compile && mvn exec:java -Dexec.mainClass="youtube.YoutubeChannelDownloader"

Also, if you prefer, you could always run or debug the project in an IDE.

You can configure this project to use either yt-dlp or youtube-dl.

• yt-dlp
  • A newer drop-in replacement of the Youtube Downloader executable
  • Actively maintained by new developers
  • Includes additional features

• youtube-dl
  • The original Youtube Downloader executable
  • As of the time of writing this it appears it may no longer be maintained
  • Throttling and failures often occur when downloading videos

Specify your choice of executable in the project configuration as explained under:

• Configuring the Youtube Downloader
• Configuring the Youtube Channel Downloader

The project will automatically download the executable that you choose.
At the beginning of each run it will check for updates to the executable and download the latest version if needed.

This project should work as it does now indefinitely, as long as yt-dlp and youtube-dl continue to exist on the same websites they do now.
However, additional improvements and features may be added from time to time.

If you wish to receive these updates then you can download the latest project files from Github.
More information about the download process is provided under: Downloading the Project.

If you cloned the repository with Git then you can update to the latest code using:

git pull origin master

⚠️WARNING: Before updating the project, make sure you backup your configuration and data files:

• conf.json^🗋
• channels.json^🗋
• src/youtube/channel/process/ChannelProcesses.java^🗋
• data/*

⚠️Updating may cause merge conflicts and could potentially overwrite your custom changes.

This program is just a simple Youtube video downloader.
It will download the videos from the Youtube urls entered into the console while the program is running.

Videos will be saved in a ~/Youtube folder in your user directory.
This can be customized by defining location.outputDir in the Youtube Downloader Configuration.

You can also create a list of Youtube urls in the file data/downloadQueue.txt and when the program is executed it will download the provided list of videos.

The program will continue to run until an empty line in entered: Enter.

You can customize the operation of the Youtube Downloader by changing the settings inside the file conf.json^🗋.

To configure the Youtube Downloader, find or create the "YoutubeDownloader" json configuration and customize the Settings it contains.

"YoutubeDownloader": {
    "location": {
        "outputDir": "C:/Users/User/Downloads",
        "browser": "Chrome"
    },
    "process": {
        "asAudio": false
    },
    "flag": {
        "neverUseBrowserCookies": true,
        "test": {
            "safeMode": false,
            "preventRun": false,
            "preventDownload": false,
            "preventVideoFetch": false,
            "preventExeAutoUpdate": false,
            "preventExeVersionCheck": false
        }
    }
}

This program is a more complex Youtube video downloader and the main focus of this project.
It will download and keep entire playlists and channels synchronized on your hard drive.

This document will explain everything you need to know to set up the Youtube Channel Downloader to work for you.

In order to use the Youtube Channel Downloader, you will need to get a Google API key.
A Google API key is completely free through your Google account.

The Youtube Channel Downloader uses the Youtube Data API to query the video lists of channels and playlists.
The free quota for the Youtube Data API is 10,000 queries per day, which is more than you should ever need.

• Go to: Google Cloud - APIs & Services
• Click Create Project
  • Set Project name: 'Youtube Downloader'
  • Click Create
• Click Enable APIs and Services
• Search for: 'Youtube'
  • Select YouTube Data API v3
• Click Enable
• Click Create Credentials
  • Select YouTube Data API v3
  • Click Public Data
  • Click Next
• Copy the API Key to the file apiKey^🗋 in the project directory

• Go to: Google Cloud - APIs & Services
• Click Select Project
  • Select Youtube Downloader
  • Click Open
• Click Credentials
  • Select your API key
  • Click Show Key
• Copy the API Key to the file apiKey^🗋 in the project directory

You can customize the operation of the Youtube Channel Downloader by changing the settings inside the file conf.json^🗋.

To configure the Youtube Channel Downloader, find or create the "YoutubeChannelDownloader" json configuration and customize the Settings it contains.

"YoutubeChannelDownloader": {
    "location": {
        "storageDrive": "C:/",
        "musicDir": "Users/User/Music/",
        "videoDir": "Users/User/Videos/",
        "browser": "Chrome"
    },
    "process": {
        "enableCustomRenameProcesses": true,
        "enableCustomFilterProcesses": true
    },
    "filter": {
        "enableFiltering": true,
        "channel": null,
        "group": null,
        "startAt": null,
        "stopAt": null
    },
    "log": {
        "printChannels": false,
        "printStats": true,
        "printReport": false
    },
    "backup": {
        "enableBackups": false,
        "backupDir": null,
        "compressBackups": true,
        "daysBetweenBackups": 7,
        "daysToKeepBackups": 30,
        "files": {
            "includeConfigs": true,
            "includeData": true,
            "includeLogs": true,
            "includeSourceCode": false,
            "includeCompiledSource": false,
            "includeApiKey": false,
            "includeExecutable": false
        }
    },
    "flag": {
        "deleteToRecyclingBin": true,
        "neverUseBrowserCookies": true,
        "retryPreviousFailures": false,
        "test": {
            "safeMode": false,
            "preventRun": false,
            "preventProcess": false,
            "preventDownload": false,
            "preventDeletion": false,
            "preventRenaming": false,
            "preventPlaylistEdit": false,
            "preventChannelFetch": false,
            "preventExeAutoUpdate": false,
            "preventExeVersionCheck": false
        }
    }
}

You can specify the Youtube channels and playlists that you want to process by configuring Channels inside the file channels.json^🗋.

There are many examples in channels-sample.json^🗋 that you can use as a reference.
The file channels-sample.json^🗋 is also my personal Channel configuration; if you see any Channels that you like you can copy them to your channels.json^🗋.

To add a Channel, create a new Channel json configuration and customize the Settings it contains.

{
    "key": "MY_CHANNEL",
    "active": true,
    "name": "MyChannel",
    "group": "FAVORITES",
    "url": "https://www.youtube.com/c/MyFavoriteChannel",
    "playlistId": "UU65aMyFavoriteChannelXY",
    "outputFolder": "Youtube/My Channel",
    "playlistFile": "Youtube/My Channel.m3u",
    "saveAsAudio": false,
    "savePlaylist": true,
    "reversePlaylist": true,
    "ignoreGlobalLocations": false,
    "keepClean": false
}

⚠️WARNING:

• Again, make sure you set the outputFolder for each Channel to an empty directory or a directory that does not exist yet
• These directories may have files deleted from them in certain circumstances
  • By default, files that are deleted in this way are not sent to the recycle bin and would be difficult, if not impossible, to recover
• Examples:
  • BAD: A directory that also contains your personal home videos
  • BAD: A directory that also contains important work documents
  • BAD: A directory that also contains videos that you have downloaded previously for this Channel before starting to use this program
  • GOOD: An empty directory or a directory that does not exist yet

When creating a Channel Configuration you need to provide a setting called playlistId.
This is what links your Channel with the actual channel or playlist on Youtube.

This playlist id already exists and you just have to get it from Youtube and set it in your Channel.
The process of obtaining the playlist id depends on whether your Channel represents a Youtube Channel or a Youtube Playlist.

• Go to the home page of the channel on Youtube
  • Right click the blank area of the page
    • Select View page source
    • Search for: 'externalId'
• Copy the external id from the page source:
  • ···,"externalId":"UC65aMyFavoriteChannelXY",···
  • ··················UC65aMyFavoriteChannelXY·····
• Change the second letter in the external id from 'C' to 'U' to get the playlist id
• Set the playlist id as playlistId in your Channel

• Go to the home page of the playlist on Youtube
• Copy the playlist id from the url:
  • https://www.youtube.com/watch?v=L_MupB3z1g4&list=PLUja9k1MyFavoriteChannel5Py_guZ3R
  • ·················································PLUja9k1MyFavoriteChannel5Py_guZ3R
• Set the playlist id as playlistId in your Channel

After you have finished adding Channel Configurations, you have the option to group your Channels; making them easier to read, search, and filter.

You can specify the groups that you want by configuring Channel Groups inside the file channels.json^🗋.

To add a Channel Group, create a new Channel Group json configuration and customize the Settings it contains.

{
    "key": "MY_GROUP",
    "channels": [
        ...
    ]
}

When creating a Channel Group Configuration you can also define the default setting configuration for that group.

In addition to the required key and channels settings, any Setting available in a Channel Configuration can also be configured in a Channel Group. (excluding: playlistFile)
When a setting is configured in this way, all children of that group will inherit the setting.

For example, if a Channel Group enables saveAsAudio, then all children of the group will save their downloaded content as audio files.
Even though saveAsAudio is usually disabled by default, within that group saveAsAudio will act as though it is enabled by default, even when that setting is not explicitly configured for a specific child configuration.
To disable saveAsAudio for a specific child, that child's configuration would have to explicitly disable saveAsAudio.

The active setting will behave slightly differently; instead of the standard inheritance described above, it will work in an all-or-nothing manner.
For example, if a child is active, but its parent group is inactive, the child will not be processed. This will also be the case if any of its enclosing groups are inactive.

If a group specifies an outputFolder, then its children may define their outputFolder and playlistFile relative to it using ~ to denote the relative path.

For example, if the parent outputFolder is configured as "Youtube/Videos", then:

• Defining the child outputFolder as "~/Favorites" would be interpreted as "Youtube/Videos/Favorites"
• Defining the child playlistFile as "~ - Favorites.m3u" would be interpreted as "Youtube/Videos - Favorites.m3u"

{
    "key": "MY_GROUP",
    "active": true,
    "outputFolder": "Youtube",
    "saveAsAudio": false,
    "reversePlaylist": true,
    "keepClean": false,
    "channels": [
        {
            "key": "FAVORITE",
            "active": true,
            "url": "https://www.youtube.com/c/MyFavoriteChannel",
            "outputFolder": "~/Favorite",
            "channels": [
                {
                    "key": "FAVORITE_VIDEOS",
                    "name": "FavoriteVideos",
                    "playlistId": "PLUja9k1MyFavoriteVideos65Py_guZ3R",
                    "outputFolder": "~ Videos",
                    "playlistFile": "~/Favorite Videos.m3u",
                    "keepClean": true
                },
                {
                    "key": "FAVORITE_SONGS",
                    "active": false,
                    "name": "FavoriteSongs",
                    "playlistId": "PLUja9k1MyFavoriteSongs865Py_guZ3R",
                    "outputFolder": "~ Songs",
                    "playlistFile": "~/Favorite Songs.m3u",
                    "saveAsAudio": true
                }
            ]
        }
    ]
}

This would be equivalent to:

{
    "key": "FAVORITE_VIDEOS",
    "active": true,
    "name": "FavoriteVideos",
    "group": "FAVORITE",
    "url": "https://www.youtube.com/c/MyFavoriteChannel",
    "playlistId": "PLUja9k1MyFavoriteVideos65Py_guZ3R",
    "outputFolder": "Youtube/Favorite Videos",
    "playlistFile": "Youtube/Favorite/Favorite Videos.m3u",
    "saveAsAudio": false,
    "reversePlaylist": true,
    "keepClean": true
},
{
    "key": "FAVORITE_SONGS",
    "active": false,
    "name": "FavoriteSongs",
    "group": "FAVORITE",
    "url": "https://www.youtube.com/c/MyFavoriteChannel",
    "playlistId": "PLUja9k1MyFavoriteSongs865Py_guZ3R",
    "outputFolder": "Youtube/Favorite Songs",
    "playlistFile": "Youtube/Favorite/Favorite Songs.m3u",
    "saveAsAudio": true,
    "reversePlaylist": true,
    "keepClean": false
}

There are certain instances where you may want to perform some additional processing on the Channel to filter which videos are downloaded, to rename the downloaded videos to something that you prefer, etc. This functionality is possible granted that you know a bit of Java.

There are two methods in the file src/youtube/channel/process/ChannelProcesses.java^🗋 :

• performSpecialPreConditions()
  • Executed before the download queue is produced for the Channel
  • Used to rename the videos that will be downloaded
  • Can be enabled or disabled using process.enableCustomRenameProcesses

• performSpecialPostConditions()
  • Executed after the download queue is produced for the Channel
  • Used to filter out videos that should not be downloaded; do not use this method to rename videos
  • Can be enabled or disabled using process.enableCustomFilterProcesses

In both of these methods, all you have to do is add another case for your Channel key to the switch statement.
You may also add whatever logic you like to the body of the method, or even call your own custom methods or classes, for special use cases.

These methods have no return value; any changes that you want to make should be achieved by modifying the following accessible objects:

You can modify the video map and Channel state lists using your own custom logic.

There is also a collection of macro methods provided that you can call to perform certain actions.
These macro methods are defined in the files src/youtube/channel/process/macro/RenameProcess.java^🗋 and src/youtube/channel/process/macro/FilterProcess.java^🗋.
All of these methods will take the videoMap as their first argument, and the macro action will be performed on each Video in the video map.

There are many examples of these processes in the file src/youtube/channel/process/ChannelProcesses_Sample.java^🗋 that you can use as a reference.
The file src/youtube/channel/process/ChannelProcesses_Sample.java^🗋 contains the processes for my personal Channel configuration; if you copied some of my Channel configurations from channels-sample.json^🗋 then you may also wish to copy the corresponding processes.

All of the configuration settings for the project are stored in the file conf.json^🗋.

{
    "YoutubeChannelDownloader": {
        ...
    },
    
    "YoutubeDownloader": {
        ...
    },
    
    "process": {
        ...
    },
    
    "sponsorBlock": {
        ...
    },
    
    "color": {
        ...
    },
    
    "log": {
        ...
    }
}

You can customize how content will be downloaded and processed by changing the settings inside the file conf.json^🗋.

Changing these settings will affect the logging for both the Youtube Downloader and the Youtube Channel Downloader.
Any of these settings can also be defined in the "process" section inside of the Youtube Downloader Configuration or Youtube Channel Downloader Configuration to override them for the respective program.

To configure the process, find or create the "process" json configuration and customize the Settings it contains.

"process": {
    "executable": "yt-dlp",
    "customFlags": null,
    "format": {
        "preMerged": true,
        "defaultVideoFormat": "mp4",
        "defaultAudioFormat": "mp3"
    }
}

You can set up SponsorBlock by changing the settings inside the file conf.json^🗋 or channels.json^🗋.
SponsorBlock is able to automatically cut out unwanted segments from the content you download. (sponsored sections, self promotions, interaction reminders, etc.)
To use SponsorBlock, you must have FFmpeg installed and accessible on the path.
This is only available when using yt-dlp as the executable.

SponsorBlock is capable of distinguishing and skipping several types of segments which can be enabled independently.
Details about these segments and what each covers can be found on the Category Breakdown page of the SponsorBlock Wiki.

You can set up a global SponsorBlock configuration for the entire project, or for the Youtube Downloader or Youtube Channel Downloader programs individually.
You can also have custom SponsorBlock configurations for individual Channel Groups or Channels.

To configure SponsorBlock, find or create a "sponsorBlock" json configuration and customize the Settings it contains.

• A configuration in conf.json^🗋 will function as a global configuration for the entire project.
• A configuration in conf.json^🗋 inside the Youtube Downloader Configuration or Youtube Channel Downloader Configuration will affect all content downloaded by the respective program.
• A configuration in channels.json^🗋 inside a Channel Configuration will affect only that Channel, and one inside a Channel Group Configuration will affect all Channels and Channel Groups which are children of the group.

"sponsorBlock": {
    "enabled": true,
    "forceGlobally": false,
    "overrideGlobal": false,
    "skipAll": false,
    "skipSponsor": false,
    "skipIntro": false,
    "skipOutro": false,
    "skipSelfPromo": false,
    "skipPreview": false,
    "skipInteraction": false,
    "skipMusicOffTopic": false
}

You can customize the colors used by the project when printing output to the console by changing the settings inside the file conf.json^🗋.

Changing these settings will affect the colors for both the Youtube Downloader and the Youtube Channel Downloader.
Any of these settings can also be defined in the "color" section inside of the Youtube Downloader Configuration or Youtube Channel Downloader Configuration to override them for the respective program.

These colors will only work if your console supports ANSI SGR codes.
The project supports this list of Supported Colors.

To configure the colors, find or create the "color" json configuration and customize the Settings it contains.

OPTION	COLOR	HEX	RGB	ANSI
WHITE		#FFFFFF	(255, 255, 255)	Esc[97m
GREY		#808080	(128, 128, 128)	Esc[37m
DARK_GREY		#595959	(89, 89, 89)	Esc[90m
BLACK		#000000	(0, 0, 0)	Esc[30m
DARK_RED		#F0524F	(240, 82, 79)	Esc[31m
RED		#FF4050	(255, 64, 80)	Esc[91m
ORANGE		#A68A0D	(166, 138, 13)	Esc[33m
YELLOW		#E5BF00	(229, 191, 0)	Esc[93m
DARK_GREEN		#5C962C	(92, 150, 44)	Esc[32m
GREEN		#4FC414	(79, 196, 20)	Esc[92m
TEAL		#00A3A3	(0, 163, 163)	Esc[36m
CYAN		#00E5E5	(0, 229, 229)	Esc[96m
DARK_BLUE		#3993D4	(57, 147, 212)	Esc[34m
BLUE		#1FB0FF	(31, 176, 255)	Esc[94m
PURPLE		#A771BF	(167, 113, 191)	Esc[35m
MAGENTA		#ED7EED	(237, 126, 237)	Esc[95m
DEFAULT		varies	varies	varies

"color": {
    "enableColors": true,
    "base": "GREEN",
    "good": "CYAN",
    "bad": "RED",
    "log": "DARK_GREY",
    "channel": "YELLOW",
    "video": "PURPLE",
    "number": "WHITE",
    "file": "GREY",
    "exe": "ORANGE",
    "link": "TEAL",
    "progressBar": {
        "base": "GREEN",
        "good": "CYAN",
        "bad": "RED"
    }
}

You can customize what is printed to the console when running the project by changing the settings inside the file conf.json^🗋.

Changing these settings will affect the logging for both the Youtube Downloader and the Youtube Channel Downloader.
Any of these settings can also be defined in the "log" section inside of the Youtube Downloader Configuration or Youtube Channel Downloader Configuration to override them for the respective program.

To configure the logging, find or create the "log" json configuration and customize the Settings it contains.

"log": {
    "printSettings": true,
    "printExeVersion": true,
    "printExecutionTime": true,
    "download": {
        "showCommand": true,
        "showWork": false,
        "showProgressBar": true,
    },
    "file": {
        "allowFileLogging": true,
        "writeMainLog": true,
        "writeDownloadLog": true,
        "writeApiLog": true,
        "daysToKeepLogs": 30
    }
}

~~ This project is for educational and testing purposes only and is not intended to be used in any way that would violate the Youtube ToS or to download copyrighted material ~~