Comments (2)
CXwudi
commented on June 8, 2024
The new downloader enablement machenism will delete the enablement section
the downloader section will take place of what to enable, it will be like:
downloader:
NicoNicoDouga:
# settings of youtube-dl/ytp-dl for downloading niconico videos
# settings must make sure nothing blocks from downloading the video (e.g. don't put --version on yt-dlp)
# do the same for all other downloaders.
- name: using yt-dlp
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyy
- name: using youtube-dl
type: youtube-dl
launch-cmd: youtube-dl.exe
external-args: -v, --username, xxx, --password, yyy
- name: using nndownload
type: nndownload
launch-cmd: nndownload.exe
username:
password:
session-cookie:
Youtube:
- name: using yt-dlp
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyy
Bilibili:
- name: using yt-dlp
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyywhere the type and name are needed for all downloader configs, and the rest of the config are customized for each downloader
will need to POC how spring boot conversion will works:
- https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config.typesafe-configuration-properties.conversion
- https://www.baeldung.com/configuration-properties-in-spring-boot#3-custom-converter
I am afraid that Spring boot configuration doesn't support conversion of a map of configuration to custom type like https://stackoverflow.com/questions/64228017/converter-using-configurationpropertiesbinding-not-work-for-complex-source-type
Use this way to get all configuration programmatically https://stackoverflow.com/questions/23506471/access-all-environment-properties-as-a-map-or-properties-object
from vocadb-video-downloader-new.
CXwudi
commented on June 8, 2024
Unfortauntely, we decided to not use spring-boot-starter-configuration to handle business logic validation of our various configs. In fact, nor should it be. So the final configuration will be like:
downloader:
NicoNicoDouga:
# settings of youtube-dl/ytp-dl for downloading niconico videos
# settings must make sure nothing blocks from downloading the video (e.g. don't put --version on yt-dlp)
# do the same for all other downloaders.
- name: using yt-dlp
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyy
- name: using youtube-dl
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyy
- name: using nndownload
type: nndownload
launch-cmd: nndownload.exe
username:
password:
session-cookie:
Youtube:
- name: using yt-dlp
type: youtube-dl
options:
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyy
Bilibili:
- name: using yt-dlp
type: youtube-dl
launch-cmd: yt-dlp.exe
external-args: -v, --username, xxx, --password, yyySo pretty much we are just mapping to Map<PVServiceEnum, List<RawDownloaderConfig>> where RawDownloaderConfig contains everything that any download config could declare (in above lists, we will have name, type, launch-cmd, external-args, username, password, session-cookie where only name and type are not null), because a POC has shown that @ConfigurationPropertiesBinding doesn't work with Collection type, only for String which is useless
And then we will have custom logic and validation to build up all downloaders, and use FailureAnalyzer to provide user-friendly error msg upon downloader config error
from vocadb-video-downloader-new.
Related Issues (20)
- How to handle error path with easy-batch HOT 1
- features about downloading pvs
- retry mechanism in downloader HOT 1
- only load downloader if needed HOT 3
- Try to complete it before the final release of 2022年V家新曲合集 HOT 2
- add thin jar config HOT 1
- how to load user command line from YAML
- Improve how PV Type is handled in pvTaskDecider
- Solve conflict on file naming HOT 1
- Try finish the project refactoring before the end of 2022
- Use docker to setup test&production env HOT 1
- Remove easy-batch, but observe the batch pattern HOT 1
- Add MP3 tagging support in extractor module HOT 1
- use aggregated parameter object HOT 3
- Dependency Dashboard
- Add sort order support for so-ID and sm-ID
- Create a new project for openapi-vocadb-java-client-autofixer
- Deprecate vocadb-openapi-client-java in favour of vocadb-openapi-dto-java HOT 2
- A way better sorting and enablement for PVs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vocadb-video-downloader-new.