Comments (11)
DeepDanbooru is a great project, be sure that reference is in your pocket already!
And thanks for the help.
from deepdanbooru.
I use v3.
I made a custom tag list made only of tags from my database.
Could you drop me an email to yozhikoff [at] protonmail.com?
Would be easier to share code there.
from deepdanbooru.
Yes, it is correct and intended. Unlike typical resnet, DeepDanbooru has too high output dimension (about 8K) and it makes that normalized loss functions converge to very small value. To getting training stability and speed, DeepDanbooru needs high learning_rate and 5.0 is empirical value what I found. You can change it freely.
from deepdanbooru.
Thanks for the explanation!
Quite surprising I should say.
FYI: I'm doing transfer learning on v3 on a custom 2 million images dataset with 6k tags and the results are incredible after the first epoch (6 hours on 3090). And that's even before backbone weights unfreezing. I'm very impressed!
I'm planning to unfreeze after 5 epochs, do you think it would enough?
Also, another random question - is there a reason why you decided to train without a validation set? Just because of huge dataset size?
from deepdanbooru.
Interesting. If you have a plan to release your network, please reference DeepDanbooru :)
I'm planning to unfreeze after 5 epochs, do you think it would enough?
It is totally dependent on your dataset. But I think 5 epochs is a good starting point.
is there a reason why you decided to train without a validation set? Just because of huge dataset size?
Yes, the dataset is huge and making reliable validation set is another hard task, so I simply didn't.
from deepdanbooru.
Also @Superfloh if you're working with this code too it would be interesting to collaborate!
from deepdanbooru.
I'm trying to use a selfmade dataset with 1.4 million images, which are already scaled to 512x512, and around 7600 tags.
First I used the default learning rate of 0.001 and a batchsize of 5 since my rtx 3080 can't take more.
After one epoch (about 24h training) the Loss was at about 350 and P=0.75.
Now I tried it with a learning rate of 5 for about 8 hours and the Loss stays at around 2000 with p=0.3, which didn't change at all since I started training.
I think I'll continue with lr=0.001 for now, since I can't figure out the problem.
from deepdanbooru.
One thing you can try is transfer learning. Take a pretrained model, remove the last layer and freeze the rest. Then put a new last layer and train updating its weights only.
This procedure significantly decreases training time.
For me it worked just fine. Can't post loss values now, but the results are pretty impressive!
I can share the code if you want @Superfloh
I probably will also try other learning rates later.
from deepdanbooru.
That would be nice, which pretrained model did you use?
Also another question, did you remove unused tags from the database?
I still have tags in the database that are not in the tags.txt
from deepdanbooru.
@KichangKim do you remember what Loss, precision, f1 and recall you had, when you trained the v3 model for 30 epochs ?
from deepdanbooru.
@KichangKim do you remember what Loss, precision, f1 and recall you had, when you trained the v3 model for 30 epochs ?
Unfortunately, I don't have any logs for previous models.
from deepdanbooru.
Related Issues (20)
- Help with training with optional tags
- pose detection
- Clarification of README.md HOT 6
- requests.exceptions.JSONDecodeError HOT 1
- module 'tensorflow' has no attribute 'lite HOT 1
- Very strange issues with Checkerboard and Argyle patterns in images HOT 1
- How to properly train it? HOT 3
- Training script can't read dataset? HOT 3
- about model HOT 3
- Add a progress bar
- Best learnig rate
- How to output the result to txt? HOT 3
- is there any GPU acceleration? HOT 1
- Model input and output? HOT 4
- About Character Tags
- Docker image with DeepDanbooru HOT 4
- Error reading tags with Unicode in them HOT 1
- Does "deepdanbooru-v3-20211112-sgd-e28" contain nsfw tags? HOT 4
- How to compile this lib into C++
- Help deploying locally HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepdanbooru.