Comments (7)
Hi Tensai,
You can do obviously do it... It won't be that easy within ngPost but not the end of the world either.
There are pros and cons, personally I see more cons.
1.: pros:
- potentially you never stop uploading once the first rar/par2 is done until you've consumed all your queue.
The upload rate doesn't improve BUT the overall time for your post could decrease significantly.
I've never touched 4k releases around 50GB, but it seems to me that packing (rar -m0) and generating par2 (with ParPar) is quite fast, a detail compare to the upload duration, even if you've a 10GBps connection no?
2.: cons:
- you use more disk space as you have 2 or even several temporary compression folders. Imagine your queue has only 50GB folders, if the packing is way faster and you end up with 10 temporary folders. That's 500GB! You may run out of disk.
- ParPar can be CPU consuming. How do I make sure that the Threads are well spread between ngPost and ParPar? i.e: you shouldn't exceed your number of core, so limit both ngPost and ParPar.
So basically I find it too risky. And anyway, if you're having a queue, what's the hurry, just launch it and come back the next morning.
from ngpost.
Hi Matthieu
many thanks for our response.
Personally I think the pros upweight the cons by far, especially for "professional" uploaders which have separate scripts to continuously fill the monitored folder with new jobs and where it is key to have all the files uploaded as fast as possible.
Disk space usually is no issue for them. But to be on the safe side, a mechanism could be implemented to simply pause the rar/par queue if disk space is low. As soon as uploaded files are deleted again and enough disk space is free, the rar/par queue could resume again.
And as you mentioned, limiting factor is almost always the upload itself. So there is no need to have rar/par working at the disk i/o limit (which usually is the limit for rar/par) and they could simply be limited to one thread each.
Unfortunately I am not as proficiency in C++/QT as you obviously are and honestly have no time to dig into it.
But be assured that there is a huge interest in this option and ngPost could become the standard usenet upload tool if this would be implemented.
Still hope you might consider doing so.
Regards, Tensai
from ngpost.
Well I didn't have any feedback from "professional" posters...
I'm not sure if any use ngPost. I suppose they already have their home-made scripts and continue to use them, which makes sense. Probably they still use Nyuu as they don't really have any reason to change...
Monitor the disk space is an hassle, especially with multi-OS support.
I could maybe implement to only have the next Job packed in advance, which means no more than 2 temporary folders, so I imagine, max 200GB.
I'm waiting for some translations to release the v4.8. I'll check if I can find a way to include it "easily".
from ngpost.
Well I didn't have any feedback from "professional" posters...
Well, now you have ;-) At least indirect because I am not actually one of them myself but I certainly can say I do represent some of them.
And I actually tried to convince them to switch to ngPost because I have no time to maintain my tool any further. And guess what, main reason why they didn't want to was the missing parallel rar/par and upload process.
But what they did like very much was the multi server support and also the UI. Two things that nyuu does not offer on its own. And of course also the speed which is beyond nyuu.
So as said, your program has already been very positively noticed and its potential is huge.
I could maybe implement to only have the next Job packed in advance, which means no more than 2 temporary folders, so I imagine, max 200GB.
This is absolutely reasonable and would be sufficient to achive the same overall performance.
Would be great to have this!
from ngpost.
ok I'm having a look.
could you drop me an email, maybe you could help for the German translation of 6 sentences for the new release, I'll add you to my mailing list. ([email protected])
from ngpost.
@Mamitomama I'm not sure I get what you want to do... Why stop the upload and/or only prepare the packing without uploading? what's the goal.
I've implemented a simple version where I'm packing the next post in queue while uploading the current one. It seems to work. I believe it is enough as packing is in general much faster than posting. So you end up waiting only the first time for the packing then it is uploading in continue the whole queue \o/
Would you guys be able to beta test? I'm still missing a little part to handle new incoming file but it should be ready in a few hours.
What OS are you on? Linux command line? do you compile from the sources?
PS: @Mamitomama are you on the same community of @Tensai75 ?
from ngpost.
the beta is pushed.
Please let me know your feedback.
Open the config file to add PREPARE_PACKING:
## when several Posts are queued, prepare the packing of the next Post while uploading the current one
PREPARE_PACKING = true
from ngpost.
Related Issues (20)
- enable symbols also in auto-generated passwords HOT 1
- edit or delete post HOT 1
- edit a post or delete a post that I made HOT 3
- How does one add posts to categories on indexers? HOT 1
- Save NZB's with full the Full Path name for posted files and folders HOT 3
- handle interruption problem HOT 4
- [Feature] Random poster for EACH_FILE / ARTICLE
- Log for the posting process? HOT 1
- in GUI monitoring and file name obfuscation checked - Endless posting loop
- Problem HOT 5
- When will the new version be released? HOT 13
- [Enhancement][Quality Of Life ''improvement''] just 2 requests. nothing huge or major.
- add command line parameter for rar compression level
- compression and encryption independently
- add in command line output nzb for every input file option
- Question about automated posting
- LinuxMint: Remote host closed the connection
- Auto compress not doing anything (GUI, Windows)
- Have some requests for new version HOT 6
- Autoposting: all files in monitored folder with same password? HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ngpost.