Sub stats showed as an unique progress bar about tqdm HOT 30 CLOSED

I believe this is solved if you set the smoothing kwarg to something small (close or equal to zero). Let me know if you can think of a better/different solution

from tqdm.

Also tqdm supports subclassing very well - tqdm_gui for example is a child class of tqdm

from tqdm.

Let me know if you prefer and have (attempted to) implement a subclass, I'll reopen the issue and we can add it here.

from tqdm.

Subclassing would be an ideal solution if module functions like "format_meter()" or "StatusPrinter()" would be methods of class tqdm. Then I could replace them easily in a subclass. For compatibility reasons maybe would be easier to add them as constructor parameters (with default values = current functions).

Without this help, I must rewrite most logic in my subclass.

from tqdm.

I could make both @classmethods. Would that work for you?

from tqdm.

Agree, this would make subclassing way easier and more powerful. However, I hope this won't hit performance too much, we'll test that out (as @classmethod or @staticmethod, this may work great without any perf hit).

from tqdm.

No idea about performance but yes, this would help a lot.

from tqdm.

Great. Any date for a PYPI release?

from tqdm.

er... yeah. when I get around to feeling OK with #87 ... Spent ages trying to optimise it ;)

from tqdm.

Crossing fingers and biting my nails...

from tqdm.

well.. I just tagged and released v3.7.0 on github which includes the inheritance version... so if you want to bug @lrq3000 he could release it on pypi.

from tqdm.

Great work Casper!
However the version is still set to 3.6.0, and I don't know how to fix that
directly in the master branch without using a PR...

2016-01-13 0:33 GMT+01:00 Casper da Costa-Luis [email protected]:

well.. I just tagged https://github.com/tqdm/tqdm/releases/tag/v3.7.0
which includes the inheritance version... so if you want to bug @lrq3000
https://github.com/lrq3000 he could release it on pypi.

—
Reply to this email directly or view it on GitHub
#77 (comment).

from tqdm.

I still changed the version and upload to pypi so we just need to fix the
GitHub release.

2016-01-13 11:48 GMT+01:00 Stephen LARROQUE [email protected]:

Great work Casper!
However the version is still set to 3.6.0, and I don't know how to fix
that directly in the master branch without using a PR...

2016-01-13 0:33 GMT+01:00 Casper da Costa-Luis [email protected]:

well.. I just tagged https://github.com/tqdm/tqdm/releases/tag/v3.7.0
which includes the inheritance version... so if you want to bug @lrq3000
https://github.com/lrq3000 he could release it on pypi.

—
Reply to this email directly or view it on GitHub
#77 (comment).

from tqdm.

I'm updating the setup.py classifier troves, so I'm going to update to version 3.7.1 by the way.

from tqdm.

oops my bad. Forgot to double check _version.py

from tqdm.

nvm it happens :) But do you have a way to update _version.py directly on master branch without using a PR? For instance, I saw you can change the version during a merge, but I didn't see the version changes in the branches. Can you do that during the merge or in fact you push to the branch before merging to master?

from tqdm.

Please, write a changelog too, inlined or linked in PYPI. I hate to "diff" between releases to know what is new :). Something to add to the "RELEASE" file :-).

from tqdm.

@jcea I write changelogs on github (https://github.com/tqdm/tqdm/releases). If you want to look through those and add them to the RELEASE file in a PR, I'll be happy to merge it in.

from tqdm.

@lrq3000 I can update _version.py with a force-push bit I already tagged v3.7.0 which makes it a bad idea to tamper with. Doesn't matter since the 3.7.1 release. I notices you copied over the release notes, too,and removed them from the 3.7.0 tag.

from tqdm.

@casperdcl Yes the 3.7.1 was just an excuse to fix the versioning by fixing the setup.py file by the way ;) 3.7.0 on Github should not be used since its version is not correct, but the one on PyPi is OK.

from tqdm.

@jcea I'm not really a huge fan of putting all changes in a CHANGELOG, because it pollutes the commits (we already have the git commit + GitHub releases descriptions to detail and summary the changes). Sure, I've done it a lot in the past for other projects, but it was just because I didn't have another place to put the changes (ie, I was not using git nor github releases).

But well, if you really think that's a necessary addition, then why not...

from tqdm.

@casperdcl, I was trying to say that the project should include a "changelog" file in the package, and it should be pasted or linked in PYPI.

My opinion, of course :).

In this particular case, a link in PYPI (named "changelog") pointing to https://github.com/tqdm/tqdm/releases would be enough for me.

from tqdm.

@lrq3000, think about people unfamiliar with the project. The changelog is for those people, not for you :-).

I have been using this project for a month and even providing some ideas and PR, and I was lost when 3.7.1 was released. I just read the code to review the changes myself.

"Changelog" is standard practice for a reason :).

from tqdm.

I'm not saying we don't need a changelog, but rather that we already have a
changelog in the form of Github Releases.

2016-01-14 3:16 GMT+01:00 jcea [email protected]:

@lrq3000 https://github.com/lrq3000, think about people unfamiliar with
the project. The changelog is for those people, not for you :-).

I have been using this project for a month and even providing some ideas
and PR, and I was lost when 3.7.1 was released. I just read the code to
review the changes myself.

"Changelog" is standard practice for a reason :).

—
Reply to this email directly or view it on GitHub
#77 (comment).

from tqdm.

Another possibility instead of manually maintaining a CHANGELOG: we can
generate a changelog from git commits on-the-fly at the moment of building,
and include it in the package. Would that be ok?

2016-01-14 8:52 GMT+01:00 Stephen LARROQUE [email protected]:

I'm not saying we don't need a changelog, but rather that we already have
a changelog in the form of Github Releases.

2016-01-14 3:16 GMT+01:00 jcea [email protected]:

@lrq3000 https://github.com/lrq3000, think about people unfamiliar
with the project. The changelog is for those people, not for you :-).

I have been using this project for a month and even providing some ideas
and PR, and I was lost when 3.7.1 was released. I just read the code to
review the changes myself.

"Changelog" is standard practice for a reason :).

—
Reply to this email directly or view it on GitHub
#77 (comment).

from tqdm.

The most simple approach for you is just to add a link in PYPI to the github changelog page.

from tqdm.

Going back to the original aim of this issue :-), I have coded my idea using TQDM 3.7.1 ability to subclass tqdm class easily.

Here is the result:

class subprogress(tqdm.tqdm) :
    def __init__(self, *args, **kwargs) :
        self._metaprogress = None
        super(subprogress, self).__init__(*args, **kwargs)

    @staticmethod
    def format_meter(*args, **kwargs) :
        pass

    def status_printer(self, *args, **kwargs) :
        def print_progress(dummy) :
            if self._metaprogress :
                self._metaprogress()

        return print_progress

    def set_metaprogress(self, metaprogress) :
        self._metaprogress = metaprogress

    def __call__(self) :
        self.update()

class metaprogress(tqdm.tqdm) :
    def __init__(self, path, initial, num_fragmentos) :
        print(os.path.basename(path), file = sys.stderr)
        super(metaprogress, self).__init__(total = num_fragmentos,
                                           initial = initial,
                                           unit = "block",
                                           dynamic_ncols = True,
                                           miniters = 1,
                                           mininterval = 0)
        self._total = num_fragmentos

        self._subprogresses = []
        self.lock = threading.Lock()

    def new_subprogress(self) :
        progress = subprogress(
                total = self._total,
                miniters = 1,
                mininterval = 0)
        self._subprogresses.append(progress)
        progress.set_metaprogress(self)
        return progress

    def __call__(self) :
        with self.lock :
            n = self.n + sum((progress.n for progress in self._subprogresses))
            rate = None
            for progress in self._subprogresses :
                avg = progress.avg_time
                if avg :
                    if rate is None :
                        rate = 1/avg
                    else :
                        rate += 1/avg
            self.sp(self.format_meter(n, self._total, self._time() - self.start_t,
                (self.dynamic_ncols(self.fp) if self.dynamic_ncols else self.ncols),
                self.desc, self.ascii, self.unit, self.unit_scale, rate, self.bar_format))

I hope this is useful to anybody. Remember that I usually have like 2-3 updates per second, so performance is a non issue. Also, the code is running in Python 3.5, I don't care about 2.x anymore. Finally, this code solves my use case, it is not ready-to-use generic code.

It is an example. Feel free to add it to the examples collection.

Thanks for your work in tqdm. It is highly appreciated.

from tqdm.

Ok, thanks. Could you provide a minimal example use case for this code?

from tqdm.

Sure. This is a fragment of my production code:

progress_bar = metaprogress(name, done, initial)
[...]
for i in range(concurrency) :
    futures.append(executor.submit(WORKER, progress = progress_bar.new_subprogress())

I create a "metaprogres" bar with the details: name, total, initial. Then I create threading workers passing to them a new instance each of a "virtual" progress bar with "metaprogress.new_subprogress()".

Each worker call its private "subprogress" object. They keep stats private and call the "metaprogress" parent to totalize and actually display the actual progress bar. Since this could be called by several threads at the same time, I use a lock to protect the totalization & display. If contention is low, the lock is not a performance problem (it will be almost always unlocked), and if the contention is high, you better have a lock protecting you :-)

I hope I have explained it easy enough. Let me know if not.

from tqdm.

Thank you @jcea, we'll see if we can turn that into a tqdm subclass.

from tqdm.