Consistent performance tests about tqdm HOT 6 CLOSED

So, to summary, we need a way to measure CPU time (process-wide time = relative time since process started) instead of system-wide, absolute time. This is well explained in this SO answer.

[Relative time is a ] type of time that has no defined relationship to real-world time, in the sense that the relationship is system and implementation specific. It can be used only to measure time intervals, i.e. a unit-less value which is proportional to the time elapsed between two instants. This is mainly used to evaluate relative performance (e.g. whether this version of code runs faster than that version of code).

To achieve that, the best candidate seems to be time.monotonic() which has the advantage of being monotonic, which means that the time can't go backward. Unfortunately, on recent OSes, it has become system-wide, so we cannot use that.

The second best candidate is time.process_time(), and it has all the criteria. However, it's only available on Python 3.3+, and it seems there's no backward port yet using __future__.

However, there is also time.clock(), which is process-wide. But there are 3 issues:

1- time resolution differs between Windows and Linux.
2- works differently on Windows and Linux (ie, on Linux it works just like time.time(), but on Windows it will return the difference since the first call to time.clock()).
3- It has been deprecated in Python 3.3+.

So I guess we want to use process_time() on Python 3.3+, and use time.clock() on Python 2, and we have to try to manage manually the difference of usage between Windows and Linux.

from tqdm.

Note that because of time.clock(), we may also have to make some tests slower in order for time.clock() to have enough resolution to time them...

from tqdm.

PS: there ARE implementations of monotonic time for previous versions of Python, but they all require either compilation of c files, or using ctypes:

• https://github.com/ThomasHabets/monotonic_clock
• https://github.com/gavinbeatty/python-monotonic-time
• http://stackoverflow.com/a/1205762/1121352
• https://github.com/atdt/monotonic

from tqdm.

I added relative timing for all the tests that required it. I did have to add a new internal parameter in tqdm class to support relative timing directly inside tqdm (I hope that's OK, because I couldn't think of any more elegant solution...).

We should try out this PR by restarting several times the Travis build at different times of the day, just to ensure that all tests always pass.

from tqdm.

Hmm I think there may be a way to avoid the extra parameter... will look at this later

from tqdm.

@casperdcl Ok thank's.

About the PR, the perf test just failed, the manually updated tqdm took way much longer than usual. So this PR doesn't completely fix the issue... But it seems it still somewhat helps.

from tqdm.