Almost all threads are named "bees" now about bees HOT 9 CLOSED

Task creates its own threads and doesn't use BeesThread, so it doesn't set the BeesThread-specific variables. That said, Task could use pthread_setname_np and BeesThread could use pthread_getname_np and then external tools (like iotop and debuggers) would all get the same names too.

Note that even if I fix this, it means most of the logs will end up with thread names like "worker_7", which is little more information than the tid in "pid.tid". The threads that remain will do only basic housekeeping.

from bees.

Well, so far that sounds like correct behavior. But in my logs I notice that the crawler no longer identifies itself as "crawl" but shows "bees" now. Was this intended?

from bees.

I haven't really thought about that much.

Certainly the transition from a stateful-thread model (where each thread contains state, e.g. subvol crawl position) to a stateless-thread model (where a fixed number of threads execute self-contained tasks in dependency order) was intentional. The logging code is based on the earlier stateful-thread assumptions which are not so useful in a stateless-thread world.

Task currently doesn't have a "name" field that we could write to the log (though we could add one).

BEESLOG could just look at Task::current_task() and run the Task's "print" function as needed--but that could be very verbose. Then again, only two "print" functions have been written in Bees, so as long as we just keep the "print" output short we're OK.

I have a patch that logs worker_7 instead of bees but I'm not sure I like it. worker_7 is not an informative thread name, and bees is shorter. I might try fetching Task::current_task() and writing that to the log as the "thread name" instead.

from bees.

Ah okay, now I start to understand the problem. The "stateless threads" was the missing key. Maybe the correct name could be pulled from the self contained task?

I don't like the name worker neither. On the other hand, constantly changing how a thread id names itself in the log, could be confusing at best. Tho, it would make it more easy to extract, as an example, all crawler work from the logs...

from bees.

Well, we could leave the thread names alone (calling pthread_setname_np several hundred times a second is probably not ideal) but have the log ask the Task to name itself directly as required (having every Task write a log entry isn't ideal either).

The Task names are things like "scan extent 0xf33234000" which could be useful to log (e.g. if there was an IO error, it would tell us the extent that was being processed at the time). On the other hand, I was planning to have Tasks with names like "dedup ..." (and the 3 lines of output that includes) and maybe we don't want to print those. Or on the gripping hand, the Task name could be "dedup 16K" and it could log the rest in only two lines, which would be a net improvement in many ways.

from bees.

At the first glance, I like the very last idea best.

from bees.

The master branch now writes the Task name on the log. Currently I made it look like the old subvol-threads output because the extent ref spec (BeesFileRange) was far too long. It will change over time as I get bees to be more extent-oriented instead of extent-ref oriented.

Let me know what you think.

from bees.

This looks more familiar now... Not sure if you use systemd, but it looks like this now:

from bees.

To me it looks good currently. If you feel like closing this, proceed.

from bees.