Investigate Potential to bypass LogRecord cloning when emitting LogRecord to LogProcessor(s) about opentelemetry-rust HOT 3 OPEN

Ok, It was much simpler to add a special case to avoid LogData cloning in case of single processor ( I believe that would be the most common use-case). Have added that change as part of #1636 (as changes are closely tied to that PR). I can see vast improvement in stress test with that:

Main branch:

Number of threads: 8
Throughput: 16,356,200 iterations/sec
Throughput: 15,894,400 iterations/sec
Throughput: 16,018,200 iterations/sec
Throughput: 16,344,800 iterations/sec
PR branch: (with Resource propagation )

PR branch: (with Resource propagation at startup):

Number of threads: 8
Throughput: 19,301,400 iterations/sec
Throughput: 19,789,000 iterations/sec
Throughput: 20,133,600 iterations/sec
Throughput: 19,763,000 iterations/sec

PR branch: (with Resource propagation + avoid LogData clone for single processor)

Number of threads: 8
Throughput: 28,997,800 iterations/sec
Throughput: 29,057,800 iterations/sec
Throughput: 28,755,800 iterations/sec
Throughput: 29,712,000 iterations/sec

from opentelemetry-rust.

I meant to say - its best if we can do this, irrespective of the number of processors, not just optimize for the 1-processor scenario. Span current does the special case for single processor..But I think it should be possible to always pass ref or mut ref to processors.

from opentelemetry-rust.

updated title accordingly. The problem is, as of now there is no-single owner of the created LogRecord. So, to achieve passing ref/mut-ref, we need to first have an owner of this LogRecord, and then ensure that the record is not dropped while any of the references is being used. Need some investigation.

from opentelemetry-rust.