I've noticed that when running ldmx-sw in run mode, if one passes a small number of e

Crash when processing a small number of events when passing a large list of inputFiles about framework HOT 2 CLOSED

ldmx-software commented on August 19, 2024

Crash when processing a small number of events when passing a large list of inputFiles

from framework.

Comments (2)

tomeichlersmith commented on August 19, 2024

In the Recon run mode branch of Process::run, there is a check for if the processing has reached the event limit.

Framework/src/Framework/Process.cxx

Lines 262 to 263 in 638f0a1

 while (masterFile->nextEvent(storageController_.keepEvent(event_completed)) 

 && (eventLimit_ < 0 || (n_events_processed) < eventLimit_)) {

This check is only within each input file though, so that is the bug causing the input files to continuously be opened and immediately closed after the event limit is reached. What's concerning to me is that this is eventually causing a seg-fault. The Bus::everybodyOff function clears the internal event object map thereby deleting all event objects from memory, I'm not sure how it is obtaining pointers that have been already deleted.

Adding a break; in the following blocks and moving these blocks to after the input/output files are closed should resolve this issue.

Framework/src/Framework/Process.cxx

Lines 294 to 300 in 638f0a1

 if (eventLimit_ > 0 && n_events_processed == eventLimit_) { 

 ldmx_log(info) << "Reached event limit of " << eventLimit_ << " events"; 

 } 

 if (eventLimit_ == 0 && n_events_processed > eventLimit_) { 

 ldmx_log(warn) << "Processing interrupted"; 

 }

I am working on writing a small testing area for this so I can validate that this works.

from framework.

tomeichlersmith commented on August 19, 2024

I am not able to reproduce the crash; however, I am able to observe the open/close of the files after the event limit is reached. I will try implementing the fix and then see if that resolves the crashing.

from framework.

Recommend Projects

Crash when processing a small number of events when passing a large list of inputFiles about framework HOT 2 CLOSED

Comments (2)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	while (masterFile->nextEvent(storageController_.keepEvent(event_completed))
	&& (eventLimit_ < 0 \|\| (n_events_processed) < eventLimit_)) {

	if (eventLimit_ > 0 && n_events_processed == eventLimit_) {
	ldmx_log(info) << "Reached event limit of " << eventLimit_ << " events";
	}

	if (eventLimit_ == 0 && n_events_processed > eventLimit_) {
	ldmx_log(warn) << "Processing interrupted";
	}