Yikes! Ariana found that the viewer may not display all non-zero pixels in the image when using user-controlled thresholding. Here is an example of a Nu in plane 1, with auto pyqtgraph intensity scale (that's why is looks like shit)

Set the imin threshold a little higher than 0, and there are non-zero pixels in the image if you slice on it the values are...
[ 2.50050282 2.87391782 2.25792432 3.22080231 3.17725968 1.86988342 1.64758134 3.24257207 2.84999657 2.88998723 3.7988553 1.76502311 1.17307568 4.05927277 1.23509991 1.21503234 4.26263475 4.66946697 1.23992372 4.14212132 2.64220333 3.4762044 3.35693979 3.57622218 3.47608447 3.77681541 3.72539306 4.3190136 3.80827284 1.18566716 5.65644979 3.04839444 2.34159589 8.38000679 1.78434169 1.01273596]

but the image is blank...

quick fix is to artificially boost the pixel values by 100 with
self.plot_mat*=100.0 placed on line 124 in

https://github.com/LArbys/LArCV/blob/master/mac/pyrgb/lib/image_types/plotimage.py

Now with an appropriate threshold, we see the pixels appear

I see 7 non zero pixels and it's confirmed on the slice
[ 4.05927277 4.26263475 4.66946697 4.14212132 4.3190136 5.65644979 8.38000679]

I've tried a number of things to get the pixels to scale which include

• pyqtgraph.ImageItem.setLevels
• opacity=1.0 in setImage function
• setCompositionMode to QtGui.QPainter.CompositionMode_Plus (Both the alpha and color of the image and background pixels are added together.)

but so far an arbitrary scaling has resolved this. I will leave this open until I find a better way to address.

At the end of the collaboration meeting today, Stephen asked to make R-G-Cyan a standard for the "RGB" viewer. Can implement a feature similar to false color mode.

The "default" ADC thresholds in the viewer might be better fetched from actual min/max ADC value from pixels in the image.

Then whether updating it or not per event should be probably an option to pick for a user (so a user can "freeze" the threshold going one event to the next).

I don't have a connector yet to send OpenCV altered image to the network, currently just takes original image.

We want to run the viewer with exactly same way image is passed onto the network training and/or forward computation.
New branch: https://github.com/LArbys/caffe/tree/process_driver/src/caffe
... has the implementation we planed already in place.

need this to be implemented in view_rgb.py

ThreadDatumFiller complains that there are two DatumFillers, when infact there is only one specified in apicaffe_cleanup

https://github.com/LArbys/LArCV/tree/apicaffe_cleanup

branch.

On line 50 of ProcessBase.h

virtual bool is(const std::string& question) const { return !(question.empty()); }

if we don't overload it then everything is a DatumFiller according to this statement, causing trouble for APICaffe

[NORMAL]  <TrainProcessDriver::configure> Instantiating Process ID=0 Type: ADCThreshold w/ Name: ADCThres
[NORMAL]  <TrainProcessDriver::configure> Instantiating Process ID=1 Type: SegFiller w/ Name: SegFiller

(ADCThreshold)
Is it datumfiller? 1
Is it an aho? 1

(SegFiller)
Is it datumfiller? 1
Is it an aho? 0

ADCThreshold is is both aho and DatumFiller since we sent a non-empty string.

Fix is to define virtual is(...) for ADCThreshold class and call it something else.

This way averages of a batch can be taken more easily. Right now for random access mode, can't repeat by simply specifying first entry.

To make our life easy we should probably have a simple Process to compress an image instead of other higher level modules to do this.

combined with #7 we want to perform caffe.Net forward computation after changing ADC values... to inspect how probability may change.

otherwise don't.

This is for analyses. This way they can run forward() with ROOT random access and have a way to get entry info without having to figure out way to align trees.

During training, will probably not want to load this info into top blob.

In order to make manual-scan easier, I thought of looping over the larcv-events and plotting 3-images with the three ROIs contents, where the ROIs are given their initial values from our list of GBDT candidates.
Does this make sense?

Hi,
i'd like to navigate in view_rgb using run/subrun/event (rse). I am thinking about looping over all ROI and IMAGE trees inputs for a user-defined rse and when finding the right entry, load the data image and the ROIs initial values. Does it make sense?
To start, i've added three widgets to the view_rgb display (mac/pyrgb/display/) and modified the scripts accordingly, so that now it works, but fairly slow (probably because i'm loading heavy images in every loop...)
can you please help me engineer it better?

Configuration does not handle "tab" correctly (i.e. we want to treat it as space, something that doesn't matter!)

But at least the forward pass is working.

First thing to check is if the image in the datum is in the same order as the data in the Image2D object.

Check the mean blob, too.

Need to check if PMTWeight is applied correctly...

Wouldn't consider it super urgent, though.

Either as a process or a part of simple filler, one would like to be able to ask for a random crop of a training image. This is to help with over-training.

The order of these operations seem to matter a lot when passing to the network. We must do exactly what is done at training time.

Also, applying a threshold probably has to be split into two contexts: 1) for changing the contrast in the viewer and 2) applying it to the image data that is going to be passed into caffe.

Line 25 of SimpleFiller has GuasSmearingSigma instead of GausSmearingSigma and initializes to -1.0. PSet never complains about extra configuration options so I never caught this, and since the var is initialized PSet didn't complain that I didn't set it. Hope this didn't effect anyone... Will leave this open for a bit for people to see, then I can fix.

https://github.com/LArbys/LArCV/blob/master/app/APICaffe/SimpleFiller.cxx#L25

We need unit test. period.

This is true for any package under core or app.

*) core/Base
*) core/DataFormat
*) core/PyUtil
*) core/CPPUtil
*) core/CVUtil

*) app/ImageMod
*) app/ImageAna

... might be a good start imho.
I pushed a unit test for IOManager here:
https://github.com/LArbys/LArCV/blob/develop/core/DataFormat/test/iotest.py

By default it sends no text to stdout nor stderr of the terminal but just return codes.

python iotest.py
echo $?
0

Three different error return codes are defined in the script. One can run verbose mode by providing "debug" or "info" as an argument of the script to change message level.

From #8 and #7

Requested features

• Slide the box and find where network is maximized, or goes above some threshold
• OpenCV in box (ADC scaling, blue et al...) with forward computation

"We want to "choose a range, modify ADC value" functionality.
To get started with, pixel-by-pixel gaussian scaling/smearing would be fine (generate gaussian random draw for specified mean & sigma + multiply to pixel value, performed per pixel)"

• Investigate Forward CPU memory

From the Filler:

  [INFO]  <SimpleFiller::fill_entry_data::L320> Label to Datum: 1
  [INFO]  <test::_batch_process_::L259> Processed good event: valid entry counter = 2
  [INFO]  <test::_batch_process_::L248> Processing entry: 8378
  [INFO]  <testIOManager::get_data::L494> Setting event id (5360,0,35)
  [INFO]  <SimpleFiller::fill_entry_data::L307>     Cosmic PdgCode=0
Vertex   (x, y, z, t) = (0,0,0,0)
Momentum (px, py, pz) = (0,0,0)
Inittial Energy = 0
Deposit  Energy = 0
# Bounding Box  = 0

Looking into the ROOT file:

root [1] partroi_tpc_tree->Show(8378)
======> EVENT:8378
 partroi_tpc_branch = (larcv::EventROI*)0x4e8f630
 _producer       = tpc
 _run            = 5607
 _subrun         = 73
 _event          = 3672
 _part_v         = (vector<larcv::ROI>*)0x4e8f680
 _part_v._index  = 0, 1, 2, 3, 4
 _part_v._shape  = 2, 1, 1, 1, 1
 _part_v._type   = 2, 6, 9, 8, 8
 _part_v._mcst_index = 65535, 0, 1, 2, 4
 _part_v._mct_index = 0, 0, 0, 0, 0

Labeled as cosmic, but definitely a neutrino image.

—— the code ——

// labels
    ROIType_t roi_type = kROICosmic;
    for(auto const& roi : roi_v) {
      if(roi.MCSTIndex() != kINVALID_USHORT) continue;
      roi_type = roi.Type();
      if(roi_type == kROIUnknown) roi_type = PDG2ROIType(roi.PdgCode());
      LARCV_INFO() << roi.dump() << std::endl;
      break;
    }

    // Convert type to class
    size_t caffe_class = _roitype_to_class[roi_type];

    if(caffe_class == kINVALID_SIZE) {
      LARCV_CRITICAL() << "ROIType_t " << roi_type << " is not among those defined for final set of class!" << std::endl;
      throw larbys();
    }

    _label = (float)(_roitype_to_class[roi_type]);

Diagnosis

In the loop, the first entry it sees is a cosmic ROI.

It then breaks, missing the MC ROIs behind it. This get's labeled as cosmic, but it's a neutrino.

... at least I want to make an attempt to get an agreement.

a) Use git flow release
b) Versioning convention, I propose typical 3 level versioning vXX.YY.ZZ where...
b-1) Increment ZZ for completely backward-compatible release
b-2) Increment YY for partially backward-compatible module, defined as backward incompatibility outside "core" directory (so a change in module configuration format, caffe interface, etc)
b-3) Increment XX for backward incompatible change in "core"

The current (initial) motivation is to push out the current develop 29ea5ed that is still backward compatible with the caffe configuration used for long by now.

FCN training so far has loss shot down to ~0.01 pretty quickly which is suspected due to ~99% of pixels being a background. One solution to this is to introduce weight-per-class when computing loss and provide smaller weight for background pixel.

The work is done by someone else already:
mohamed-ezz/caffe@876e387

Just need to be ported in.

Need a dedicated process base class that performs preparation for datum filling for caffe.

Please change it

even when opencv was not used.

sigh....

Wanted: a tool to label pixels by different particle types. Provides a sample of data that has information useful for semantic segmentation task.

please.

Inheriting from other projects, larcv has good portion of code with doxygen comments.
I think we should extend those comments and generate doxygen rawgit page.

0. Enable doxygen script
1. Set up automated doxygen running either per commit or periodically
2. Set up web page w/ doxygen output OR push doxygen outcome to rawgit
3. Extend doxygen comments to non-commented code

... are the to-do's.

The branch apicaffe_cleanup has a lot of enhancements (segmentation training and full implementation of ProcessDriver for caffe training) while it is supposed to support old features as well. We have to merge back into master at some point.

We must make sure to test with:

*) Supera
*) Neutrino + Cosmic overlay image generation
*) Semantic Segmentation training
*) Single particle classification training

yamlcpp exists on github. probably fork one version and put inside larcv?

OpenCV only alters the image that has been modified to avoid overlapping pixels on the viewer, this has to change so that the original image is modified not the one with missing pixels in each channel. Not sure how to resolve this yet

Thought I'd open a fresh issue with all requested features

1. The original image image (loaded from ROOT file) is thresholded according to min and max. Hitting "replot" loads a new image from the ROOT file, thresholds it and sends to the screen. There is an intermediate layer that makes sure pixels do not overlap and cause false color. The original image is not mean subtracted, but thresholded. When the image goes down to caffe, it's mean is subtracted from the thresholded image. Is this ok? No, it depends on how you trained your network i.e.

• threshold and mean subtract (from perhaps mean image of the thresholded image)
• or mean subtract and then threshold

this issue is unsolved by the viewer
2. You can modify the channels individually on the viewer, the ones you don't modify don't currently get sent to caffe.
3. Slide the box and find where network is maximized, or goes above some threshold

A pretty picture of fake neutrino image, then me removing neutrino, the network knew what it was! See the "score" at the bottom of the screen. Left is cosmic, right is nu

I copy and paste background on top of the nu, score flips

I0505 01:19:39.141175 4696 layer_factory.hpp:77] Creating layer input
I0505 01:19:39.141774 4696 net.cpp:91] Creating Layer input
I0505 01:19:39.141790 4696 net.cpp:399] input -> data
I0505 01:19:39.142422 4696 net.cpp:399] input -> label
I0505 01:19:39.142920 4696 net.cpp:141] Setting up input
I0505 01:19:39.142936 4696 net.cpp:148] Top shape: 1 3 768 768 (1769472)
I0505 01:19:39.142945 4696 net.cpp:148] Top shape: 1 (1)
I0505 01:19:39.142951 4696 net.cpp:156] Memory required for data: 7077892
I0505 01:19:39.142958 4696 layer_factory.hpp:77] Creating layer data_trimese
I0505 01:19:39.143282 4696 net.cpp:91] Creating Layer data_trimese
I0505 01:19:39.143296 4696 net.cpp:425] data_trimese <- data
I0505 01:19:39.143306 4696 net.cpp:399] data_trimese -> Slice1
I0505 01:19:39.143317 4696 net.cpp:399] data_trimese -> Slice2
I0505 01:19:39.143326 4696 net.cpp:399] data_trimese -> Slice3
F0505 01:19:39.143342 4696 blob.cpp:32] Check failed: shape[i] >= 0 (-5 vs. 0)

In many ProcessBase inherit process class we don't use TFile* fout which is given in the argument of finalize() call. This causes compiler warning an extremely annoying. Should be fixed @ ProcessBase class. Requires breaking change to all ProcessBase inherit class.

I ran into an annoying snag running multiple inference in parallel with pycaffe. I like the threading feature implemented in ThreadDatumFiller but I think 1 millisecond delay usleep(1000) in batch_process located here

https://github.com/LArbys/LArCV/blob/develop/app/APICaffe/ThreadDatumFiller.cxx#L246

could be improved. If I launch multiple ThreadDatumFillers at once, each with their own threads (for example multiple inferences in parallel), the thread may not allocate and _batch_process_ may not begin execution in time for heproot.cpp.

This means that the ProcessDriver instance may not be initialized for the first time, and _processing may be false when heproot.cpp queries ThreadDatumFiller for the data dimensions, as they may not be available yet. Yikes! Maybe replacing usleep(1000) with

 while (!_processing)                                                                                                                                              
        usleep(1000);                                                                                                                               

is better. This would ensure that the first thread creation time doesn't race against the first memcpy here

https://github.com/LArbys/caffe/blob/master/src/caffe/util/heproot.cpp#L38

Specific example is the 12 channel data used in the previous neutrino ID network. Channels go (TPC image, pmt weighted, MIP channels, HIP channels) x 3 planes.

I imagine that we will have many combinations of channels.

There are inconsistency handling of OpenCV dependency.
Original intent (which I plan to follow) is:
0) shell environment variable OPENCV_INCDIR and OPENCV_LIBDIR must be defined each pointing to a conventional top directory path to headers and libraries of opencv.

1. when 0) is satisfied, configure.sh sets LARCV_OPENCV environment variable to 1 (or 0 otherwise). This should be used in package makefile to judge whether or not to use OpenCV.

Issue made apparent by Tari: we should set a compiler variable to set compiler guards in source codes.

I am going to make a note of desired features/bugs of my first tests of the roi feature (thanks vic!).

I implemented the Unpooling layer based of the work of HyeonwooNoh

https://github.com/HyeonwooNoh/caffe

And it visually does what you expect it to. I also tried out Jeff Donahoue's implementation and it just plain didn't work (BVLC/caffe#3601) even after monkeying with it. I pushed to the apicaffe_cleanup branch.

new commit allows for enum to string conversion, integrate this into viewer 679c7ba

We need:

*) Threaded read-and-process method
*) Ideally runs ProcessDriver so we can attach Process and execute on the fly
*) Need base class to define how to fill datum (i.e. hand over data to caffe)

Error messages from compilation.

But do we care? I am upgrading to 6 as we speak (and fly)

In file included from /Users/twongjirad/working/larbys/LArCV/core/build/Base/BaseDict.cxx:20:
In file included from /Users/twongjirad/software/root_v5.34.34/include/TClass.h:42:
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:228:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(T_) ptr = NULL;
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:229:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(Bool_t) isInit(kFALSE);
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:239:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(Array_) ptr = NULL;
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:240:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(Bool_t) isInit(kFALSE);
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:250:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(T_) ptr = NULL;
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:251:4: error: thread-local storage is not supported for the current target
TTHREAD_TLS(Bool_t) isInit(kFALSE);
^
/Users/twongjirad/software/root_v5.34.34/include/ThreadLocalStorage.h:103:29: note: expanded from macro 'TTHREAD_TLS'

define TTHREAD_TLS(type) thread_local type

                        ^

6 errors generated.
make[1]: *_* [/Users/twongjirad/working/larbys/LArCV/core/build/Base/BaseDict.o] Error 1

We want to "choose a range, modify ADC value" functionality.
To get started with, pixel-by-pixel gaussian scaling/smearing would be fine (generate gaussian random draw for specified mean & sigma + multiply to pixel value, performed per pixel)

This way, I can erase data from all channels.

Right now, I have to hit Replot to change the viewed channels. But this reloads the image from disk and undoes any modifications.

For certain scanning tasks, we might want to be able to set a vertex in conjunction with the ROI. The immediate use case is for scanning Michel electrons. The ROI encapsulates the entire shower while the vertex marks the point of the decay. Together, along with Kalman filter, we'd start from vertex and follow muon in order to mask it, leaving only the electron charge. Also, vertex provides target for regression.

I've made a first attempt at 0511351

An example of what a user might label is here:

If can develop algorithm to follow muon, then we can avoid having users mark pixels directly.

Lock the contrast if user is not allowed to change. To unlock user contrast you have to click the checkbox marked "User Contrast".

Need (run,subrun,event) id to be shown on the viewer!
These are available from any of EventBase inherit class (EventImage2D, EventROI, etc) via function EventBase::run(), EventBase::subrun(), and EventBase::event().

Should we have a check between use_thread: in caffe's prototxt and UseThread: in the ProcessDriver configuration? I can set use_thread: false in my root_data_layer configuration, but my filler will still use threading as we not making a check. Users may forget to set both.