MicMac results in vignetting and holes (no data) in orthomosaic about nodemicmac HOT 26 CLOSED

Thanks for providing detailed information and screenshots. I think there are a few optimizations to the Ortho/Radiometric Balance process that may improve your situation. I'll do an update tomorrow and you can re-test/process.

from nodemicmac.

@kikislater no worries, probably other Tawny commands / options as well. I switched it over to Porto with a custom config xml that we've been using at DroneMapper. Should work better for many cases.... fingers crossed
dronemapper-io/micmac@1646049

from nodemicmac.

MM and NodeMicMac updated, should improve results: fa7a53e
dronemapper-io/micmac@c6923e6

from nodemicmac.

closing for now, please attempt processing with latest docker and improved parameters. thanks

from nodemicmac.

Updated to WebODM 1.0.0 via command: $ ./webodm.sh update

after update was successful ran command $ ./webodm.sh restart --with-micmac

Then reprocessed same 697 images but till get same results as before. Is this sufficient to capture your changes?

Thank you

from nodemicmac.

Nop, you should update with:

./webodm.sh update --with-micmac

from nodemicmac.

After update --with-micmac reprocessed same 697 images (with exif tags) now I get no Orthophoto or Point Cloud ??

(btw, when I try to download the console file it gives me a Failed - Network error)

DSM that was generated appears to have some differences from the previous process. The overall range in raster values (meters) went from 44.0235 - 27.1404 to 44.6808 - 26.5689, (16.8831‬ to 18.1119).

Q1: is there an indication in the processing that exif tags are not being correctly processed?
Q2: are dsm files generated from the same original photos going to vary from each processing?

Thanks

from nodemicmac.

Sorry for the problems. If you could share the options/parameters you used for processing that might help. Also, if you would like to share the log file privately -- we can evaluate.

Q1 reply: The only indication is the failure of the processing after tie-point extraction and during the first bundle block stage
Q2 reply: Yes the DSM could differ but it shouldn't significantly change

Something that might help would be to use the "multi-scale" tie-point options and set the initial image size to 800. It will take longer but produce more feature matches in homogenous terrain.

If you could share the dataset privately, I could evaluate that as well.

from nodemicmac.

Options were default. Log file download from webodm fails. Do you have the path to the file in docker?

Before the last update I tried the 'multi-scale' option but it failed, seemed to overload already minimal memory resources.

Unfortunately I can not share the dataset.

from nodemicmac.

No worries, understand. If the 'multi-scale' option fails due to server resources / memory that could be related to the issue. You can try the default settings but use the initial image size as 1000 or more pixels. I noticed it took 62 hrs to process which seems abnormally long.

from nodemicmac.

...It has always taken 60+ hours. However, the micmac orthomosaic qualities (minus the vignetting and no data holes) seemed to be better than the default of any other cloud based processing option...is there any system benchmarking data for processing times?

Do you have an alternative path to the console (process log) file? Hoping that will provide insight into what is causing the problem.

from nodemicmac.

I'm not sure if WebODM saves the console log somewhere. Maybe others can help identify that location if it exists?

from nodemicmac.

I'm currently reprocessing original 697 images running default options with nodeMicMac on a Google Cloud Platform instance (Machine type: n1-standard-8 (8 vCPUs, 30 GB memory; 100GB storage) in hopes it will isolate any local resource issues.

The inital console output is indicating the following:

Error: Directory Iop: Next pointer is out of bounds; ignored.
Error: Directory (Last IFD item) with 65205 entries considered invalid; not read...

(error message is repeated with different numbers before the 'entries', eventually followed by a subset of the images like shown below, then more 'error:'...followed by the next subset of images)

Not sure if this error was present when I first ran MicMac and was able to generate an orthophoto on the same dataset, but so far this is the only indication something is not processing correctly...

from nodemicmac.

MicMac failed to generate an Orthophoto & LAS on the dataset it had previously been successful with. The failure was the same on a Google Cloud instance so I can't see it being a local resource issue...

...although it only took 13 hrs on Google Cloud vs 60+ on my laptop.

Complete console output file here

from nodemicmac.

Thanks for the log. That helps! I believe the problem might be a recently introduced option to Tawny DEq=1 or other options ... I will reverse that and push a new release. It might help.

[DEBUG]   running mm3d Tawny Ortho-MEC-Malt DEq=1 RadiomEgal=1 DegRapXY=4 SzV=25
------------------------------------------------------------
|   Sorry, the following FATAL ERROR happened
|
|    REadPt
|
------------------------------------------------------------
-------------------------------------------------------------
|       (Elise's)  LOCATION :
|
| Error was detected
|          at line : 1169
|          of file : /var/www/micmac/src/util/pt2di.cpp
-------------------------------------------------------------
Bye  (press enter)
[ERROR]   Child returned 1
[DEBUG]   running mm3d Nuage2Ply MEC-Malt/NuageImProf_STD-MALT_Etape_9.xml Attr=Ortho-MEC-Malt/Orthophotomosaic.tif 64B=1 Out=/var/www/data/73277a88-0d38-4a6a-8148-5553eeeb62a6/odm_georeferencing/odm_georeferenced_model.ply
ERROR: colour image [Ortho-MEC-Malt/Orthophotomosaic.tif] does not exist
[ERROR]   Child returned 1

from nodemicmac.

I've made some updates to the ortho creation/color balance. 4070a6e

I would wait for docker hub to finish building the new image and give it a try. Thanks

from nodemicmac.

@dronemapper-io strange for DEq=1!

from nodemicmac.

After running update --with-micmac attempt 2 on GCP with MicMac failed to generate an Orthophoto and render PLY...

...I was able to download a .ply file individually that was in the MB range of previous MicMac processing, but no luck with the orthophoto

Console output here

from nodemicmac.

I've reverted one other MicMac commit -- you can test it once the docker image is done building. dronemapper-io/micmac@b947576

I would also recommend testing your entire process with the DroneMapper 4th Ave. example data set as well. Thanks

from nodemicmac.

After starting a new instance on Google Cloud and running latest updates yesterday the issues with no ortho/PLY outputs on the 697 image set persisted, same output as noted July 1 (screenshots here)

However I did run a different 263 image set, over varying forest conditions, on a new instance Google Cloud and was able to get all the outputs...though there are some ortho and dsm discrepancies. The ortho/dsm comparison between nodeODM and nodeMicMac is here

Btw, is there a way to get a '.las' file instead of the '.ply' in the MicMac output?

Initial assessment of 4th Ave output using node MicMac in Google Cloud looks good (screenshot here)

from nodemicmac.

Update
Still no luck with the 697 image dataset...latest console output txt file and WebODM screenshots here

Any thoughts on why the ortho is not rendering?

Just fyi, after analyzing the 4th Ave ortho and dsm my processing varied considerably, particularly for the trees...GCP instance is 8vCPUs x 30GB memory x 150GB storage: monitoring graphs
Results .zip
Console output txt file

from nodemicmac.

I'll take a look.. but unfortunately, I can't really diagnose and fix the issue or determine the issue without the data/imagery. Thanks

from nodemicmac.

from nodemicmac.

We can definitely take a look and try the processing locally with nodeMicMac.. please send a contact via https://dronemapper.com

from nodemicmac.

I wasn't able to fix the bug yet. But for this data set a potential workaround is to remove every other image and run the processing. I got a well balanced Ortho, DEM and Point Cloud. I shared the results privately to you. Thanks

from nodemicmac.

Closed due to inactivity

from nodemicmac.