enricoahlers / agi2nerf Goto Github PK

View Code? Open in Web Editor NEW

106.0 106.0 14.0 24 KB

Simple tool for converting Agisoft XML files to NERF JSON files for https://github.com/NVlabs/instant-ngp

License: Other

Python 100.00%

3d-reconstruction agisoft-metashape computer-graphics computer-vision nerf neural-graphics

agi2nerf's People

Contributors

Stargazers

Watchers

Forkers

ashishd yttiktak 3a1b2c3 kpjmcg miguelbandera moj-dev saizyca mszarvas joreeves datomi79 hiroaki-santo peamon n1ckfg

agi2nerf's Issues

RuntimeWarning: invalid value encountered in true_divide totp /= totw

z:\instant-ngp\scripts\agi2nerf.py:41: RuntimeWarning: invalid value encountered in true_divide
totp /= totw
[nan nan nan]

Results in a 1kb output file with only one pose in it

Child index out of range

I'm getting this error when running the script

File "E:\instant-ngp\instatest\scripts\agi2nerf.py", line 117, in
p2 = float(calibration[8].text)
IndexError: child index out of range

This is the first part of my camera.xml

<?xml version="1.0" encoding="UTF-8"?>
<document version="1.5.0">
  <chunk label="Chunk 1" enabled="true">
    <sensors next_id="1">
      <sensor id="0" label="unknown" type="frame">
        <resolution width="2160" height="3840"/>
        <property name="layer_index" value="0"/>
        <bands>
          <band label="Red"/>
          <band label="Green"/>
          <band label="Blue"/>
        </bands>
        <data_type>uint8</data_type>
        <calibration type="frame" class="adjusted">
          <resolution width="2160" height="3840"/>
          <f>3183.0178924306147</f>
          <cx>-3.0191925037205407</cx>
          <cy>12.686446764725096</cy>
          <k1>0.063641885377290805</k1>
          <k2>-0.071023441493781495</k2>
          <p1>-0.00066411558984484957</p1>
          <p2>0.00010212604713778139</p2>
        </calibration>
        <covariance>
          <params>f cx cy k1 k2 p1 p2</params>
          <coeffs>4.1246824410526330e-02 1.3776612207456870e-03 -4.2411697298901437e-02 3.9313934678412929e-06 5.1613719565122260e-06 3.7981705742701592e-07 -2.9472749628554441e-06 1.3776612207456870e-03 1.1165396229370850e-01 -6.6260286477075336e-04 -1.0713637960316206e-06 1.1724380451499228e-06 1.3390395933655887e-05 -3.5979429998066325e-07 -4.2411697298901437e-02 -6.6260286477075336e-04 1.5522734082239464e-01 -9.1994667509820920e-06 -2.6694619398151444e-06 -5.7232391216941584e-07 1.2638657326310680e-05 3.9313934678412929e-06 -1.0713637960316206e-06 -9.1994667509820920e-06 1.8435970633465647e-08 -3.2760959388177222e-08 -1.4119818296228413e-12 -5.1504711251021114e-10 5.1613719565122260e-06 1.1724380451499228e-06 -2.6694619398151444e-06 -3.2760959388177222e-08 7.2144288085468655e-08 -4.5890939388192768e-11 -3.3191860613579288e-10 3.7981705742701592e-07 1.3390395933655887e-05 -5.7232391216941584e-07 -1.4119818296228413e-12 -4.5890939388192768e-11 1.8001366517951657e-09 -7.3755948430424168e-11 -2.9472749628554441e-06 -3.5979429998066325e-07 1.2638657326310680e-05 -5.1504711251021114e-10 -3.3191860613579288e-10 -7.3755948430424168e-11 1.3159310546254461e-09</coeffs>
        </covariance>
      </sensor>
    </sensors>
    <components next_id="1" active_id="0">
      <component id="0" label="Component 1">
        <region>
          <center>4.2977201944667870e-01 1.7502952481358156e+00 -5.0967459918202662e+00</center>
          <size>5.7349717330932620e+01 2.9160322952270505e+01 2.2148103713989258e+01</size>
          <R>-4.2651881859495350e-01 -3.1558976947696127e-01 8.4763482395772505e-01 8.9233607291355810e-01 -2.9986785270389826e-01 3.3736568274197981e-01 1.4770927649127080e-01 9.0026794251280085e-01 4.0951141781663902e-01</R>
        </region>
        <partition>
          <camera_ids>0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99</camera_ids>
        </partition>
      </component>
    </components>
    <cameras next_id="100" next_group_id="0">
      <camera id="0" sensor_id="0" component_id="0" label="frame0000">
        <transform>6.6159962375703119e-01 -8.5504944701918273e-02 7.4496633633747211e-01 -2.1656220275301973e+01 -1.5662539868020420e-01 -9.8732166525167608e-01 2.5776225730780811e-02 -4.4206577200835371e+00 7.3331740899341891e-01 -1.3373419067754888e-01 -6.6660388831074180e-01 1.2121529258735372e+01 0 0 0 1</transform>
      </camera>
      <camera id="1" sensor_id="0" component_id="0" label="frame0001">
        <transform>6.8946094121357437e-01 -7.6902871731454520e-02 7.2022882395829457e-01 -2.0820557600907911e+01 -1.4003034579829973e-01 -9.8974074949144719e-01 2.8368134441959958e-02 -4.4201067110215693e+00 7.1065822502557530e-01 -1.2041261194560963e-01 -6.9315632442323927e-01 1.2306224779598960e+01 0 0 0 1</transform>
        <rotation_covariance>9.4240934718966737e-11 -3.3469162716744908e-11 7.2126630848199150e-11 -3.3469162716744914e-11 2.6820691786248343e-10 -2.2300135151498894e-10 7.2126630848199137e-11 -2.2300135151498889e-10 7.4342025435068256e-10</rotation_covariance>
        <location_covariance>4.3269853815255404e-06 -4.7583491511762746e-08 8.8558589241920762e-07 -4.7583491511762746e-08 3.0072993779693618e-08 2.7626120814449582e-08 8.8558589241920762e-07 2.7626120814449582e-08 2.6487925057985070e-07</location_covariance>
      </camera>

Any tips for converting .nvm files for use in nerfs?

Each scene in the Cambridge Landmarks dataset contains a file called reconstruction.nvm which is structured somewhat like this:
Could it be converted to transforms.json?

seq3/frame00158.jpg 1670.93139648 0.701128730309 0.671537135685 0.167975667142 -0.170998696429 81.6888803844 -43.725472823 1.54144922434 -0.0482305919283 0
seq4/frame00120.jpg 1670.82202148 0.662491959075 0.60159433005 0.285813173261 -0.342781995155 58.3095673469 -36.3420175582 1.25028735974 -0.0481262493079 0

171912
1.21410552758 -0.675720154582 2.23173935933 105 96 67 5 0 5781 -580.159790039 319.822509766 17 3438 -782.346923828 352.135559082 16 3980 -838.51940918 286.500854492 20 5082 -678.909545898 416.275085449 105 6408 -892.775390625 325.697937012
61.8203945802 2.80735688069 40.9514236521 210 215 221 12 308 493 302.760375977 -361.124328613 411 464 502.132080078 -290.715576172 325 542

The "out" argument isn't being respected

Thank you for making this! Looks like transforms.json gets dumped in the root dir regardless of the out argument.

"No calibration found" for spherical cameras

Hey,

Would it be possible to get this working with spherical / equirectangular camera positions exported from Metashape? Colmap doesn't support 360 images, so would be amazing if your script worked for this!

Here is a snippet from the xml, which is quite different to the regular "frame" type.

<?xml version="1.0" encoding="UTF-8"?> <document version="1.5.0"> <chunk label="Chunk 1" enabled="true"> <sensors next_id="1"> <sensor id="0" label="unknown" type="spherical"> <resolution width="5760" height="2880"/> <property name="layer_index" value="0"/> <bands> <band label="Red"/> <band label="Green"/> <band label="Blue"/> </bands> <data_type>uint8</data_type> <black_level>0 0 0</black_level> <sensitivity>1 1 1</sensitivity> </sensor> </sensors> <components next_id="1" active_id="0"> <component id="0" label="Component 1"> <region> <center>-8.9246900915267902e-01 2.0682838682109413e+01 2.5898418643679420e+01</center> <size>1.5354224936167401e+02 6.7941420046488446e+01 6.0285307184855142e+01</size> <R>-4.0138366329283741e-02 8.4499921858002310e-03 -9.9915840044533299e-01 -9.9821655039162638e-01 -4.4561741820354980e-02 3.9723666625547417e-02 -4.4188574005706568e-02 9.9897089487031399e-01 1.0223557568002074e-02</R> </region> <partition> <camera_ids>0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89</camera_ids> </partition> </component> </components> <cameras next_id="90" next_group_id="0"> <camera id="0" sensor_id="0" component_id="0" label="output_0001"> <transform>9.9959579221709571e-01 2.4701712245502279e-02 1.4074004192741969e-02 7.8205878137425722e-01 2.4443283329927897e-02 -9.9953467003790475e-01 1.8247446184902816e-02 -1.5206475018149446e-01 1.4518198301779174e-02 -1.7896055553066947e-02 -9.9973444129614353e-01 6.6758626639336242e+01 0 0 0 1</transform> </camera>

Swap axis

Hi,

Nice work. Why do you swap axis as this line https://github.com/EnricoAhlers/agi2nerf/blob/f10c758a710e691807578bea40bbccc24bbd43c2/agi2nerf.py#L152C6-L152C6?

Fisheye support

Hi,

It would be great if this tool worked with Metashape's fisheye camera type, now that Instant NGP supports OpenCV fisheye.

Nerfstudio's Metashape import was recently updated to support this, in case it helps!

https://github.com/nerfstudio-project/nerfstudio/blob/f015245ca4be13b1d3778e4d139c1f1590013757/nerfstudio/process_data/metashape_utils.py#L15

Currently using Colmap to align fisheye images, but it's painfully slow compared to Metashape!

Thanks!

totp /= totw [nan nan nan]

Hi, I'm sorry to tell you but the problem still exist. I tryed metashape 1.8
I used mask to remove background.
RuntimeWarning: invalid value encountered in divide
totp /= totw
[nan nan nan]
example.txt

transforms.txt

Bad calibration export when estimate K4 parameters in metashape or index not found if not estimate unused K3 parameter

Hi, when you export camera parameters you use an index but, if you calculate K4 for example in metashape, the index are not correct anymore. I propose this change in the code to be sure having no problem. it will also create a better error message when camera parameters that is needed is not present in xml file.

Replace at line 113:

		fl_x = float(calibration[1].text)
		fl_y = fl_x
		k1 = float(calibration[4].text)
		k2 = float(calibration[5].text)
		p1 = float(calibration[7].text)
		p2 = float(calibration[8].text)
		cx = float(calibration[2].text) + w/2
		cy = float(calibration[3].text) + h/2

By:

		for chld in calibration:
			if chld.tag == "f":
				fl_x = float(chld.text)
				fl_y = fl_x
			elif chld.tag == "cx":
				cx = float(chld.text) + w/2
			elif chld.tag == "cy":
				cy = float(chld.text) + h/2
			elif chld.tag == "k1":
				k1 = float(chld.text)
			elif chld.tag == "k2":
				k2 = float(chld.text)
			elif chld.tag == "p1":
				p1 = float(chld.text)
			elif chld.tag == "p2":
				p2 = float(chld.text)

Agisoft export with only k1 k2 estimated is like this:

        <calibration type="frame" class="adjusted">
          <resolution width="540" height="960"/>
          <f>768.68340771397811</f>
          <cx>-10.291821702893024</cx>
          <cy>3.7142687950142763</cy>
          <k1>0.054258424223576199</k1>
          <k2>-0.1179548245864302</k2>
          <p1>-0.0011294399122456702</p1>
          <p2>0.0022433983854041159</p2>
        </calibration>

Agisoft export with k3 and k4 estimated is like this:

        <calibration type="frame" class="adjusted">
          <resolution width="540" height="960"/>
          <f>768.68340771397811</f>
          <cx>-10.291821702893024</cx>
          <cy>3.7142687950142763</cy>
          <k1>0.054258424223576199</k1>
          <k2>-0.1179548245864302</k2>
          <k3>0.054258424223576199</k1>
          <k4>-0.1179548245864302</k2>
          <p1>-0.0011294399122456702</p1>
          <p2>0.0022433983854041159</p2>
        </calibration>

No calibration found

I cannot get the script to run because it cant find calibration in my XML from Metashape. Error is:

No calibration found
Traceback (most recent call last):
  File "instant-ngp\scripts\agi2nerf.py", line 112, in <module>
    fl_x = float(calibration[1].text)
TypeError: 'NoneType' object is not subscriptable

Here is my XML from Metashape:

<?xml version="1.0" encoding="UTF-8"?>
<document version="1.5.0">
  <chunk label="Chunk 1" enabled="true">
    <sensors next_id="2">
      <sensor id="1" label="Camera01f0" type="frame">
        <resolution width="1920" height="1080"/>
        <property name="pixel_width" value="0.01296458333"/>
        <property name="pixel_height" value="0.01296458333"/>
        <property name="focal_length" value="11.31841936"/>
        <property name="fixed" value="true"/>
        <property name="layer_index" value="0"/>
      </sensor>
    </sensors>
    <components next_id="1" active_id="0">
      <component id="0" label="Component 1">
        <region>
          <center>1.2481421084628337e+01 -1.8469269542611835e+00 1.8608684188197770e+01</center>
          <size>2.5819289989471434e+02 2.2276758861541748e+02 1.1039086513519288e+02</size>
          <R>-8.7265108278865244e-01 4.8080027400931119e-01 -8.5505463102446957e-02 -4.8012457320594654e-01 -8.7669884745308990e-01 -2.9656788063299924e-02 -8.9221532779928628e-02 1.5173245763366896e-02 9.9589622486552520e-01</R>
        </region>
        <partition/>
      </component>
    </components>
    <cameras next_id="197" next_group_id="0">
      <camera id="0" sensor_id="1" component_id="0" label="GX012479_000">
        <transform>-9.9118418179757506e-01 5.1225057342011542e-02 -1.2218801600230318e-01 -1.4860043204050000e+00 1.3180424224569562e-01 2.8744283979091040e-01 -9.4868554093491664e-01 1.1100930558570001e+02 -1.3474400925809109e-02 -9.5642700053545382e-01 -2.9166047583868115e-01 8.4215535887470008e+00 0 0 0 1</transform>
      </camera>
      <camera id="1" sensor_id="1" component_id="0" label="GX012479_001">
        <transform>-9.9640772611230455e-01 3.4467137049120716e-02 -7.7354119524064596e-02 -5.1429624606209998e+00 8.3998529396958024e-02 2.8613425338533011e-01 -9.5450062131921487e-01 1.0788322625570000e+02 -1.0765240492182452e-02 -9.5756942594427941e-01 -2.8800156960316847e-01 8.1889919950790002e+00 0 0 0 1</transform>
      </camera>
      <camera id="2" sensor_id="1" component_id="0" label="GX012479_002">
        <transform>-9.9927989764140190e-01 1.8124558638240425e-02 -3.3334464806846743e-02 -8.8111988987629992e+00 3.7116708383451082e-02 2.8456703551629686e-01 -9.5793734255234253e-01 1.0506086203709999e+02 -7.8763017062434357e-03 -9.5848479522193542e-01 -2.8503484208039653e-01 7.8865012121530000e+00 0 0 0 1</transform>
      </camera>
    </cameras>
    <reference>LOCAL_CS["Local Coordinates (m)",LOCAL_DATUM["Local Datum",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]]]</reference>
    <region>
      <center>1.2481421084628337e+01 -1.8469269542611835e+00 1.8608684188197770e+01</center>
      <size>2.5819289989471434e+02 2.2276758861541748e+02 1.1039086513519288e+02</size>
      <R>-8.7265108278865244e-01 4.8080027400931119e-01 -8.5505463102446957e-02 -4.8012457320594654e-01 -8.7669884745308990e-01 -2.9656788063299924e-02 -8.9221532779928628e-02 1.5173245763366896e-02 9.9589622486552520e-01</R>
    </region>
    <settings>
      <property name="accuracy_tiepoints" value="1"/>
      <property name="accuracy_cameras" value="10"/>
      <property name="accuracy_cameras_ypr" value="10"/>
      <property name="accuracy_markers" value="0.0050000000000000001"/>
      <property name="accuracy_scalebars" value="0.001"/>
      <property name="accuracy_projections" value="0.5"/>
    </settings>
  </chunk>
</document>

(I have deleted most of the cameras because it was too long to paste into this text box.)

about colmap data

I wonder if it can be trans into colmap data ？

enricoahlers / agi2nerf Goto Github PK

agi2nerf's People

Contributors

Stargazers

Watchers

Forkers

agi2nerf's Issues

RuntimeWarning: invalid value encountered in true_divide totp /= totw

Child index out of range

Any tips for converting .nvm files for use in nerfs?

The "out" argument isn't being respected

"No calibration found" for spherical cameras

Swap axis

Fisheye support

totp /= totw [nan nan nan]

Bad calibration export when estimate K4 parameters in metashape or index not found if not estimate unused K3 parameter

No calibration found

about colmap data

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent