This is a tutorial for tie-points extraction in diachronic images in MicMac.

Automatically generated by Colaboratory.

Adjusted from https://colab.research.google.com/drive/14okQ8bBhEZmy6EGRIQvazTqrN39oc_K5

• 0. About the dataset
• 1. Intra-epoch processing
  • 1.1. Feature matching
    • 1.1.1. Epoch 2014
    • 1.1.2. Epoch 1971
  • 1.2. Relative orientation
    • 1.2.1. Epoch 2014
    • 1.2.2. Epoch 1971
  • 1.3. DSM generation
    • 1.3.1. Epoch 2014
    • 1.3.2. Epoch 1971
• 2. Inter-epoch processing
  • 2.1. Automated pipeline
    • 2.1.1. Option 1: SuperGlue
    • 2.1.2. Option 2: SIFT
  • 2.2. Deep-dive in the pipeline's submodules
    • 2.2.1. Rough co-registration
      • 2.2.1.1. Preprocess DSM
      • 2.2.1.2. Match DSM
      • 2.2.1.3. Create virtual GCPs from DSMs
      • 2.2.1.4. 3D Helmert transformation
    • 2.2.2 Precise matching
      • 2.2.2.1 Get overlapped image pairs
      • 2.2.2.2 Get patch pairs
      • 2.2.2.3 Get tentative tie-points
      • 2.2.2.4 Get enhanced tie-points
      • 2.2.2.5 Get final tie-points
• 3. Evaluation
  • 3.1. DoD of roughly co-registered result
  • 3.2. DoD of SuperGlue refined result
  • 3.3. DoD of GuidedSIFT refined result

In this tutorial we will introduce you to tie-points extraction in diachronic images (both satellite and aerial images) in MicMac. (For processing aerial images only, please refer to Historical_Aerial_TiePtExtraction_pipeline.ipynb) The goal of the exercise is to extract tie-points between diachronic images. We refer to images obtained within one epoch as intra-epoch images, and images obtained at different epochs as inter-epoch images.

Given a set of images in 2 epochs, our pipeline performs an intra-epoch processing, followed by an inter-epoch processing. The latter consists of 2 main steps: rough co-registration and precise matching.

After setting-up MicMac and downloading the dataset, the pipeline is as follows:

1. Intra-epoch processing

   1.1. Feature matching: Applying feature matching based on SIFT on images within the same epoch.

   1.2. Relative orientation: Computing relative orientations for each epoch.

   1.3. DSM generation: Computing DSM of each epoch based on relative orientations.

   If the orientation of each epoch is provided, the step 1.2 can be skipped.

2. Inter-epoch processing

   2.1. Automated pipeline

   We provide automated pipeline which will launch the whole inter-epoch processing pipeline by calling several subcommands.

   2.2. Deep-dive in the pipeline's submodules

   We also provide deep-dive to explain all the submodules used in the automated pipeline. It consists of:

   • 2.2.1. Rough co-registration

   This step is for roughly co-registering the DSMs and image orientations from different epochs.

   • 2.2.2. Precise matching

   This step is for obtaining precise tie-points under the guidance of rough co-registration.

If the 2 epochs are already co-registered (e.g. both epochs are georeferenced), the step 2.2.1 can be skipped.

The full workflow is displayed below:

For more information please refer to:

• Publication: L. Zhang, E. Rupnik, M. Pierrot-Deseilligny, Feature matching for multi-epoch historical aerial images, ISPRS journal of photogrammetry and remote sensing, 182 (2021), pp. 176-189 (PDF)
• Code: MicMac, GitHub, https://github.com/micmacIGN/micmac/tree/master/src/uti_phgrm/TiePHistorical, 2021.
• Introduction video: Feature matching for multi-epoch historical aerial images
• Slides: Feature matching for multi-epoch historical aerial images
• Blog: Feature matching for multi-epoch historical aerial images
• Project website: Feature matching for multi-epoch historical aerial images

The notion of the "tie-point" and the "correspondence" are synonymous in this tutorial and the publication.

The datasets in our publication will be soon published in an open-access repository.

Contacts:

[email protected]

[email protected]

License: We adopt SuperGlue in our pipeline. Note that you can only use SuperGlue for academic or non-profit organization noncommercial research. Full paper PDF: SuperGlue: Learning Feature Matching with Graph Neural Networks.

The dataset is a 20 km2 rectangular area located in Pezenas in the Occitanie region in southern France. It is mainly covered with vegetation and several sparsely populated urban zones. We have at our disposal 2 epochs: epoch 1971 of aerial images, and epoch 2014 of satellite images.

Details of images in epoch 1971:

Details of images in epoch 2014:

Dataset can be downloaded here: https://drive.google.com/u/0/uc?id=1lbbBiHFHyVDGXlLMtl3iHMWW-Mdj5Vjy&export=download

The dataset consists of:

• 6 images taken in epoch 1971 (tif)
• 2 images taken in epoch 2014 (tif)
• 1 folder of orientations of epoch 2014
• 1 configuration file of the camera (xml)

First, we will process each epoch individually to obtain:

• intra-epoch tie-points
• relative orientations
• DSMs

The images in epoch 2014 are satellite images and their orientations are known. Consequently, the feature matching within this epoch can be skipped. This epoch is considered as ground truth in the processing.

Extract SIFT tie-points between images within the same epoch with command "Tapioca" (i.e., a version of SIFT). The input, output and parameter interpretation of the command "Tapioca" are listed below:

Input:

• images

Output:

• SIFT tie-points

The meaning of obligatory parameters:

• MulScale: Compute tie-points for images in low resolution and then for high resolution
• OIS-Reech_IGNF_PVA_1-0__1971.*tif: the pattern of input images
• 500: Size of Low Resolution Images
• 1700: Size of High Resolution Images

The meaning of optional parameters:

• PostFix: Homologue extenion for NB/NT mode

mm3d Tapioca MulScale OIS-Reech_IGNF_PVA_1-0__1971.*tif 500 1700 PostFix=_1971

You'll get tie points like this:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1068.tif KH=NB SH=_1971 SzW=[600,600]

Since historical images contain fiducial marks and they would yield nonsensical tie-points, we create a mask to remove any points in the vicinity of the fiducial marks. To create the mask we use the "SaisieMasq" program:

The input, output and parameter interpretation of the command "SaisieMasq" are listed below:

Input:

• image

Output:

• mask image drawn by user

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1068.tif: input image

The meaning of optional parameters:

• Name: the output mask image

mm3d SaisieMasq OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1068.tif Name=Fiducial.tif

Once the mask created, we will use it to remove all tie-points outside the mask with "HomolFilterMasq".

The input, output and parameter interpretation of the command "HomolFilterMasq" are listed below:

Input:

• images; tie-points; mask file

Output:

• filtered tie-points

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971.*tif: the pattern of input images

The meaning of optional parameters:

• GlobalMasq: Global Masq to add to all image

• PostIn: Post for Input dir Hom, Def=

• PostOut: Post for Output dir Hom, Def=MasqFiltered

  mm3d HomolFilterMasq OIS-Reech_IGNF_PVA_1-0__1971.*tif GlobalMasq=Fiducial.tif PostIn=_1971 PostOut=_1971-Masq
  

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1068.tif KH=NB SH=_1971-Masq SzW=[600,600]

The intra-epoch tie-points are generally very abundant. We reduce their quantity for several reasons:

• quantity of tie points does not impact the quality of the bundle adjustement (only the distribution is crucial)
• we wish to speed-up the bundle adjustement
• we want to "balance" the number of inter- and intra-epoch tie-points

To ensure good distribution and high multiplicity of the "reduced" tie-points, MicMac first computes the relative orientation between pairs of images using TestLib NO_AllOri2Im, followed by tie-points reduction TestLib Ratafia.

mm3d TestLib NO_AllOri2Im OIS-Reech_IGNF_PVA_1-0__1971.*tif SH=_1971-Masq

mm3d Ratafia OIS-Reech_IGNF_PVA_1-0__1971.*tif SH=_1971-Masq Out=_1971-Ratafia

You'll get tie points like this:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1068.tif KH=NB SH=_1971-Ratafia SzW=[600,600]

Recover relative orientations of images within the same epoch using structure-from-motion implemented in MicMac under the command "Tapas".

The input, output and parameter interpretation of the command "Tapas" are listed below:

Input:

• images; tie-points

Output:

• relative orientations

The meaning of obligatory parameters:

• FraserBasic: use the Fraser model to calibrate the cameras
• OIS-Reech_IGNF_PVA_1-0__1971.*tif: the pattern of input images

The meaning of optional parameters:

• Out: Directory of Output Orientation
• SH: Set of Hom, Def="", give MasqFiltered for result of HomolFilterMasq

The images in epoch 2014 are satellite images and their orientations are known. The computation of relative orientations is therefore skipped.

Recover relative orientations of images in epoch 1971:

mm3d Tapas FraserBasic OIS-Reech_IGNF_PVA_1-0__1971.*tif Out=1971 SH=_1971-Ratafia

Compute DSM of each epoch based on orientations using the command "Malt".

The input, output and parameter interpretation of the command "Malt" are listed below:

Input:

• images; orientations

Output:

• DSM

The meaning of obligatory parameters:

• Ortho: for a matching adapted to ortho photo generation.
• Crop.*tif: the pattern of input images
• 2014: orientation of images

The meaning of optional parameters:

• NbVI: Number of Visible Images required (Def = 3)
• DirMEC: Subdirectory where the results will be stored
• EZA: Export Z Absolute
• ZoomF: Final zoom, (Def 2 in ortho,1 in MNE)
• DoOrtho: generate the orthophoto at the same time
• MasqImGlob: Glob Masq per Im : if uses, give full name of masq (for ex toto.tif)

Generate DSM in epoch 2014:

mm3d Malt Ortho Crop.*tif 2014 NbVI=2 DirMEC=MEC-Malt_2014 EZA=1 ZoomF=8 DoOrtho=0

You'll get DSM like this:

You can also visualize the DSM in MicMac using the command 'Vino':

Vino MEC-Malt_2014/Z_Num6_DeZoom8_STD-MALT.tif

Generate DSM in epoch 1971:

mm3d Malt Ortho OIS-Reech_IGNF_PVA_1-0__1971.*tif 1971 NbVI=2 MasqImGlob=Fiducial.tif DirMEC=MEC-Malt_1971 EZA=1 ZoomF=8 DoOrtho=0

You'll get DSM like this:

You can also visualize the DSM in MicMac using the command 'Vino':

Vino MEC-Malt_1971/Z_Num6_DeZoom8_STD-MALT.tif

We provide both (1) automated pipeline which will automatically launch the whole inter-epoch processing pipeline, and (2) deep-dive to explain all the submodules used in the automated pipeline.

Use the TiePHistoP program to launch the automated inter-epoch processing pipeline.

The input, output and parameter interpretation of the command "TiePHistoP" are listed below:

Input:

• 2 sets of image lists, images, orientations and DSMs in 2 epochs

Output:

• roughly co-registered orientations; precise inter-epoch tie-points

The meaning of obligatory parameters:

• Ori-1971: orientation of images in epoch 1971
• Ori-2014: orientation of images in epoch 2014
• OIS-Reech_IGNF_PVA_1-0__1971.*tif: all RGB images in epoch 1971 (Dir+Pattern, or txt file of image list), referred to as ImgList1
• Crop.*tif: all RGB images in epoch 2014 (Dir+Pattern, or txt file of image list), referred to as ImgList2
• MEC-Malt_1971: DSM direcotry of epoch 1971
• MEC-Malt_2014: DSM direcotry of epoch 2014

The meaning of optional parameters (here we use epoch1 to refer to epoch 1971, and epoch2 to epoch 2014):

• Feature: Feature matching method used for precise matching (SuperGlue or SIFT. Generally the former has better performance while the latter is more efficient in terms of the use of memory and CPU resources when dealing with big dataset), Def=SuperGlue
• SkipCoReg: Skip the step of rough co-registration, when the input orientations of epoch1 and epoch2 are already co-registrated, Def=false
• CoRegPatchLSz: Patch size of the tiling scheme for master image in rough co-registration part, which means the master images to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]. Normally CoRegPatchLSz of [640, 480] is recommanded, here we use [1280, 960] to speed up the process
• CoRegPatchRSz: Patch size of the tiling scheme for secondary image in rough co-registration part, which means the secondary images to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]. Normally CoRegPatchRSz of [640, 480] is recommanded, here we use [1280, 960] to speed up the process
• CoRegBufferLSz: Buffer zone size around the patch of the tiling scheme for master image in rough co-registration part, Def=[0, 0]
• CoRegBufferRSz: Buffer zone size around the patch of the tiling scheme for secondary image in rough co-registration part, Def=[0, 0]
• PrecisePatchSz: Patch size of the tiling scheme in precise matching part, which means the images to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]. Normally PrecisePatchSz of [640, 480] is recommanded, here we use [1280, 960] to speed up the process
• PreciseBufferSz: Buffer zone size around the patch of the tiling scheme in precise matching part, Def=10%*PrecisePatchSz
• CheckNbCoReg: Radius of the search space for SuperGlue in rough co-registration step (which means correspondence [(xL, yL), (xR, yR)] with (xL-xR)×(xL-xR)+(yL-yR)×(yL-yR) > CheckNb×CheckNb will be removed afterwards), Def=-1 (means don't check search space)
• CheckNbPrecise: Radius of the search space for SuperGlue in precise matching step (which means correspondence [(xL, yL), (xR, yR)] with (xL-xR)×(xL-xR)+(yL-yR)×(yL-yR) > CheckNb×CheckNb will be removed afterwards), Def=100
• MinPt: Minimun number of input correspondences required, Def=10
• IL1: RGB images in epoch 1971 for extracting inter-epoch correspondences (Dir+Pattern, or txt file of image list), Def=ImgList1
• IL2: RGB images in epoch 2014 for extracting inter-epoch correspondences (Dir+Pattern, or txt file of image list), Def=ImgList2
• Exe: Execute all, Def=true. If this parameter is set to false, the pipeline will not be executed and the command of all the submodules will be printed.
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false
• UseDep: GetPatchPair for depth maps as well (this option is only used for developper), Def=false
• RotateDSM: The angle of rotation from the master DSM to the secondary DSM for rough co-registration (only 4 options available: 0, 90, 180, 270), Def=-1 (means all the 4 options will be executed, and the one with the biggest inliers will be kept)
• SkipPrecise: Skip the step of the whole precise matching pipeline, Def=false
• SkipGetPatchPair: Skip the step of "GetPatchPair" in precise matching (this option is used when the results of "GetPatchPair" already exist), Def=false
• SkipTentativeMatch: Skip the step of "SuperGlue" or SIFT matching (this option is used when the results of "SuperGlue" or SIFT matching already exist), Def=false
• SkipRANSAC3D: Skip the step of "3D RANSAC" (this option is used when the results of "3D RANSAC" already exist), Def=false
• SkipCrossCorr: Skip the step of "cross correlation" (this option is used when the results of "cross correlation" already exist), Def=false
• DSMFileL: DSM File of epoch1, Def=MMLastNuage.xml
• DSMFileR: DSM File of epoch2, Def=MMLastNuage.xml
• STDRange: Only pixels with their value within STDRange times of std will be considered (in order to ignore altitude outliers), Def=5
• SubPXml: The output xml file name to record the homography between the patches and original image, Def=SubPatch.xml
• ImgPair: Output txt file that records the patch pairs, Def=SuperGlueInput.txt
• Dyn: The Dyn parameter in "to8Bits" if the input RGB images are 16 bits, Def=0.1
• InDir: The input directory of the images for SuperGlue, Def=./
• SpGOutSH: Homologue extenion for NB/NT mode of SuperGlue, Def=-SuperGlue
• Resize: The goal size for resizing the input image for SuperGlue, Def=[640, 480], if you don't want to resize, please set to [-1, -1]
• Viz: Visualize the matches and dump the plots of SuperGlue, Def=false
• Model: Pretrained indoor or outdoor model of SuperGlue, Def=outdoor
• MaxPt: Maximum number of keypoints detected by Superpoint, Def=1024
• KeepNpzFile: Keep the original npz file that SuperGlue outputed, Def=false
• EntSpG: The SuperGlue program entry (this option is only used for developper), Def=../micmac/src/uti_phgrm/TiePHistorical/SuperGluePretrainedNetwork-master/match_pairs.py
• opt: Other options for SuperGlue (this option is only used for developper), Def=none
• MergeInSH: Input Homologue extenion for NB/NT mode for MergeTiePt, Def=none
• HomoXml: Input xml file that recorded the homograhpy from patches to original image for MergeTiePt, Def=SubPatch.xml
• MergeOutSH: Output Homologue extenion for NB/NT mode of MergeTiePt, Def=-SubPatch
• 2DRANInSH: Input Homologue extenion for NB/NT mode for 2D RANSAC, Def=none
• 2DRANOutSH: Output Homologue extenion for NB/NT mode of 2D RANSAC, Def='2DRANInSH'-2DRANSAC
• 2DIter: 2D RANSAC iteration, Def=1000
• 2DRANTh: 2D RANSAC threshold, Def=10
• CreateGCPsInSH: Input Homologue extenion for NB/NT mode for CreateGCPs, Def=none
• Out2DXml1: Output xml files of 2D obersevations of the GCPs in epoch1, Def=OutGCP2D_epoch1.xml
• Out3DXml1: Output xml files of 3D obersevations of the GCPs in epoch1, Def=OutGCP3D_epoch1.xml
• Out2DXml2: Output xml files of 2D obersevations of the GCPs in epoch2, Def=OutGCP2D_epoch2.xml
• Out3DXml2: Output xml files of 3D obersevations of the GCPs in epoch2, Def=OutGCP3D_epoch2.xml
• OutPairXml: Output Xml file to record the overlapped inter-epoch image pairs, Def=OverlappedImages.xml
• GSIFTOutSH: Output Homologue extenion for NB/NT mode of Guided SIFT, Def=-GuidedSIFT
• SkipSIFT: Skip extracting SIFT key points in case it is already done, Def=false
• SearchSpace: Radius of the search space for GuidedSIFT (the search space is the circle with the center on the predicted point), Def=100 (this value is based on master image, the search space on secondary image will multiply the scale difference if "CheckScale" is set to true)
• MutualNN: Apply mutual nearest neighbor on GuidedSIFT, Def=true
• RatioT: Apply ratio test on GuidedSIFT, Def=true
• RootSift: Use RootSIFT as descriptor on GuidedSIFT, Def=true
• CheckScale: Check the scale of the candidate tie-points on GuidedSIFT, Def=true
• CheckAngle: Check the angle of the candidate tie-points on GuidedSIFT, Def=true
• ScaleTh: The threshold for checking scale ratio, Def=0.2; (0.2 means the ratio of master and secondary SIFT scale between [(1-0.2)×Ref, (1+0.2)×Ref] is considered valide. Ref is automatically calculated by reprojection.)
• AngleTh: The threshold for checking angle difference, Def=30; (30 means the difference of master and secondary SIFT angle between [Ref - 30 degree, Ref + 30 degree] is considered valide. Ref is automatically calculated by reprojection.)
• Predict: Use the predicted key points to guide the matching in GuidedSIFT, Def=true
• ScaleL: Extract SIFT points on master images downsampled with a factor of "ScaleL", Def=1
• ScaleR: Extract SIFT points on secondary images downsampled with a factor of "ScaleR", Def=1
• 3DRANInSH: Input Homologue extenion for NB/NT mode for 3D RANSAC, Def=none
• 3DRANOutSH: Output Homologue extenion for NB/NT mode of 3D RANSAC, Def='3DRANInSH'-3DRANSAC
• 3DIter: 3D RANSAC iteration, Def=1000
• 3DRANTh: 3D RANSAC threshold, Def=10*(GSD of secondary image)
• CCInSH: Input Homologue extenion for NB/NT mode for cross correlation, Def=none
• CCOutSH: Output Homologue extenion for NB/NT mode of cross correlation, Def='CCInSH'-CrossCorrelation
• SzW: Window size of cross correlation, Def=32
• CCTh: Corss correlation threshold, Def=0.5

Note: As we lack of redundant obeservations due to limited number of images, we set 3DRANTh=10 and CCTh=0.5 to keep more tie-points.

mm3d TiePHistoP Ori-1971 Ori-2014 OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif MEC-Malt_1971 MEC-Malt_2014 CoRegPatchLSz=[1280,960] CoRegPatchRSz=[1280,960] PrecisePatchSz=[1280,960] Feature=SuperGlue Viz=1 3DRANTh=10 CCTh=0.5 SkipCoReg=0 SkipPrecise=0 Exe=1

This command will produce 2 kinds of results:

(1) roughly co-registered orientations, which will be stored in the folder "./Ori-2014";

(2) precise inter-epoch tie-points, which will be stored in the folder "./Homol-SuperGlue-3DRANSAC-CrossCorrelation".

You'll get tie points like this:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-SuperGlue-3DRANSAC-CrossCorrelation

The tie points displayed below is just from one image pair. You can also visualize the inter-epoch tie-points on all the image pairs with the command "TestLib VisuTiePtIn3D", which allows you to check the distribution of the tie-points across the entire image block by visualising them in a common reference frame.

The input, output and parameter interpretation of the command "TestLib VisuTiePtIn3D" are listed below:

Input:

• 2 sets of image lists, orientations and DSMs in 2 epochs; inter-epoch tie-points

Output:

• 2 txt files to record the tie-points in master and secondary images respectively (in 3D coordinate)

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971.*tif: All master images (Dir+Pattern, or txt file of image list), referred to as ImgList1
• Crop.*tif: All secondary images (Dir+Pattern, or txt file of image list), referred to as ImgList2
• Ori-1971: Orientation of images in epoch 1971
• Ori-2014: Orientation of images in epoch 2014

The meaning of optional parameters:

• Print: Print supplementary information, Def=false
• Dir: Work directory, Def=./
• InSH: Input Homologue extenion for NB/NT mode, Def=none
• OutFile: Output file name of 3D points, Def=VisuTiePtIn3D.txt
• DSMDirL: DSM of master image (for improving the reprojecting accuracy), Def=none
• DSMDirR: DSM of secondary image (for improving the reprojecting accuracy), Def=none
• DSMFileL: DSM File of master image, Def=MMLastNuage.xml
• DSMFileR: DSM File of secondary image, Def=MMLastNuage.xml
• Thres: The threshold of reprojection error (unit: pixel) when prejecting patch corner to DSM, Def=2
• Para3DHL: Input xml file that recorded the paremeter of the 3D Helmert transformation for points in master images, Def=none
• Para3DHR: Input xml file that recorded the paremeter of the 3D Helmert transformation for points in secondary images, Def=none

mm3d TestLib VisuTiePtIn3D OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif Ori-1971 Ori-2014 DSMDirL=MEC-Malt_1971 DSMDirR=MEC-Malt_2014 InSH=-SuperGlue-3DRANSAC-CrossCorrelation OutFile=Visu-SuperGlue-3DRANSAC-CrossCorrelation-all.txt

The transformed points can be visulized in QGIS. You'll see tie points like this:

Note: (1) We set Feature=SIFT to switch to option SIFT; (2) the rough co-registration has been performed in the previous step, so we can skip it here; (3) the PrecisePatchSz is the same with the previous step, therefore we can skip the step of getting patch pairs; (4) as the resolution of secondary images is high, we speed up the processing by setting ScaleR=2 so that SIFT points extracted on secondary images are based on downsamepled images by a factor of 2.

mm3d TiePHistoP Ori-1971 Ori-2014 OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif MEC-Malt_1971 MEC-Malt_2014 PrecisePatchSz=[1280,960] Feature=SIFT SkipCoReg=1 SkipGetPatchPair=1 ScaleR=2 CoRegOri1=1971_CoReg_SuperGlue 3DRANTh=10 CCTh=0.5 SkipPrecise=0 Exe=1

You'll get tie points like this:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-GuidedSIFT-3DRANSAC-CrossCorrelation

We again use the "TestLib VisuTiePtIn3D" to check the tie-points distribution across the scene.

mm3d TestLib VisuTiePtIn3D OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif Ori-1971 Ori-2014 DSMDirL=MEC-Malt_1971 DSMDirR=MEC-Malt_2014 InSH=-GuidedSIFT-3DRANSAC-CrossCorrelation OutFile=Visu-GuidedSIFT-3DRANSAC-CrossCorrelation-all.txt

You'll see tie points like this:

The TiePHistoP command automatically calls several subcommands including:

(1) For rough co-registration

• TestLib DSM_Equalization
• TestLib Wallis
• TestLib GetPatchPair
• TestLib SuperGlue
• TestLib MergeTiePt
• TestLib RANSAC R2D
• TestLib CreateGCPs
• TestLib GCPBascule

(2) For precise matching

• TestLib GetOverlappedImages
• TestLib GetPatchPair
• TestLib SuperGlue
• TestLib MergeTiePt
• TestLib GuidedSIFTMatch
• TestLib RANSAC R3D
• TestLib CrossCorrelation

In order to help the users understand the pipeline better, we will introduce the subcommands in section 2.2.1 and 2.2.2.

To co-register images in different epochs in a common coordinate frame, we perform sparse point matching between their respective DSM rasters. The workflow is displayed as below:

(1) DSM Equalization

(1.1) Epoch 2014

Transform the DSM in float point type to equalized gray scale image with the command "TestLib DSM_Equalization", so that feature matching method can be applied later.

The input, output and parameter interpretation of the command "TestLib DSM_Equalization" are listed below:

Input:

• DSM

Output:

• equlized DSM gray image

The meaning of obligatory parameters:

• MEC-Malt_2014: DSM direcotry of epoch 2014

The meaning of optional parameters:

• STDRange: Only pixels with their value within STDRange times of std will be considered (in order to ignore altitude outliers), Def=5
• DSMFile: the xml file that recorded the structure information of the DSM, Def=MMLastNuage.xml
• OutImg: output image name, Def='input'_gray.tif

mm3d TestLib DSM_Equalization MEC-Malt_2014 DSMFile=MMLastNuage.xml OutImg=DSM2014-gray.tif

The equalized DSM is visulized below:

Vino MEC-Malt_2014/DSM2014-gray.tif

(1.2) Epoch 1971

Perform DSM equalization in epoch 1971, so that feature matching method can be applied later.

mm3d TestLib DSM_Equalization MEC-Malt_1971 DSMFile=MMLastNuage.xml OutImg=DSM1971-gray.tif

The equalized DSM is visulized below:

Vino MEC-Malt_1971/DSM1971-gray.tif

(2)DSM Wallis filter

(2.1) Epoch 2014

Apply wallis filter on the DSM gray image with the command "TestLib Wallis" to get rid of uneven illumination.

The input, output and parameter interpretation of the command "TestLib Wallis" are listed below:

Input:

• DSM gray image

Output:

• illumination equalized DSM gray image

The meaning of obligatory parameters:

• DSM2014-gray.tif: the input image

The meaning of optional parameters:

• Dir: work directory, Def=./
• OutImg: Output image name, Def='input'_sfs.tif

Apply wallis filter on DSM in epoch 2014:

mm3d TestLib Wallis DSM2014-gray.tif Dir=MEC-Malt_2014 OutImg=DSM2014-gray.tif_sfs.tif

You'll get DSM like this:

You can also visualize the DSM in MicMac:

Vino MEC-Malt_2014/DSM2014-gray.tif_sfs.tif

(2.2) Epoch 1971

Apply wallis filter on DSM in epoch 1971 to get rid of uneven illumination.

mm3d TestLib Wallis DSM1971-gray.tif Dir=MEC-Malt_1971 OutImg=DSM1971-gray.tif_sfs.tif

You'll get DSM like this:

You can also visualize the DSM in MicMac:

Vino MEC-Malt_1971/DSM1971-gray.tif_sfs.tif

This part will co-register 2 DSMs from different epochs by applying one-to-many tiling scheme (for more details please refer to our publication: link to be added) based on SuperGlue, followed by RANSAC to remove outliers.

The one-to-many tiling scheme will be performed 4 times by rotating the master DSM 90 degrees each time in order to test 4 rotation hypotheses (because the rough co-registration method is only invariant to rotation smaller than 45 degree), followed by keeping the one with the largest RANSAC inliers.

It includes 4 parts:

(1) Split DSM image pair into patch pairs with command "TestLib GetPatchPair";

(2) Match patch pairs with command "TestLib SuperGlue";

(3) Merge tie-points with command "TestLib MergeTiePt";

(4) Run 2D RANSAC with command "TestLib RANSAC R2D".

Note: step (2-4) are repeated 4 times to test rotation hypotheses.

Get patch pairs (One-to-many tiling)

Split DSM image pair into patch pairs with command "TestLib GetPatchPair"

The input, output and parameter interpretation of the command "TestLib GetPatchPair" are listed below:

Input:

• 2 sets of DSM gray images in 2 epochs

Output:

• patches of DSM gray images, patch pair list

The meaning of obligatory parameters:

• BruteForce: one of the 2 options (BruteForce and Guided) in this command. BruteForce means that we split master and secondary DSM images into M and N patches individually and consider matching M*N patch pairs by brute force.
• MEC-Malt_1971/DSM1971-gray.tif_sfs.tif: master DSM image
• MEC-Malt_2014/DSM2014-gray.tif_sfs.tif: secondary DSM image

The meaning of optional parameters:

• Rotate: Rotate the master image by 90 degree 4 times for matching methods which are not invariant to rotation (e.g. SuperGlue), Def=false
• Dir: Work directory, Def=./
• PatchLSz: Patch size of the tiling scheme for master image, which means the master image to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]. Normally PatchSz of [640, 480] is recommanded, here we use [1280, 960] to speed up the process.
• BufferLSz: Buffer zone size around the patch of the tiling scheme for master image, Def=[0,0]
• PatchRSz: Patch size of the tiling scheme for secondary image, which means the secondary image to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]. Normally PatchSz of [640, 480] is recommanded, here we use [1280, 960] to speed up the process.
• BufferRSz: Buffer zone size around the patch of the tiling scheme for secondary image, Def=[0,0]
• SubPXml: The output xml file name to record the homography between the patches and original image, Def=SubPatch.xml
• OutDir: Output direcotry of the patches, Def=./Tmp_Patches-CoReg
• ImgPair: Output txt file that records the patch pairs, Def=SuperGlueInput.txt

mm3d TestLib GetPatchPair BruteForce MEC-Malt_1971/DSM1971-gray.tif_sfs.tif MEC-Malt_2014/DSM2014-gray.tif_sfs.tif  OutDir=./Tmp_Patches-CoReg Rotate=1 PatchLSz=[1280,960] PatchRSz=[1280,960]

Besides splitting DSM image pair into patch pair, this command will also automatically (1) generate the SuperGlueInput.txt and HomoXml files which will serve as input in the following parts, (2) copy the master and secondary DSM images into the work directory and rename them to "MEC-Malt_1971.tif" and "MEC-Malt_2014.tif".

If the Rotate parameter is set to 1, this command will generate 4 sets of SuperGlueInput.txt and HomoXml files by rotating master DSM image 4 times.

An example of the patch pairs is visulized in the following:

You can also visualize the patch pairs in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971_0_0.tif MEC-Malt_2014_0_0.tif KH=S

Rotation hypothesis 1 (0 degree)

(1) Match patch pairs with command "TestLib SuperGlue":

This command will call SuperGlue to match the patch pairs which are listed in "SuperGlueInput.txt".

The input, output and parameter interpretation of the command "TestLib SuperGlue" are listed below:

Input:

• images; list of image pairs

Output:

• SuperGlue tie-points

The meaning of obligatory parameters:

• SuperGlueInput.txt: list of patch pairs to be matched (which is output from previous command "TestLib GetPatchPair")

The meaning of optional parameters:

• InDir: The input directory of the images for SuperGlue, Def=./
• SpGOutSH: Homologue extenion for NB/NT mode of SuperGlue, Def=-SuperGlue
• OutDir: The output directory of the match results of SuperGlue, Def=./
• Resize: The goal size for resizing the input image for SuperGlue, Def=[640, 480], if you don't want to resize, please set to [-1, -1]
• Viz: Visualize the matches and dump the plots of SuperGlue, Def=false
• Model: Pretrained indoor or outdoor model of SuperGlue, Def=outdoor
• MaxPt: Maximum number of keypoints detected by Superpoint, Def=1024
• KeepNpzFile: Keep the original npz file that SuperGlue outputed, Def=false
• EntSpG: The SuperGlue program entry (this option is only used for developper), Def=../micmac/src/uti_phgrm/TiePHistorical/SuperGluePretrainedNetwork-master/match_pairs.py
• opt: Other options for SuperGlue (this option is only used for developper), Def=none
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false
• CheckNb: Radius of the search space for SuperGlue (which means correspondence [(xL, yL), (xR, yR)] with (xL-xR)×(xL-xR)+(yL-yR)×(yL-yR) > CheckNb×CheckNb will be removed afterwards), Def=-1 (means don't check search space)

mm3d TestLib SuperGlue SuperGlueInput.txt  InDir=./Tmp_Patches-CoReg/ OutDir=./Tmp_Patches-CoReg/ SpGOutSH=-SuperGlue Viz=1

An example of the matched patch pairs is visulized below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971_0_0.tif MEC-Malt_2014_0_0.tif KH=NT SzW=[600,600] SH=-SuperGlue

(2) Merge tie-points with command "TestLib MergeTiePt"

The tie-points are saved as individual files for each patch pair, this command will merge them together.

The input, output and parameter interpretation of the command "TestLib MergeTiePt" are listed below:

Input:

• images; tie-points in format of patch pairs; xml file that recorded the homograhpy from patches to original image

Output:

• tie-points in format of image pair

The meaning of obligatory parameters:

• ./Tmp_Patches-CoReg/: work directory

The meaning of optional parameters:

• OutDir: Output directory of the merged tie-points, Def=Work directory
• MergeInSH: Input Homologue extenion for NB/NT mode for MergeTiePt, Def=none
• HomoXml: Input xml file that recorded the homograhpy from patches to original image for MergeTiePt, Def=SubPatch.xml
• MergeOutSH: Output Homologue extenion for NB/NT mode of MergeTiePt, Def=-SubPatch
• PatchSz: Patch size of the tiling scheme (since we use the patches resulted from "GetPatchPair", this parameter should be set the same as the PatchSz in command GetPatchPair), Def=[640, 480]
• BufferSz: Buffer zone size around the patch of the tiling scheme (since we use the patches resulted from "GetPatchPair", this parameter should be set the same as the BufferSz in command GetPatchPair), Def=[0,0]

mm3d TestLib MergeTiePt ./Tmp_Patches-CoReg/  HomoXml=SubPatch.xml MergeInSH=-SuperGlue MergeOutSH=-SubPatch PatchSz=[1280,960]

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971.tif MEC-Malt_2014.tif KH=NT SzW=[600,600] SH=-SubPatch

You can see that the tie-points are not good, which indicates the current rotation hypothesis is not the good one.

(3) 2D RANSAC with command "TestLib RANSAC R2D"

This command will apply 2D RANSAC on the tie-points to reject outliers.

The input, output and parameter interpretation of the command "TestLib RANSAC R2D" are listed below:

Input:

• images; tie-points

Output:

• filtered tie-points

The meaning of obligatory parameters:

• R2D: one of the 2 options (R2D and R3D) that are available in this command. R2D means we apply RANSAC directly on tie-points.
• MEC-Malt_1971.tif: master DSM image
• MEC-Malt_2014.tif: secondary DSM image

The meaning of optional parameters:

• Dir: Work directory, Def=./
• 2DRANInSH: Input Homologue extenion for NB/NT mode for 2D RANSAC, Def=none
• 2DRANOutSH: Output Homologue extenion for NB/NT mode of 2D RANSAC, Def='2DRANInSH'-2DRANSAC
• 2DIter: 2D RANSAC iteration, Def=1000
• 2DRANTh: 2D RANSAC threshold, Def=10

mm3d TestLib RANSAC R2D MEC-Malt_1971.tif MEC-Malt_2014.tif Dir=./Tmp_Patches-CoReg/  2DRANInSH=-SubPatch 2DRANOutSH=-SubPatch-2DRANSAC

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971.tif MEC-Malt_2014.tif KH=NT SzW=[600,600] SH=-SubPatch-2DRANSAC

You can see that the tie-points are not good, which indicates the current rotation hypothesis is not the good one.

Rotation hypothesis 2 (90 degree)

mm3d TestLib SuperGlue SuperGlueInput_R90.txt  InDir=./Tmp_Patches-CoReg/ OutDir=./Tmp_Patches-CoReg/ SpGOutSH=-SuperGlue Viz=1  
mm3d TestLib MergeTiePt ./Tmp_Patches-CoReg/  HomoXml=SubPatch_R90.xml MergeInSH=-SuperGlue MergeOutSH=-SubPatch_R90 PatchSz=[1280,960]   
mm3d TestLib RANSAC R2D MEC-Malt_1971.tif MEC-Malt_2014.tif Dir=./Tmp_Patches-CoReg/  2DRANInSH=-SubPatch_R90 2DRANOutSH=-SubPatch_R90-2DRANSAC

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971.tif MEC-Malt_2014.tif KH=NT SzW=[600,600] SH=-SubPatch_R90-2DRANSAC

You can see that the tie-points are not good, which indicates the current rotation hypothesis is not the good one.

Rotation hypothesis 3 (180 degree)

mm3d TestLib SuperGlue SuperGlueInput_R180.txt  InDir=./Tmp_Patches-CoReg/ OutDir=./Tmp_Patches-CoReg/ SpGOutSH=-SuperGlue  Viz=1
mm3d TestLib MergeTiePt ./Tmp_Patches-CoReg/  HomoXml=SubPatch_R180.xml MergeInSH=-SuperGlue MergeOutSH=-SubPatch_R180 PatchSz=[1280,960]    
mm3d TestLib RANSAC R2D MEC-Malt_1971.tif MEC-Malt_2014.tif Dir=./Tmp_Patches-CoReg/  2DRANInSH=-SubPatch_R180 2DRANOutSH=-SubPatch_R180-2DRANSAC

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971.tif MEC-Malt_2014.tif KH=NT SzW=[600,600] SH=-SubPatch_R180-2DRANSAC

You can see that the tie-points are not good, which indicates the current rotation hypothesis is not the good one.

Rotation hypothesis 4 (270 degree)

mm3d TestLib SuperGlue SuperGlueInput_R270.txt  InDir=./Tmp_Patches-CoReg/ OutDir=./Tmp_Patches-CoReg/ SpGOutSH=-SuperGlue Viz=1
mm3d TestLib MergeTiePt ./Tmp_Patches-CoReg/  HomoXml=SubPatch_R270.xml MergeInSH=-SuperGlue MergeOutSH=-SubPatch_R270 PatchSz=[1280,960]    
mm3d TestLib RANSAC R2D MEC-Malt_1971.tif MEC-Malt_2014.tif Dir=./Tmp_Patches-CoReg/  2DRANInSH=-SubPatch_R270 2DRANOutSH=-SubPatch_R270-2DRANSAC

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-CoReg/ MEC-Malt_1971.tif MEC-Malt_2014.tif KH=NT SzW=[600,600] SH=-SubPatch_R270-2DRANSAC

You can see that the tie-points are good, which indicates the current rotation hypothesis is the good one.

In conclusion, the rotation hypothesis 4 is obviously with the largest RANSAC inliers and hence should be kept.

Based on the largest number of RANSAC inlier from previous step, the command "TestLib CreateGCPs" will create GCPs between the 2 epochs.

The input, output and parameter interpretation of the command "TestLib CreateGCPs" are listed below:

Input:

• 2 sets of DSM, DSM gray image, orientations and image list in 2 epochs; tie-points between DSM gray images in 2 epochs

Output:

• 2 xml files to record the 2D and 3D coordinate of the GCPs in each epoch

The meaning of obligatory parameters:

• ./Tmp_Patches-CoReg: The directory of gray images of DSM
• MEC-Malt_1971.tif: The gray image of DSM of epoch 1971
• MEC-Malt_2014.tif: The gray image of DSM of epoch 2014
• ./: The directory of RGB images
• OIS-Reech_IGNF_PVA_1-0__1971.*tif: All RGB images in epoch 1971 (Dir+Pattern, or txt file of image list), referred to as ImgList1
• Crop.*tif: All RGB images in epoch 2014 (Dir+Pattern, or txt file of image list), referred to as ImgList2
• Ori-1971: Orientation of images in epoch 1971
• Ori-2014: Orientation of images in epoch 2014
• MEC-Malt_1971: DSM direcotry of epoch 1971
• MEC-Malt_2014: DSM direcotry of epoch 2014

The meaning of optional parameters:

• CreateGCPsInSH: Input Homologue extenion for NB/NT mode for CreateGCPs, Def=none
• Out2DXml1: Output xml files of 2D obersevations of the GCPs in epoch 1971, Def=OutGCP2D_epoch1.xml
• Out3DXml1: Output xml files of 3D obersevations of the GCPs in epoch 1971, Def=OutGCP3D_epoch1.xml
• Out2DXml2: Output xml files of 2D obersevations of the GCPs in epoch 2014, Def=OutGCP2D_epoch2.xml
• Out3DXml2: Output xml files of 3D obersevations of the GCPs in epoch 2014, Def=OutGCP3D_epoch2.xml
• DSMFileL: DSM File of epoch 1971, Def=MMLastNuage.xml
• DSMFileR: DSM File of epoch 2014, Def=MMLastNuage.xml

mm3d TestLib CreateGCPs ./Tmp_Patches-CoReg MEC-Malt_1971.tif MEC-Malt_2014.tif ./ OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif Ori-1971 Ori-2014 MEC-Malt_1971 MEC-Malt_2014 CreateGCPsInSH=-SubPatch_R270-2DRANSAC Out2DXml1=OutGCP2D_epoch1971.xml Out3DXml1=OutGCP3D_epoch1971.xml Out2DXml2=OutGCP2D_epoch2014.xml Out3DXml2=OutGCP3D_epoch2014.xml

Using the previously found GCPs, compute and apply a 3D Helmert transformation to transform the orientations of epoch 1971 (Ori-1971) to the frame of epoch 2014.

The input, output and parameter interpretation of the command "GCPBascule" are listed below:

Input:

• images; orientations; 2 xml files that recorded the 2D and 3D coordinate of the GCPs

Output:

• transformed orientations; 2 xml files to record the transformation parameters from one epoch to another

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971.*tif: the pattern of input images
• 1971: orientation in
• 2014: orientation out
• OutGCP3D_epoch2014.xml: Ground Control Points File
• OutGCP3D_epoch1971.xml: Image Measurements File

mm3d GCPBascule "OIS-Reech_IGNF_PVA_1-0__1971.*tif" 1971 2014 OutGCP3D_epoch2014.xml OutGCP2D_epoch1971.xml

This command will output:

(1) a file "Basc-1971-2-2014.xml", which indicates the transformation parameters from epoch 1971 to epoch 2014;

(2) a file "Basc-2014-2-1971.xml", which indicates the transformation parameters from epoch 2014 to epoch 1971.

The workflow of precise matching is displayed as below:

The command "TestLib GetOverlappedImages" will recognize overlapping inter-epoch image pairs based on co-registered result.

The input, output and parameter interpretation of the command "TestLib GetOverlappedImages" are listed below:

Input:

• 2 sets of image lists and orientations in 2 epochs; xml file that recorded the parameter of the 3D Helmert transformation from epoch 1 to epoch 2

Output:

• xml file to record the overlapping image pairs

The meaning of obligatory parameters:

• 1971: relative orientation of images in epoch 1971
• 2014: relative orientation of images in epoch 2014
• OIS-Reech_IGNF_PVA_1-0__1971.*tif: RGB images in epoch 1971 for extracting inter-epoch correspondences (Dir+Pattern, or txt file of image list)
• Crop.*tif: RGB images in epoch 2014 for extracting inter-epoch correspondences (Dir+Pattern, or txt file of image list)

The meaning of optional parameters:

• Dir: Work directory, Def=./
• OutPairXml: Output Xml file to record the overlapped image pairs, Def=OverlappedImages.xml
• Para3DH: Input xml file that recorded the parameter of the 3D Helmert transformation from epoch 1971 to epoch 2014, Def=none
• Print: Print corner coordinate, Def=false

mm3d TestLib GetOverlappedImages 1971 2014 OIS-Reech_IGNF_PVA_1-0__1971.*tif Crop.*tif  Para3DH=Basc-1971-2-2014.xml

The command "TestLib GetPatchPair" is used to get patch pairs for each overlapping inter-epoch RGB image pairs. It divides the master image into patches, and get the respective roughly aligned patches in the secondary image.

Note: This step is used as input for getting tentative tie-points with SuperGlue, as well as input for cross correlation.

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib GetPatchPair" are listed below:

Input:

• 2 sets of images and orientations from 2 epochs; xml file that recorded the parameter of the 3D Helmert transformation from epoch 1 to epoch 2

Output:

• image patches; patch pair list; xml file that recorded the homography between the patches and original image

The meaning of obligatory parameters:

• Guided: one of the 2 options (BruteForce and Guided) in this command. Guided means that we use the co-registered orientations to locate the corresponding patches in secondary image.
• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif: master image
• Crop-IMG_PHR1A_P_201406121049386.tif: secondary image
• Ori-1971: relative orientation of images in epoch 1971
• Ori-2014: relative orientation of images in epoch 2014

The meaning of optional parameters:

• Dir: Work directory, Def=./
• PatchSz: The patch size of the tiling scheme, which means the images to be matched by SuperGlue will be split into patches of this size, Def=[640, 480]
• BufferSz: Buffer zone size around the patch of the tiling scheme, Def=10%*PatchSz
• SubPXml: The output xml file name to record the homography between the patch and original image, Def=SubPatch.xml
• OutDir: Output direcotry of the patches, Def=./Tmp_Patches-Precise
• ImgPair: Output txt file that records the patch pairs, Def=SuperGlueInput.txt
• Para3DH: Input xml file that recorded the parameter of the 3D Helmert transformation from orientation of master image to secondary image, Def=none
• Print: Print corner coordinate, Def=false
• DSMDirL: DSM directory of master image, Def=none
• DSMFileL: DSM File of master image, Def=MMLastNuage.xml
• Prefix: The prefix for the name of images (for debug only), Def=none

mm3d TestLib GetPatchPair Guided OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif Ori-1971 Ori-2014  OutDir=./Tmp_Patches-Precise SubPXml=OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SubPatch.xml ImgPair=OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SuperGlueInput.txt  PatchSz=[1280,960] Para3DH=Basc-1971-2-2014.xml DSMDirL=MEC-Malt_1971

An example of patch pair is visualised below:

You can also visualize the patch pair in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-Precise/ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_1_0.tif Crop-IMG_PHR1A_P_201406121049386_OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_1_0.tif KH=S

There are 2 options for getting tentative tie-points: SuperGlue and SIFT.

Option 1: SuperGlue

(1) Get tie-points in patch pair

For each overlapping inter-epoch RGB image pairs, the command "Testlib SuperGlue" gets tentative tie-points in patch pairs based on SuperGlue.

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib SuperGlue" are listed below:

Input:

• images; list of image pairs

Output:

• SuperGlue tie-points

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SuperGlueInput.txt: the list of patch pairs (which is output from previous command "TestLib GetPatchPair")

The meaning of optional parameters:

• InDir: The input directory of the images for SuperGlue, Def=./
• SpGOutSH: Homologue extenion for NB/NT mode of SuperGlue, Def=-SuperGlue
• OutDir: The output directory of the match results of SuperGlue, Def=./
• Resize: The goal size for resizing the input image for SuperGlue, Def=[640, 480], if you don't want to resize, please set to [-1, -1]
• Viz: Visualize the matches and dump the plots of SuperGlue, Def=false
• Model: Pretrained indoor or outdoor model of SuperGlue, Def=outdoor
• MaxPt: Maximum number of keypoints detected by Superpoint, Def=1024
• KeepNpzFile: Keep the original npz file that SuperGlue outputed, Def=false
• EntSpG: The SuperGlue program entry (this option is only used for developper), Def=../micmac/src/uti_phgrm/TiePHistorical/SuperGluePretrainedNetwork-master/match_pairs.py
• opt: Other options for SuperGlue (this option is only used for developper), Def=none
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false
• CheckNb: Radius of the search space for SuperGlue (which means correspondence [(xL, yL), (xR, yR)] with (xL-xR)×(xL-xR)+(yL-yR)×(yL-yR) > CheckNb×CheckNb will be removed afterwards), Def=-1 (means don't check search space)

Note: Here we set CheckNb=100 as the patch pairs are roughly aligned.

mm3d TestLib SuperGlue OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SuperGlueInput.txt  InDir=./Tmp_Patches-Precise/ OutDir=./Tmp_Patches-Precise/ SpGOutSH=-SuperGlue CheckNb=100 Viz=1

As a result, it will extract tie-points for each patch pair, an example of tie-points of one patch pair is visulized below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL Tmp_Patches-Precise/ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_1_0.tif Crop-IMG_PHR1A_P_201406121049386_OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_1_0.tif KH=NT SzW=[600,600] SH=-SuperGlue

(2) Transform tie-points in patch pairs to image pair

Then the command "TestLib MergeTiePt" will be launched to merge the tentative tie-points in the format of patch pairs to the format of image pair.

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib MergeTiePt" are listed below:

Input:

• images; tie-points in format of patch pairs; xml file that recorded the homograhpy from patches to original image

Output:

• tie-points in format of image pair

The meaning of obligatory parameters:

• ./Tmp_Patches-Precise/: work directory

The meaning of optional parameters:

• HomoXml: the input xml file that recorded the patch pairs whose tie-points should be merged
• MergeInSH: input Homologue extenion
• MergeOutSH: output Homologue extenion
• HomoXml: input xml file that recorded the homograhpy from patches to original image for MergeTiePt, Def=SubPatch.xml
• PatchSz: Patch size of the tiling scheme (since we use the patches resulted from "GetPatchPair", this parameter should be set the same as the PatchSz in command GetPatchPair), Def=[640, 480]
• BufferSz: Buffer zone size around the patch of the tiling scheme (since we use the patches resulted from "GetPatchPair", this parameter should be set the same as the BufferSz in command GetPatchPair), Def=[0,0]

mm3d TestLib MergeTiePt ./Tmp_Patches-Precise/  HomoXml=OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SubPatch.xml MergeInSH=-SuperGlue MergeOutSH=-SuperGlue  OutDir=./ PatchSz=[1280,960] BufferSz=[128,96]

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-SuperGlue

Option 2: SIFT

For each overlapping inter-epoch RGB image pairs, the command "Testlib GuidedSIFTMatch" gets tentative tie-points based on SIFT.

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib GuidedSIFTMatch" are listed below:

Input:

• 2 sets of images, orientations and DSMs of 2 epochs; xml file that recorded the paremeter of the 3D Helmert transformation from one epoch to another

Output:

• SIFT tie-points

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif: master image
• Crop-IMG_PHR1A_P_201406121049386.tif: secondary image
• Ori-1971: orientation of images in epoch 1971
• Ori-2014: orientation of images in epoch 2014

The meaning of optional parameters:

• Dir: Work directory, Def=./
• GSIFTOutSH: Output Homologue extenion for NB/NT mode of Guided SIFT, Def=-GuidedSIFT
• SkipSIFT: Skip extracting SIFT key points in case it is already done, Def=false
• SearchSpace: Radius of the search space for GuidedSIFT (the search space is the circle with the center on the predicted point), Def=100
• MutualNN: Apply mutual nearest neighbor on GuidedSIFT, Def=true
• RatioT: Apply ratio test on GuidedSIFT, Def=true
• RootSift: Use RootSIFT as descriptor on GuidedSIFT, Def=true
• CheckScale: Check the scale of the candidate tie-points on GuidedSIFT, Def=true
• CheckAngle: Check the angle of the candidate tie-points on GuidedSIFT, Def=true
• ScaleTh: The threshold for checking scale ratio, Def=0.2; (0.2 means the ratio of master and secondary SIFT scale between [(1-0.2)×Ref, (1+0.2)×Ref] is considered valide. Ref is automatically calculated by reprojection.)
• AngleTh: The threshold for checking angle difference, Def=30; (30 means the difference of master and secondary SIFT angle between [Ref - 30 degree, Ref + 30 degree] is considered valide. Ref is automatically calculated by reprojection.)
• Predict: Use the predicted key points to guide the matching, Def=true
• DSMDirL: DSM of master image (for improving the reprojecting accuracy), Def=none
• DSMDirR: DSM of secondary image (for improving the reprojecting accuracy), Def=none
• DSMFileL: DSM File of master image, Def=MMLastNuage.xml
• DSMFileR: DSM File of secondary image, Def=MMLastNuage.xml
• Para3DH: Input xml file that recorded the parameter of the 3D Helmert transformation from orientation of master image to secondary image, Def=none
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false
• ScaleL: Extract SIFT points on master images downsampled with a factor of "ScaleL", Def=1
• ScaleR: Extract SIFT points on secondary images downsampled with a factor of "ScaleR", Def=1

mm3d TestLib GuidedSIFTMatch OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif Ori-1971 Ori-2014  SkipSIFT=true DSMDirL=MEC-Malt_1971 DSMDirR=MEC-Malt_2014 Para3DH=Basc-1971-2-2014.xml ScaleR=2

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-GuidedSIFT

For each overlapping inter-epoch image pairs, the command "TestLib RANSAC R3D" applies 3D RANSAC on tentative tie-points to get enhanced tie-points.

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib RANSAC R3D" are listed below:

Input:

• 2 sets of images, orientations and DSMs of 2 epochs; tie-points

Output:

• filtered tie-points

The meaning of obligatory parameters:

• R3D: one of the 2 options (R2D and R3D) available in this command
• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif: master image
• Crop-IMG_PHR1A_P_201406121049386.tif: secondary image
• Ori-1971: relative orientation of images in epoch 1971
• Ori-2014: relative orientation of images in epoch 2014

The meaning of optional parameters:

• Dir: Work directory, Def=./
• 3DRANInSH: Input Homologue extenion for NB/NT mode for 3D RANSAC, Def=none
• 3DRANOutSH: Output Homologue extenion for NB/NT mode of 3D RANSAC, Def='3DRANInSH'-3DRANSAC
• 3DIter: 3D RANSAC iteration, Def=1000
• 3DRANTh: 3D RANSAC threshold, Def=10*(GSD of secondary image)
• DSMDirL: DSM directory of master image, Def=none
• DSMDirR: DSM directory of secondary image, Def=none
• DSMFileL: DSM File of master image, Def=MMLastNuage.xml
• DSMFileR: DSM File of secondary image, Def=MMLastNuage.xml
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false

mm3d TestLib RANSAC R3D OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif Ori-1971 Ori-2014 Dir=./  DSMDirL=MEC-Malt_1971 DSMDirR=MEC-Malt_2014 DSMFileL=MMLastNuage.xml DSMFileR=MMLastNuage.xml 3DRANInSH=-SuperGlue 3DRANOutSH=-SuperGlue-3DRANSAC 3DRANTh=10

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-SuperGlue-3DRANSAC

The command "TestLib CrossCorrelation" applies cross correlation on enhanced tie-points to get final tie-points, with the help of patches resulted from step "GetPatchPair".

Each overlapping inter-epoch RGB image pair corresponds to a command line, here we only show one image pair as example.

The input, output and parameter interpretation of the command "TestLib CrossCorrelation" are listed below:

Input:

• images; patches; xml file that recorded the homography between the patches and original image; tie-points

Output:

• filtered tie-points

The meaning of obligatory parameters:

• OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif: master image
• Crop-IMG_PHR1A_P_201406121049386.tif: secondary image

The meaning of optional parameters:

• Dir: Work directory, Def=./
• CCInSH: Input Homologue extenion for NB/NT mode for cross correlation, Def=none
• CCOutSH: Output Homologue extenion for NB/NT mode of cross correlation, Def='CCInSH'-CrossCorrelation
• SzW: Window size of cross correlation, Def=32
• CCTh: Corss correlation threshold, Def=0.5
• PatchSz: the patch size of the tiling scheme (since we use the patches resulted from "GetPatchPair" to calculate the cross correlation, this parameter should be set the same as the PatchSz in command GetPatchPair), Def=[640, 480]
• BufferSz: Buffer zone size around the patch of the tiling scheme (since we use the patches resulted from "GetPatchPair" to calculate the cross correlation, this parameter should be set the same as the BufferSz in command GetPatchPair), Def=10%*PatchSz
• SubPXml: The xml file name to record the homography between the patch and original image, Def=SubPatch.xml
• PatchDir: The input directory of patches, Def=./Tmp_Patches
• CheckFile: Check if the result files of inter-epoch tie-points exist (if so, skip to avoid repetition), Def=false

mm3d TestLib CrossCorrelation OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif  CCInSH=-SuperGlue-3DRANSAC CCOutSH=-SuperGlue-3DRANSAC-CrossCorrelation SzW=32 CCTh=0.5 PatchSz=[1280,960] BufferSz=[30,60] PatchDir=./Tmp_Patches-Precise SubPXml=OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067_Crop-IMG_PHR1A_P_201406121049386_SubPatch.xml

The resulting tie-points are visualised below:

You can also visualize the tie-points in MicMac using the command 'SEL':

mm3d SEL ./ OIS-Reech_IGNF_PVA_1-0__1971-06-21__C2844-0141_1971_FR2117_1067.tif Crop-IMG_PHR1A_P_201406121049386.tif KH=NT SzW=[600,600] SH=-SuperGlue-3DRANSAC-CrossCorrelation

In this section we will compare the DoD (Difference of DSMs) for evaluation.

We use 3 sets of image orientations to compute the DSMs in epoch 1971 and 2014 individually and display the difference between them. The 3 sets of orientations are:

(1) Roughly co-registered orientations: which are the orientations we got at the end of rough co-registration.

(2) SuperGlue refined orientations: based on the roughly co-registered orientations, we refine them in a boudle adjustment routine using the SuperGlue inter-epoch tie-points.

(3) SIFT refined orientations: based on the roughly co-registered orientations, we refine them in a boudle adjustment routine using the SIFT inter-epoch tie-points.

As the co-registered orientations are based on the reference of epoch 2014, we can use directly the DSM of epoch 2014 resulted from section 1.3.1.

As for the DSM of epoch 1971, the roughly co-registered orientations are stored in folder "Ori-1971_CoReg_SuperGlue", therefore we use it to calculate the DSM of epoch 1971 as follwing:

mm3d Malt Ortho OIS-Reech_IGNF_PVA_1-0__1971.*tif Ori-1971_CoReg_SuperGlue NbVI=2 DirMEC=MEC-Malt_1971_CoReg EZA=1 MasqImGlob=Fiducial.tif ZoomF=8 DoOrtho=0

We use the command "CmpIm" to generate the DoD.

The input, output and parameter interpretation of the command "CmpIm" are listed below:

Input:

• DSMs in 2 epochs

Output:

• DoD

The meaning of obligatory parameters:

• MEC-Malt_1971_CoReg/Z_Num6_DeZoom8_STD-MALT.tif: DSM in epoch 1971.
• MEC-Malt_2014/Z_Num6_DeZoom8_STD-MALT.tif: DSM in epoch 2014.

The meaning of optional parameters:

• UseFOM: Consider file as DTSM and use XML FileOriMnt
• FileDiff: Difference image output file
• 16Bit: Output file in float format

mm3d CmpIm MEC-Malt_2014/Z_Num6_DeZoom8_STD-MALT.tif MEC-Malt_1971_CoReg/Z_Num6_DeZoom8_STD-MALT.tif UseFOM=1 FileDiff=DoD-CoReg.tif 16Bit=1

The resulting DoD can be visualised below:

You can also visualize it in QGIS.

As can be seen, a dome artifact is present in DoD-CoReg. This kind of systematic error is known to originate from poorly modeled camera internal parameters.

We need to refine the roughly co-registered orientations in a bundle adjustment (BA) routine using the SuperGlue inter-epoch tie-points resulted from section 2.1, as well as the reduced intra-epoch tie-points resulted from section 1.1.

First of all, we use the command "TestLib TiePtAddWeight" to set the weight of the inter-epoch tie-points to be 2, so that they will play a more important role in BA. (Please notice that the weight of the intra-epoch tie-points is by default 1.)

The input, output and parameter interpretation of the command "TestLib TiePtAddWeight" are listed below:

Input:

• tie-points

Output:

• tie-points with weight set

The meaning of obligatory parameters:

• 2: Weight to be set

The meaning of optional parameters:

• InSH: Input Homologue extenion for NB/NT mode, Def=none
• OutSH: Output Homologue extenion for NB/NT mode, Def=InSH-WN (N means the weight)
• ScaleL: The factor used to scale the points in master images (for developpers only), Def=1

mm3d TestLib TiePtAddWeight 2 InSH=-SuperGlue-3DRANSAC-CrossCorrelation

The SuperGlue inter-epoch tie-points we got are in txt format, we should transform them into binary format with the help of "HomolFilterMasq", so that they can be recognized in the following process.

mm3d HomolFilterMasq "[O|C].*tif" PostIn=-SuperGlue-3DRANSAC-CrossCorrelation-W2 PostOut=-SuperGlue-3DRANSAC-CrossCorrelation-W2-dat ANM=1 ExpTxt=1 ExpTxtOut=0

Then we need to merge the intra- and inter-epoch tie-points from different folders together using the command "MergeHomol".

The input, output and parameter interpretation of the command "MergeHomol" are listed below:

Input:

• tie-points in different folders

Output:

• tie-points merged in a single folder

The meaning of obligatory parameters:

• "Homol_1971-Ratafia|Homol-SuperGlue-3DRANSAC-CrossCorrelation-W2-dat": input tie-point folders
• Homol_Merged-SuperGlue: out tie-point folder

Note: As the images in epoch 2014 are satellite images, their orientations are already georeferenced and therefore will be treated as ground truth in our processing, so we don't need the intra-epoch tie-points of epoch 2014.

mm3d MergeHomol "Homol_1971-Ratafia|Homol-SuperGlue-3DRANSAC-CrossCorrelation-W2-dat" Homol_Merged-SuperGlue

The roughly co-registered orientations of epoch 1971 are stored in folder "Ori-1971_CoReg_SuperGlue", we use the following command to copy them into the folder "Ori-2014" in order to combine the orientations of 2 epochs:

cp -r Ori-1971_CoReg_SuperGlue/. Ori-2014/

The merged folder "Ori-2014" serves as intial orientations for the next step of "bundle adjustment".

Now it is time to run BA with the command "Campari".

The input, output and parameter interpretation of the command "Campari" are listed below:

Input:

• images, rough orientations

Output:

• refined orientations

The meaning of obligatory parameters:

• "[O|C].*tif": Full Directory (Dir+Pattern)
• 2014: Input Orientation
• Campari_Refined-SuperGlue: Output Orientation

The meaning of optional parameters:

• SH: Set of Hom, Def="", give MasqFiltered for result of HomolFilterMasq, set NONE if unused
• AllFree: Refine all calibration parameters (Def=false)
• NbIterEnd: Number of iteration at end, Def = 4
• SigmaTieP: Sigma use for TieP weighting (Def=1)
• AllFreePat: Pattern of images that will be subject to AllFree (Def=.*)
• FrozenPoses: List of frozen poses (pattern)

Note: As the images in epoch 2014 are satellite images, their orientations are already georeferenced and therefore will be treated as ground truth in our processing, so we set FrozenPoses="C.*tif" combined with AllFreePat=".19." to ensure only the orientations of epoch 1971 will be refined.

mm3d Campari "[O|C].*tif" 2014 Campari_Refined-SuperGlue SH=_Merged-SuperGlue AllFree=1 AllFreePat=".*19.*" FrozenPoses="C.*tif" NbIterEnd=20

As the orientations of epoch 2014 are kept the same, we can use directly the DSM of epoch 2014 resulted from section 1.3.1.

Based on the SuperGlue refined orientations "Campari_Refined-SuperGlue", we compute the DSMs in epoch 1971 using the command "Malt":

mm3d Malt Ortho OIS-Reech_IGNF_PVA_1-0__1971.*tif Campari_Refined-SuperGlue NbVI=2 DirMEC=MEC-Malt_1971_Refined-SuperGlue EZA=1 MasqImGlob=Fiducial.tif ZoomF=8 DoOrtho=0

Finally we use the command "CmpIm" to generate the DoD.

mm3d CmpIm MEC-Malt_2014/Z_Num6_DeZoom8_STD-MALT.tif MEC-Malt_1971_Refined-SuperGlue/Z_Num6_DeZoom8_STD-MALT.tif UseFOM=1 FileDiff=DoD-Refined-SuperGlue.tif 16Bit=1

The resulting DoD can be visualised below:

You can also visualize it in QGIS.

As can be seen in the visualized DoDs, the systematic errors are effectively mitigated in the DoD of SuperGlue refined result, thanks to our dense and precise inter-epoch tie-points.

We need to refine the roughly co-registered orientations in a bundle adjustment (BA) routine using the SIFT inter-epoch tie-points resulted from section 2.1, as well as the reduced intra-epoch tie-points resulted from section 1.1.

First of all, we use the command "TestLib TiePtAddWeight" to set the weight of the inter-epoch tie-points to be 2, so that they will play a more important role in BA. (Please notice that the weight of the intra-epoch tie-points is by default 1.)

The input, output and parameter interpretation of the command "TestLib TiePtAddWeight" are listed below:

Input:

• tie-points

Output:

• tie-points with weight set

The meaning of obligatory parameters:

• 2: Weight to be set

The meaning of optional parameters:

• InSH: Input Homologue extenion for NB/NT mode, Def=none
• OutSH: Output Homologue extenion for NB/NT mode, Def=InSH-WN (N means the weight)
• ScaleL: The factor used to scale the points in master images (for developpers only), Def=1

mm3d TestLib TiePtAddWeight 2 InSH=-GuidedSIFT-3DRANSAC-CrossCorrelation

The SIFT inter-epoch tie-points we got are in txt format, we should transform them into binary format with the help of "HomolFilterMasq", so that they can be recognized in the following process.

mm3d HomolFilterMasq "[O|C].*tif" PostIn=-GuidedSIFT-3DRANSAC-CrossCorrelation-W2 PostOut=-GuidedSIFT-3DRANSAC-CrossCorrelation-W2-dat ANM=1 ExpTxt=1 ExpTxtOut=0

Then we need to merge the intra- and inter-epoch tie-points from different folders together using the command "MergeHomol".

The input, output and parameter interpretation of the command "MergeHomol" are listed below:

Input:

• tie-points in different folders

Output:

• tie-points merged in a single folder

The meaning of obligatory parameters:

• "Homol_1971-Ratafia|Homol-GuidedSIFT-3DRANSAC-CrossCorrelation-W2-dat": input tie-point folders
• Homol_Merged-GuidedSIFT: out tie-point folder

mm3d MergeHomol "Homol_1971-Ratafia|Homol-GuidedSIFT-3DRANSAC-CrossCorrelation-W2-dat" Homol_Merged-GuidedSIFT

Now it is time to run BA with the command "Campari".

The input, output and parameter interpretation of the command "Campari" are listed below:

Input:

• images, rough orientations

Output:

• refined orientations

The meaning of obligatory parameters:

• "[O|C].*tif": Full Directory (Dir+Pattern)
• 2014: Input Orientation
• Campari_Refined-GuidedSIFT: Output Orientation

The meaning of optional parameters:

• SH: Set of Hom, Def="", give MasqFiltered for result of HomolFilterMasq, set NONE if unused
• AllFree: Refine all calibration parameters (Def=false)
• NbIterEnd: Number of iteration at end, Def = 4
• SigmaTieP: Sigma use for TieP weighting (Def=1)
• AllFreePat: Pattern of images that will be subject to AllFree (Def=.*)
• FrozenPoses: List of frozen poses (pattern)

Note: As the images in epoch 2014 are satellite images, their orientations are already georeferenced and therefore will be treated as ground truth in our processing, so we set FrozenPoses="C.*tif" combined with AllFreePat=".19." to ensure only the orientations of epoch 1971 will be refined.

mm3d Campari "[O|C].*tif" 2014 Campari_Refined-GuidedSIFT SH=_Merged-GuidedSIFT AllFree=1 AllFreePat=".*19.*" FrozenPoses="C.*tif" NbIterEnd=20

As the orientations of epoch 2014 are kept the same, we can use directly the DSM of epoch 2014 resulted from section 1.3.1.

Based on the GuidedSIFT refined orientations "Campari_Refined-GuidedSIFT", we compute the DSMs in epoch 1971 using the command "Malt":

mm3d Malt Ortho OIS-Reech_IGNF_PVA_1-0__1971.*tif Campari_Refined-GuidedSIFT NbVI=2 DirMEC=MEC-Malt_1971_Refined-GuidedSIFT EZA=1 MasqImGlob=Fiducial.tif ZoomF=8 DoOrtho=0

Finally we use the command "CmpIm" to generate the DoD.

mm3d CmpIm MEC-Malt_2014/Z_Num6_DeZoom8_STD-MALT.tif MEC-Malt_1971_Refined-GuidedSIFT/Z_Num6_DeZoom8_STD-MALT.tif UseFOM=1 FileDiff=DoD-Refined-GuidedSIFT.tif 16Bit=1

The resulting DoD can be visualised below:

You can also visualize it in QGIS.

As can be seen in the visualized DoDs, the systematic errors are effectively mitigated in the DoD of SuperGlue and SIFT refined result, thanks to our dense and precise inter-epoch tie-points.

We chose a small region to keep the dataset compact to improve the processing efficiency of this tutorial. The performance of our method might improve when applied to bigger regions.