Hello <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-ur

I made an experimental stitcher: <div class="highlight highlight-source-python not

Hello again, how large are your input files? 1k x 1k pixels?</

I made a version using composite: <div class="highlight highlight-source-python no

memory and writing issue during stitching of large images about pyvips HOT 8 OPEN

libvips commented on June 19, 2024

memory and writing issue during stitching of large images

from pyvips.

Comments (8)

jcupitt commented on June 19, 2024 1

I made an experimental stitcher:

#!/usr/bin/env python

import sys
import pyvips

# overlap joins by this many pixels
H_OVERLAP = 100
V_OVERLAP = 100

# number of images in mosaic
ACROSS = 40
DOWN = 40

if len(sys.argv) < 2 + ACROSS * DOWN:
    print 'usage: %s output-image input1 input2 ..'
    sys.exit(1)

def join_left_right(filenames):
    images = [pyvips.Image.new_from_file(filename) for filename in filenames]
    row = images[0]
    for image in images[1:]:
        row = row.merge(image, 'horizontal', H_OVERLAP - row.width, 0)

    return row

def join_top_bottom(rows):
    image = rows[0]
    for row in rows[1:]:
        image = image.merge(row, 'vertical', 0, V_OVERLAP - image.height)

    return image

rows = []
for y in range(0, DOWN):
    start = 2 + y * ACROSS
    end = start + ACROSS
    rows.append(join_left_right(sys.argv[start:end]))

image = join_top_bottom(rows)

image.write_to_file(sys.argv[1])

I can run it like this:

$ export VIPS_DISC_THRESHOLD=100
$ export VIPS_PROGRESS=1
$ export VIPS_CONCURRENCY=1
$ mkdir sample
$ for i in {1..1600}; do cp ~/pics/k2.jpg sample/$i.jpg; done
$ time ./mergeup.py x.dz sample/*.jpg
real  3m16.625s
user  3m1.062s
sys   0m35.010s
peak RES 14gb

k2.jpg is a 1500 x 2000 pixel RGB JPG.

The VIPS_DISC_THRESHOLD env var sets the point at which libvips flips between loading to memory and loading via a temp file: we need to save memory here, so we set the threshold low to make it decode the JPG images to temporary files in /tmp.

CONCURRENCY sets the size of the libvips worker pool. Setting it to 1 saves memory for per-thread buffers, though it does make it run a little slower.

PROGRESS make it output some stuff as it runs, which is handy for debugging.

So: I was able to blend 1600 images with merge on this machine into 54100 x 78020 pixel deepzoom pyramid in about 3m. With 64gb of memory, you should be able to join 80 x 80 images in about 12m, though it will vary a bit with image size.

To go higher, I think you'll need to assemble your image in sections, sorry. Write each section out to a .v file to keep transparency (you'll need a LOT of disc space), then do a second pass where you paste the sections together with more merge operators.

from pyvips.

jcupitt commented on June 19, 2024

Hello again,

how large are your input files? 1k x 1k pixels?
I guess they are jpg format, is that right?
how many are you hoping to join at once? 500 x 500?

from pyvips.

jcupitt commented on June 19, 2024

I made a version using composite:

#!/usr/bin/env python

import sys
import random
import pyvips

# number of images in mosaic
ACROSS = 40
DOWN = 40

# size of output image
WIDTH = ACROSS * 1500
HEIGHT = DOWN * 2000

# sigma of the feather we add, and approximate radius in pixels
FEATHER_SIGMA = 4
FEATHER_WIDTH = 7

if len(sys.argv) < 2 + ACROSS * DOWN:
    print 'usage: %s output-image input1 input2 ..'
    sys.exit(1)

images = [pyvips.Image.new_from_file(filename) for filename in sys.argv[2:]]

def make_white(width, height):
    white = pyvips.Image.black(width, height) + 255
    white = white.cast('uchar')
    
    return white

# make a one-band uchar image with a feathered edge
def soft_alpha(width, height):
    black = pyvips.Image.black(width, height)  
    white = make_white(width - 2 * FEATHER_WIDTH,  
                       height - 2 * FEATHER_WIDTH)
    mask = black.insert(white, FEATHER_WIDTH, FEATHER_WIDTH)
    alpha = mask.gaussblur(FEATHER_SIGMA, precision='integer')
    white = make_white(width - 4 * FEATHER_WIDTH,  
                       height - 4 * FEATHER_WIDTH)
    alpha = alpha.insert(white, 2 * FEATHER_WIDTH, 2 * FEATHER_WIDTH)

    return alpha

# add a soft alpha and pad each image out to the final size
images = [image
            .bandjoin(soft_alpha(image.width, image.height))
            .embed(random.randint(0, WIDTH),
                   random.randint(0, HEIGHT),
                   WIDTH, HEIGHT) for image in images]

# composite all the images together
final = images[0].composite(images[1:], 'over')

final.write_to_file(sys.argv[1])

It's slow, but I'll leave it running, perhaps it'll actually use less memory for very large mosaics.

from pyvips.

shivamchaubey commented on June 19, 2024

how large are your input files? 1k x 1k pixels?
I guess they are jpg format, is that right?
how many are you hoping to join at once? 500 x 500?

Currently, I am using Input image of resolution 640X480. Later on, Image of resolution 1920X1200 can also be used.
Yes, images are in jpg format.
Maximum it will go up to 500X500 images.

I would be running stitcher code on 8GB RAM, so the code should take less than 8GB RAM.
I have tried other way to do it, I have used only join function assuming there is no vertical shift accross the row and for stitching vertical stitched rows I have used both vertical and horizontal translation, added mask to make x,y translation. Using this method I have stitched 156X56 images in less than 3GB RAM, the problem I am facing is to save the final result either in .dzi or tiff format. In saving the result sometimes I get the error segmentation fault. Is there any argument which has to be set to save the large mosiaced image?

from pyvips.

shivamchaubey commented on June 19, 2024

I have tried some attempts to make it work. I have stitched all the images and eventually cropped the final mosaiced image in four part. All the cropped image joined in the last and saved.
Code:

#tile1 == mosaiced image
height=tile1.height
width=tile1.width
#cropping top left part
top_left = tile1.extract_area(0, 0, tile1.width/2, tile1.height/2)
#cropping top right part
top_right = tile1.extract_area(tile1.width/2, 0, tile1.width/2, tile1.height/2)
#joining top left and right image
top=top_left.join(top_right, "horizontal")        
#cropping bottom left part
bottom_left = tile1.extract_area(0, tile1.height/2, tile1.width/2, tile1.height/2)
#cropping bottom right part
bottom_right = tile1.extract_area(tile1.width/2, tile1.height/2, tile1.width/2, tile1.height/2)    
#joining bottom left and right image 
bottom=bottom_left.join(bottom_right, "horizontal")        
#joining top and bottom image
result=top.join(bottom, "vertical")
#saving image in dzi format        
result.dzsave("test")

By doing this RAM uses is decreased by half in one case, actually it is dependent on size of the mosaicd image.
Any suggestion from your side? I have to stitch at least 200X200 image of resoltution 640X480 in less than 6GB RAM.

from pyvips.

shivamchaubey commented on June 19, 2024

I am posting test code for stitching images using join function.

from os import listdir,remove
from os.path import isfile, join
import re
import pyvips
import time
from tqdm import tqdm
import numpy as np

if __name__=="__main__":
    mypath='./CASE4A_1_focused'    
    onlyfiles = [ f for f in listdir(mypath) if isfile(join(mypath,f)) ]
    images = np.empty(len(onlyfiles), dtype=object)
    onlyfiles.sort(key=lambda var:[int(x) if x.isdigit() else x for x in re.findall(r'[^0-9]|[0-9]+', var)])
    column=156
    row=8
    overlap_x=192
    overlap_y=96
    
    for i in tqdm(range(row)):
        files=onlyfiles[i*column:(i+1)*column]
        if i%2==1:
            files.reverse()
        if i==1:
                row_tile1=tile1
            
        for j in tqdm(range(column-1)):
            if j==0:
                tile1 = pyvips.Image.new_from_file(join(mypath,files[j]), access="sequential")
            tile2 = pyvips.Image.new_from_file(join(mypath,files[j+1]), access="sequential")
            tile2 = tile2.crop(overlap_x,0, tile2.width-overlap_x,tile2.height)            
            tile1=tile1.join(tile2, "horizontal")
        if i>0:
            tile1 = tile1.crop(0,overlap_y, tile1.width,tile1.height-overlap_y)            
            row_tile1=row_tile1.join(tile1, "vertical")
    print "saving mosaiced image"
    row_tile1.write_to_file('test.tif')

Above code is not working for more than 8 rows. It is throwing an error while saving the final mosaiced image:

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  4.40it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 155/155 [00:00<00:00, 690.47it/s]
 saving mosaiced image
Traceback (most recent call last):
  File "githubpost.py", line 36, in <module>
    row_tile1.write_to_file('test.tif')
  File "/home/lab/.virtualenvs/project/local/lib/python2.7/site-packages/pyvips/vimage.py", line 481, in write_to_file
    ), **kwargs)
  File "/home/lab/.virtualenvs/project/local/lib/python2.7/site-packages/pyvips/voperation.py", line 188, in call
    raise Error('unable to call {0}'.format(operation_name))
pyvips.error.Error: unable to call VipsForeignSaveTiffFile
  TIFFSetField: test.tif: Unknown tag 317
vips__file_open_read: unable to open file "./CASE4A_1_focused/1019_corrected.jpg" for reading
unix error: Too many open files

Uses: python file_name.py
mpypath, row and column need to be chaged to run code on your image set.
This code is written only for testing purpose only.

from pyvips.

shivamchaubey commented on June 19, 2024

I have tried your second code composite one. I am getting following error.

Traceback (most recent call last):
  File "./github_pyvips_test.py", line 95, in <module>
    final = images[0].composite(images[1:], 'over')
  File "/home/lab/.virtualenvs/project/local/lib/python2.7/site-packages/pyvips/vimage.py", line 1055, in composite
    **kwargs)
  File "/home/lab/.virtualenvs/project/local/lib/python2.7/site-packages/pyvips/voperation.py", line 188, in call
    raise Error('unable to call {0}'.format(operation_name))
pyvips.error.Error: unable to call composite
  composite: for 1600 input images there must be 1599 blend modes
real	0m33.222s
user	0m32.115s
sys	0m0.969s

I have followed this to run the code.

$ export VIPS_DISC_THRESHOLD=100
$ export VIPS_PROGRESS=1
$ export VIPS_CONCURRENCY=1
$ mkdir sample
$ for i in {1..1600}; do cp ~/pics/k2.jpg sample/$i.jpg; done
$ time ./mergeup.py x.dz sample/*.jpg

Please look at it.

from pyvips.

shivamchaubey commented on June 19, 2024

Hello @jcupitt

I have tried to check where the code is taking maximum RAM. I am attaching massif file generated from valgrind https://github.com/KratosMultiphysics/Kratos/wiki/Checking-memory-usage-with-Valgrind. Please have a look on the massif file https://drive.google.com/file/d/1SCek6dNx521PMQolEa3vII8ufqRPcX4r/view?usp=sharing. The generated file is a result of stitching 156 column X 10 rows of images.
I am attaching a screenshot of the massif visualizer. Here you can see function vips_region_fill is taking max RAM uses while saving the final tiff or dzi image result. Location- Region.c: 903

In the python code I have just used only join function and the problem is a segmentation fault. RAM uses is less than Maximum RAM size <16 GB. Please give some suggestion to save the final result.

from pyvips.

memory and writing issue during stitching of large images about pyvips HOT 8 OPEN

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent