czq142857 / decor-gan Goto Github PK

View Code? Open in Web Editor NEW

83.0 83.0 13.0 1.2 MB

PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement.

License: MIT License

Python 86.73% Cython 13.27%

decor-gan's People

Contributors

Stargazers

Watchers

Forkers

peterzhousz wx-b zebrajack gitbosun tiamat-tech trendingtechnology johndpope sea-xiaod nathanielwei deltamarine101 qyunhuang qiminchen rucha97

decor-gan's Issues

understanding data preparation

Hello,
thanks for publishing your code :)

I have the following questions regarding the 3 possible scripts for step 3:

is there some documentation on how depth fusion is done & what is the difference between depth fusion and flood fill approach?
Is the dimension 2560 for rendering variable arbitrarily chosen? and why is state_ctr shape 512x512x64, 2?
what should be the difference between the initial .binvox created in step 2 and the .binvox created in step 3?

Thanks in advance

How can I prepare my data to test?

Hello,

On the trained dataset, I want to pass a point cloud and check how Decor GAN works.
The current guidelines for data preparation only mention processing the ShapeNet dataset. But how can I convert my data into ShapeNet hierarchy?
Any help would be appreciated. TIA!

the dilation in conv3d

Thanks for your great work!
After reading your paper, I have some questions about the dilation in conv3d and the patchDiscriminator, which puzzle me a lot.
The discriminator and generator architecture figures in your paper show that you use conv3d with dilation=0 in some layers, for example, the layers in the discriminator. But in the script, the conv3d layers use default dilation, which is 1 according to official explanation.
So I want to know why is that. And I also can't find any articles to explain what conv3d with dilation=0 dose and how to utilize that to accomplish patchDiscriminator.

class discriminator(nn.Module):
    def __init__(self, d_dim, z_dim):
        super(discriminator, self).__init__()
        self.d_dim = d_dim
        self.z_dim = z_dim

        self.conv_1 = nn.Conv3d(1,             self.d_dim,    4, stride=1, padding=0, bias=True)
        self.conv_2 = nn.Conv3d(self.d_dim,    self.d_dim*2,  3, stride=2, padding=0, bias=True)
        self.conv_3 = nn.Conv3d(self.d_dim*2,  self.d_dim*4,  3, stride=1, padding=0, bias=True)
        self.conv_4 = nn.Conv3d(self.d_dim*4,  self.d_dim*8,  3, stride=1, padding=0, bias=True)
        self.conv_5 = nn.Conv3d(self.d_dim*8,  self.d_dim*16, 3, stride=1, padding=0, bias=True)
        self.conv_6 = nn.Conv3d(self.d_dim*16, self.z_dim,    1, stride=1, padding=0, bias=True)

Suggesting bug fix in get_voxel_bbox

Hello,
I'd like to suggest a bug fix I found while running the code on my meshes ...
I don't know if it was on purpose, but the current implementation of get_voxel_bbox is always returning 1 as xmin, ymin, zmin, and never 0, thus my voxelized meshes were cut when crop_voxel method was called.

Therefore I modified get_voxel_bbox into the following:

    def get_voxel_bbox(self, vox):
        # minimap
        vox_tensor = torch.from_numpy(vox).to(self.device).unsqueeze(0).unsqueeze(0).float()
        smallmaskx_tensor = F.max_pool3d(vox_tensor, kernel_size=self.upsample_rate, stride=self.upsample_rate,
                                         padding=0)
        smallmaskx = smallmaskx_tensor.detach().cpu().numpy()[0, 0]
        smallmaskx = np.round(smallmaskx).astype(np.uint8)
        # x
        ray = np.max(smallmaskx, (1, 2))
        indices = np.where(ray == 1)
        xmin = indices[0][0]
        xmax = indices[0][-1]
        # y
        ray = np.max(smallmaskx, (0, 2))
        indices = np.where(ray == 1)
        ymin = indices[0][0]
        ymax = indices[0][-1]
        # z
        ray = np.max(smallmaskx, (0, 1))
        if self.asymmetry:
            indices = np.where(ray == 1)
            zmin = indices[0][0]
            zmax = indices[0][-1]
        else:
            raise Exception("cant")
        return xmin, xmax + 1, ymin, ymax + 1, zmin, zmax + 1

Below are examples of the cropped voxelized mesh with the old implementation and the new implementation.

For my visualizations I used the following code:

def get_points_from_voxel(vox_model):
    xp, yp, zp = np.where(vox_model == 1)
    normalized_mid_point_xs = np.linspace(0, 1, vox_model.shape[0]) + 1 / (vox_model.shape[0] * 2)
    normalized_mid_point_ys = np.linspace(0, 1, vox_model.shape[1]) + 1 / (vox_model.shape[1] * 2)
    normalized_mid_point_zs = np.linspace(0, 1, vox_model.shape[2]) + 1 / (vox_model.shape[2] * 2)
    xp = normalized_mid_point_xs[xp]
    yp = normalized_mid_point_ys[yp]
    zp = normalized_mid_point_zs[zp]
    points = np.vstack([xp, yp, zp]).T
    return points


def get_bound_points_from_voxel(vox_model):
    xp = np.array([0,0,0,0,1,1,1,1]) * (vox_model.shape[0]-1)
    yp = np.array([0,0,1,1,0,0,1,1]) * (vox_model.shape[1]-1)
    zp = np.array([0,1,0,1,0,1,0,1]) * (vox_model.shape[2]-1)
    normalized_mid_point_xs = np.linspace(0, 1, vox_model.shape[0]) + 1 / (vox_model.shape[0] * 2)
    normalized_mid_point_ys = np.linspace(0, 1, vox_model.shape[1]) + 1 / (vox_model.shape[1] * 2)
    normalized_mid_point_zs = np.linspace(0, 1, vox_model.shape[2]) + 1 / (vox_model.shape[2] * 2)
    xp = normalized_mid_point_xs[xp]
    yp = normalized_mid_point_ys[yp]
    zp = normalized_mid_point_zs[zp]
    points = np.vstack([xp, yp, zp]).T
    return points


def PointCloud(points, colors=None):
    pcd = o3d.geometry.PointCloud()
    pcd.points = o3d.utility.Vector3dVector(points)
    if colors is not None:
        pcd.colors = o3d.utility.Vector3dVector(colors)
    return pcd


def interactive_plot(meshes):
    viewer = open3d.visualization.VisualizerWithKeyCallback()
    viewer.create_window()
    for mesh in meshes:
        viewer.add_geometry(mesh)
    viewer.run()
    viewer.destroy_window()


tmp_raw = get_vox_from_binvox_1over2(binvox_file).astype(np.uint8)
xmin1, xmax1, ymin1, ymax1, zmin1, zmax1 = dset.get_voxel_bbox(tmp_raw)
tmp_raw_cropped1 = crop_voxel(tmp_raw, xmin1, xmax1, ymin1, ymax1, zmin1, zmax1)

points_cropped1vox = get_points_from_voxel(tmp_raw_cropped1)
bound_points_cropped1vox = get_bound_points_from_voxel(tmp_raw_cropped1)
reds = np.array([[255,0,0]]*len(bound_points_pooled_tmp_raw1))
blues = np.array([[0,0,255]]*len(points_pooled_tmp_raw1))
interactive_plot([PointCloud(bound_points_cropped1vox, reds), PointCloud(points_cropped1vox, blues)])

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.