style_transfer_3d's Introduction

2D-to-3D style transfer using Neural Renderer

This is the code for 2D-to-3D style transfer in the paper Neural 3D Mesh Renderer (CVPR 2018) by Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada.

Related repositories:

Neural Renderer

Installation

# install neural_renderer
git clone https://github.com/hiroharu-kato/neural_renderer.git
cd neural_renderer
python setup.py install --user
# or, sudo python setup.py install

Example

bash ./examples/run.sh

Citation

@inproceedings{kato2018renderer
    title={Neural 3D Mesh Renderer},
    author={Kato, Hiroharu and Ushiku, Yoshitaka and Harada, Tatsuya},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}

style_transfer_3d's People

Contributors

Stargazers

Watchers

style_transfer_3d's Issues

Code compatibility without GPU

I am working on AWS Deep Learning instance without GPU.

Is this code compatible with only GPU? Why this part of code is important ? Can this function be replaced with CPU compatible VGG trained-model and feature extraction? Let me know the feasibility and challenges, I can raise a PR for the same.

Failure to use save_obj() function to save mesh model with textures

Hi, I tried to save style transferred mesh using save_obj() function like following.

...

# draw object
model.renderer.background_color = (1, 1, 1)
loop = tqdm.tqdm(range(0, 360, 4))
for num, azimuth in enumerate(loop):
    loop.set_description('Drawing')
    model.renderer.eye = neural_renderer.get_points_from_angles(2.732, 30, azimuth)
    images = model.renderer.render(*model.mesh.get_batch(1))
    image = images.data.get()[0].transpose((1, 2, 0))
    scipy.misc.toimage(image, cmin=0, cmax=1).save('%s/_tmp_%04d.png' % (directory_output, num))
make_gif(directory_output, args.filename_output)

# save model using save_obj() function
vertices = model.mesh.vertices.data
faces = model.mesh.faces
textures = model.mesh.textures.data
neural_renderer.save_obj("output.obj", vertices, faces, textures=textures)

But outputted texture is not same as renderred image.