nperraud / download-celeba-hq Goto Github PK
View Code? Open in Web Editor NEWPython script to download the celebA-HQ dataset from google drive
Python script to download the celebA-HQ dataset from google drive
The above exception was raised in running download_celebA_HQ.py
Command:
python download_celebA_HQ.py ./img_align_celeba
Output log:
Deal with file: deltas00000.zip
[*] ./img_align_celeba/deltas00000.zip already exists
Traceback (most recent call last):
File "download_celebA_HQ.py", line 93, in
with zipfile.ZipFile(save_path) as zf:
File "/u/jliang/anaconda3/envs/tf_gpu/lib/python3.6/zipfile.py", line 1108, in init
self._RealGetContents()
File "/u/jliang/anaconda3/envs/tf_gpu/lib/python3.6/zipfile.py", line 1175, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
Given that this issue hasn't seemed to occur previously, is this an issue associated with the file itself? Or is it due to my setup?
I get stuck at the same error whether I try the Docker approach or follow the instructions myself.
PackagesNotFoundError: The following packages are not available from current channels:
- pillow==3.1.1
- jpeg=8d
Am I doing something wrong? Or does the script need to be udpated?
To get this to work on Windows 10 I had to do the following:
Create a new environement
conda activate celebaHQ
Install the packages
conda install -c anaconda jpeg=8d tqdm requests pillow==3.1.1 urllib3 numpy cryptography scipy
I had to add a if __name__ == '__main__'
: line just before num_workers.
if __name__ == '__main__':
num_workers = mp.cpu_count() - 1
print('Starting a pool with {} workers'.format(num_workers))
with mp.Pool(processes=num_workers) as pool:
pool.map(do_the_work, list(range(expected_dat)))
if len(glob.glob(os.path.join(delta_dir, '*.npy'))) != 30000:
raise ValueError('Expected to find {} npy files\n Something went wrong!'.format(30000))
# Remove the dat files
for filepath in glob.glob(os.path.join(delta_dir, '*.dat')):
os.remove(filepath)
print('All done! Congratulations!')
Script was working until this error:
Traceback (most recent call last):
File "/workspace/make_HQ_images.py", line 164, in pool.map(do_the_work, list(range(expected_dat)))
File "/opt/conda/lib/python3.5/multiprocessing/pool.py", line 266, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.5/multiprocessing/pool.py", line 644, in get raise self._value
OSError: 3145728 requested and 1671040 written
sum([1 for file in os.listdir(img_dir) if file[-4:] == '.jpg'])
could be simplified in a bit efficient way
from glob import glob
len(glob(os.path.join(img_dir,'*.jpg')))
conda install jpeg=8d tqdm requests pillow==3.1.1 urllib3 numpy cryptography scipy
results in dependency conflict.
conda install pillow==3.1.1 alone causes anaconda to demand switching to python2
doing
conda install -c conda-forge tqdm (first anaconda result after google 'conda tqdm') and same for pillow (without setting version) numpy, criptography, scipy,
and also comment out the the warnings works
I am using ubuntu 19
I ran the first two python files, than found that i didn't setup conda environment correctly and was installing onto the default env, so removed anaconda and reinstalled, than roughly followed the above procedure and it worked.
Dropbox is limited in term of download. Replace it with google drive in the script download_celebA.py
.
Some of it has already been done.
Hello, and thanks for your work! Could you tell me what the total size of the dataset is? I need to know on what kind of hard drive I should run the program. Maybe it would be a useful information to display somewhere on the README.
Hello and thanks for sharing your code! ๐
I've been browsing in the generated images and most of them are of high quality.
However, I found that images 70 and 2815 are not that great.
Image 70 - half of the face is cut
Image 2815 - there is an eye in the mouth
Is it the same for you? Is it the same from the original repo?(NIVIDIA)
Did you find other images with small flaws?
I want the celeba hq dataset for some task in which I just want 2000 images.
So I have downloaded the celeba dataset, but please tell me which delta should I download ( Because I have slow internet connection ) to convert it to HQ, and what will be the process
Hello, I am currently unable to use dropbox links in download_celebA.py file, are there any alternatives for those or how else I can get the dataset?
Thank you.
I was trying to run this with docker, it didn't work:
After clone:
$ docker build -t celeba_hq . && docker run -it -v $(pwd):/data celeba_hq
...
PackagesNotFoundError: The following packages are not available from current channels:
Current channels:
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
...
It successfully loaded Celeba and Celeba-HQ deltas, then it crashed because "image_list.txt" doesn't exist
Loading CelebA from ./celebA/Img/img_celeba Loading CelebA-HQ deltas from ./celebA-HQ Traceback (most recent call last): File "/workspace/make_HQ_images.py", line 45, in with open(os.path.join('image_list.txt'), 'rt') as file: FileNotFoundError: [Errno 2] No such file or directory: 'image_list.txt'
As in the title, can't get the data thru codes
urllib.error.HTTPError: HTTP Error 429: Too Many Requests
Looking forward to ur response!
I've got the following exception during the downloading
if os.name != 'posix':
raise NotImplementedError('Only Linux and Mac OS X support .7z '
It downloads and unpacks the first 200K images easily but when it comes to archives like
img_celeba.7z.001
it raises the exception.
What should I do with Windows 10 then? 7z is already installed.
Command:
cd download-celebA-HQ/ && python download_celebA.py ./
Output:
Downloading img_align_celeba.zip to ./celebA/img_align_celeba.zip
./celebA/img_align_celeba.zip: 1.00B [00:00, 847B/s]
Done!
Check SHA1 ./celebA/img_align_celeba.zip
Traceback (most recent call last):
File "download_celebA.py", line 219, in
download_celabA(dataset_dir)
File "download_celebA.py", line 183, in download_celabA
filepaths = download_and_check(_ALIGNED_IMGS_DRIVE, dataset_dir)
File "download_celebA.py", line 106, in download_and_check
raise RuntimeError('Checksum mismatch for %s.' % save_path)
RuntimeError: Checksum mismatch for ./celebA/img_align_celeba.zip.
Hi,
thank you for this code !
I have an error in the make_celeba_hQ.py
Traceback (most recent call last):
File "/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.5.4/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.5.4/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "make_HQ_images.py", line 157, in do_the_work
img = process_func(img_num)
File "make_HQ_images.py", line 115, in process_func
img = np.pad(np.float32(img), ((pad[1], pad[3]), (pad[0], pad[2]), (0, 0)), 'reflect')
TypeError: float() argument must be a string or a number, not 'Image'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "make_HQ_images.py", line 167, in
pool.map(do_the_work, list(range(expected_dat)))
File "/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.5.4/lib/python3.5/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.5.4/lib/python3.5/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: float() argument must be a string or a number, not 'Image'
Do you know where it can come from??
Thanks in advance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.