tencentyun / coscmd Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
希望delete操作能增加-y
免确认参数
我在通过命令行匹配结果批量删除时,没法执行
请问,创建两个桶后,如何配置并查看对应桶内容?
coscmd-1.8.6.20
coscmd config -a $SECRET_ID -s $SECRET_KEY -b $BUCKET -e cos.accelerate.myqcloud.com -m 30
'code': 'ActionAccelerateNotSupported', 'message': 'The action is not supported by the accelerate host.'
the version of pytz
is 2017.2, a very old one
Many projects have updated it to new ones, eg. 2018.5, just like apache-beam
So if our project import pytz
and apache-beam at the same time, the independence will be incompatible.
So could you update the version of pytz?
Thanks
coscmd --version
检查版本因为返回代码是非0造成ci中断#!/usr/bin/env bash
set -ex
coscmd --version
echo $?
+ coscmd --version
coscmd 1.8.5.11
+ echo 255
希望可以通过 python 代码来实现 cos 操作
We are using coscmd to list and download files for COS bucket auth by VPC id, the config in .cos.conf
is:
anonymous = True
but from 1.8.5.12, list would return:
<?xml version='1.0' encoding='utf-8' ?>
<Error>
<Code>InvalidAccessKeyId</Code>
<Message>The Access Key Id you provided does not exist in our records</Message>
<Resource>XXXXXXXXXXX</Resource>
<RequestId>NWMwNjQ4YzFfNjNhYTk0MGFfMThhYl8yYmY3ZTk=</RequestId>
<TraceId>OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODc0OWRkZjk0ZDM1NmI1M2E2MTRlY2MzZDhmNmI5MWI1OTBjYzE2MjAxN2M1MzJiOTdkZjMxMDVlYTZjN2FiMmI0ZTI5NDE1MDVlZjUxMTI1NGUwOTAzYzUxYTRmMzdlNjE=</TraceId>
</Error>
by the way, download
works fine.
[executionStep begins at 2023-05-12 10:08:13]
[INFO] PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/alibaba/install/python-3.8.5/bin/python3
/root/entry.sh: line 53: export: `,': not a valid identifier
/root/entry.sh: line 53: export: `,': not a valid identifier
�[92m[10:08:14] [SUCCESS]
�[1m[10:08:14] [INFO] /root/workspace/blog
�[1m[10:08:14] [INFO]
[10:08:14] [INFO]PIPELINE_ID=2026193
[10:08:14] [INFO]PIPELINE_NAME=blog
[10:08:14] [INFO]BUILD_NUMBER=88
[10:08:14] [INFO]EMPLOYEE_ID=61245fd2cd146a5e9cabbe6f
[10:08:14] [INFO]WORK_SPACE=/root/workspace
[10:08:14] [INFO]PROJECT_DIR=/root/workspace/blog
[10:08:14] [INFO]BUILD_JOB_ID=126009117
[10:08:14] [INFO]执行用户命令
�[1;36m[10:08:14] [User Command] + pip install coscmd
[10:08:15] Collecting coscmd
[10:08:15] Using cached coscmd-1.8.6.31-py3-none-any.whl
[10:08:16] Collecting datetime>=4.2
[10:08:16] Using cached DateTime-5.1-py3-none-any.whl (52 kB)
[10:08:16] Collecting pyyaml>=3
[10:08:16] Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
[10:08:16] Collecting prettytable>=0.7
[10:08:16] Using cached prettytable-3.7.0-py3-none-any.whl (27 kB)
[10:08:16] Collecting tqdm>=4.15.0
[10:08:16] Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
[10:08:16] Collecting requests>=2.8
[10:08:16] Downloading requests-2.30.0-py3-none-any.whl (62 kB)
[10:08:16] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.5/62.5 kB 5.9 MB/s eta 0:00:00
[10:08:16] Collecting argparse>=1.4.0
[10:08:16] Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB)
[10:08:16] Collecting pytz>=2017.2
[10:08:16] Using cached pytz-2023.3-py2.py3-none-any.whl (502 kB)
[10:08:16] Requirement already satisfied: six>=1.11.0 in /usr/local/lib/python3.8/site-packages (from coscmd) (1.15.0)
[10:08:17] Collecting cos-python-sdk-v5>=1.9.8
[10:08:17] Downloading cos-python-sdk-v5-1.9.24.tar.gz (96 kB)
[10:08:17] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.3/96.3 kB 27.1 MB/s eta 0:00:00
[10:08:17] Preparing metadata (setup.py): started
[10:08:17] Preparing metadata (setup.py): finished with status 'done'
[10:08:17] Collecting xmltodict
[10:08:17] Using cached xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)
[10:08:17] Collecting crcmod
[10:08:17] Using cached crcmod-1.7-cp38-cp38-linux_x86_64.whl
[10:08:17] Collecting pycryptodome
[10:08:17] Using cached pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
[10:08:18] Collecting zope.interface
[10:08:18] Using cached zope.interface-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (249 kB)
[10:08:18] Collecting wcwidth
[10:08:18] Using cached wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
[10:08:18] Collecting certifi>=2017.4.17
[10:08:18] Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
[10:08:18] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.0/157.0 kB 42.9 MB/s eta 0:00:00
[10:08:18] Collecting charset-normalizer<4,>=2
[10:08:18] Using cached charset_normalizer-3.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (195 kB)
[10:08:18] Collecting idna<4,>=2.5
[10:08:18] Using cached idna-3.4-py3-none-any.whl (61 kB)
[10:08:18] Collecting urllib3<3,>=1.21.1
[10:08:18] Downloading urllib3-2.0.2-py3-none-any.whl (123 kB)
[10:08:18] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.2/123.2 kB 34.9 MB/s eta 0:00:00
[10:08:18] Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from zope.interface->datetime>=4.2->coscmd) (47.1.0)
[10:08:18] Building wheels for collected packages: cos-python-sdk-v5
[10:08:18] Building wheel for cos-python-sdk-v5 (setup.py): started
[10:08:19] Building wheel for cos-python-sdk-v5 (setup.py): finished with status 'done'
[10:08:19] Created wheel for cos-python-sdk-v5: filename=cos_python_sdk_v5-1.9.24-py3-none-any.whl size=89922 sha256=e93675113caf37b42274af3a4efd8b28eb1c5f76c7ec2001fb2a67fb98196d78
[10:08:19] Stored in directory: /root/.cache/pip/wheels/d5/19/5a/ae9a72439af360b20f29096db8b1a160369ffb33e666519c0f
[10:08:19] Successfully built cos-python-sdk-v5
[10:08:19] Installing collected packages: wcwidth, pytz, crcmod, argparse, zope.interface, xmltodict, urllib3, tqdm, pyyaml, pycryptodome, prettytable, idna, charset-normalizer, certifi, requests, datetime, cos-python-sdk-v5, coscmd
[10:08:20] Successfully installed argparse-1.4.0 certifi-2023.5.7 charset-normalizer-3.1.0 cos-python-sdk-v5-1.9.24 coscmd-1.8.6.31 crcmod-1.7 datetime-5.1 idna-3.4 prettytable-3.7.0 pycryptodome-3.17 pytz-2023.3 pyyaml-6.0 requests-2.30.0 tqdm-4.65.0 urllib3-2.0.2 wcwidth-0.2.6 xmltodict-0.13.0 zope.interface-6.0
[10:08:20] WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[10:08:21]
[10:08:21] [notice] A new release of pip is available: 23.0.1 -> 23.1.2
[10:08:21] [notice] To update, run: pip install --upgrade pip
�[1;36m[10:08:21] [User Command] + coscmd config -a **** -s **** -b **** -r ap-shanghai
[10:08:21] Traceback (most recent call last):
[10:08:21] File "/usr/local/bin/coscmd", line 5, in <module>
[10:08:21] from coscmd.cos_cmd import _main
[10:08:21] File "/usr/local/lib/python3.8/site-packages/coscmd/cos_cmd.py", line 10, in <module>
[10:08:21] import requests
[10:08:21] File "/usr/local/lib/python3.8/site-packages/requests/__init__.py", line 43, in <module>
[10:08:21] import urllib3
[10:08:21] File "/usr/local/lib/python3.8/site-packages/urllib3/__init__.py", line 38, in <module>
[10:08:21] raise ImportError(
[10:08:21] ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2e 3 Dec 2015. See: https://github.com/urllib3/urllib3/issues/2168
�[91m[10:08:21] [ERROR] BUILD ERROR
�[91m[10:08:21] [ERROR] 126009117
�[91m[10:08:21] [ERROR] 1
您好:
申请.cos.log日志文件上限扩容至1G,谢谢!
存储桶名称:st-rm-1255501786
所属地域 广州 (**)(ap-guangzhou)
上传文件的时候可以显示上传文件的总进度
When i try to download images with coscmd, it usually stuck. Then I add some output in the progress, it turns out that the program always stop at self._pbar.update(chunk_len)
. So I remove the tqdm, and it fixed.
I guess because the tqdm is not thread-safe, when multi-thread runs.
能否支持优先在当前目录找.cos.conf
:
find conf -> local .cos.conf
-> global .cos.conf
yum -y install wget
wget https://bootstrap.pypa.io/2.6/get-pip.py
python get-pip.py
pip install coscmd
没有补全,很难用
coscmd/cos_client.py:471
Traceback (most recent call last):
File "/Users/happyelements/miniconda3/envs/heai/bin/coscmd", line 7, in
from coscmd.cos_cmd import _main
File "/Users/happyelements/miniconda3/envs/heai/lib/python3.6/site-packages/coscmd/cos_cmd.py", line 2, in
from cos_client import CosConfig, CosS3Client
ModuleNotFoundError: No module named 'cos_client'
python version is 3.6.5
from cos_client import CosConfig, CosS3Client
should be like:
from coscmd.cos_client import CosConfig, CosS3Client
Hi. I lead the security team at Evernote.
The default value for CosConfig(schema=http)
is insecure. You are encouraging Qcloud users to transfer files with no transport encryption. QCloud customers expect data to be secure by default, so you need to change the default value to schema=https
.
-Rich Tener
Currently, a bad network connection can ruin the whole upload, even if 99% was uploaded.
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
'ascii' codec can't encode characters in position 6-83: ordinal not in range(128)
[root@Server ~]# ll
total 232252
-rw-r--r-- 1 root root 81 Mar 7 16:28 。cos.log
drwxr-xr-x 7 root root 4096 Mar 7 15:29 lnmp
-rw-r--r-- 1 root root 237578075 Mar 6 09:19 lnmp-full.tar.gz
[root@Server ~]# cat 。cos.log
2018-03-07 16:28:41,856 - [INFO]: Created configuration file in /root/.cos.conf
。cos.log
中文句号耶!
Traceback (most recent call last):
File "/usr/local/bin/coscmd", line 5, in
from coscmd.cos_cmd import _main
File "/usr/local/lib/python2.7/dist-packages/coscmd/cos_cmd.py", line 18, in
from cos_client import CoscmdConfig, CosS3Client
File "/usr/local/lib/python2.7/dist-packages/coscmd/cos_client.py", line 18, in
import yaml
File "/usr/local/lib/python2.7/dist-packages/yaml/init.py", line 362
class YAMLObject(metaclass=YAMLObjectMetaclass):
^
SyntaxError: invalid syntax
加一个命令就完美了 coscmd list all Bucket https://cloud.tencent.com/document/product/436/8291
这玩意 比 狗屁的网页版 高级多了
Download cos://rein-houthooft/exp_data/lf_dict_501_310.json => /home/ubuntu/data/exp_data/lf_dict_501_310.json
Downloading /home/ubuntu/data/exp_data/lf_dict_501_310.json
0%| | 0.00/1.59M [00:00<?, ?B/s][Errno 17] File exists: '/home/ubuntu/data/exp_data'
50%|##### | 793K/1.59M [00:00<00:00, 6.88MB/s]
90%|######### | 1.43M/1.59M [00:00<00:00, 10.1MB/s]
Completing mget
[Errno 2] No such file or directory: u'/home/ubuntu/data/exp_data/lf_dict_501_310.json_5'
Mget file failure
Getting this error sometimes, could you investigate?
我每次上传一堆图片,都要在上传完之后,再去控制台设置header
怎样才能在批量上传的时候统一设置呢
错误信息
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 765, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/site-packages/coscmd/cos_cmd.py", line 564, in command_thread
res = args.func(args)
File "/usr/lib/python2.7/site-packages/coscmd/cos_cmd.py", line 265, in list
if Interface.list_objects(cos_path=args.cos_path, _recursive=args.recursive, _all=args.all, _num=args.num, _human=args.human):
File "/usr/lib/python2.7/site-packages/coscmd/cos_client.py", line 892, in list_objects
rt = self._session.get(url=url, auth=CosS3Auth(self._conf._secret_id, self._conf._secret_key))
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 521, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPSConnectionPool(host='-mybucket.cos.ap-chengdu.myqcloud.com', port=443): Max retries exceeded with url: /?prefix=&marker=&delimiter=/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x1fa7650>: Failed to establish a new connection: [Errno -2] Name or service not known',))
配置文件
secret_id = ak
secret_key = sk
bucket = mybucket
region = ap-chengdu
max_thread = 5
part_size = 1
问题版本coscmd 1.8.2.10
pythonPython 2.7.5
PIPpip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
coscmd -b testbucket upload aaa.txt aaa.txt
您好,是否支持下 bash-completion 呢, 在某些堡垒机后使用,不能复制张贴,打路径好累。
希望支持
Version: coscmd 1.8.4.16
Cmd: coscmd -b heai-seed-rec-service-1256590953 download -rs train_log train_log
Error:
Download cos://heai-seed-rec-service/train_log/neural_bandit_v1/0/531/327/train.log => train_log/neural_bandit_v1/0/531/327/train.log
Downloading train_log/neural_bandit_v1/0/531/327/train.log
0%| | 0.00/12.2M [00:00<?, ?B/s][Errno 17] File exists: 'train_log/neural_bandit_v1/0/531/327'
90%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 11.0M/12.2M [00:07<00:00, 1.42MB/s]
Completing mget
[Errno 2] No such file or directory: u'train_log/neural_bandit_v1/0/531/327/train.log_4'
Mget file failure
Usually, only a small portion of files are changed and it is time-consuming to upload all files in advance.
要安装这个先安装pip
要安装pip又会报新的错
新的错网上有人让安装一个 distribute_setup
经过从大量多年以前的过期文章中筛选
终于找到方法安装好pip,当然中间会有更多报错
输入 pip install coscmd
又有新的报错
Traceback (most recent call last):
File "C:\Python27\Scripts\pip-script.py", line 11, in
load_entry_point('pip==9.0.1', 'console_scripts', 'pip')()
File "C:\Python27\lib\site-packages\distribute-0.6.27-py2.7.egg\pkg_resources.py", line 337, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Python27\lib\site-packages\distribute-0.6.27-py2.7.egg\pkg_resources.py", line 2307, in load_entry_point
return ep.load()
File "C:\Python27\lib\site-packages\distribute-0.6.27-py2.7.egg\pkg_resources.py", line 2017, in load
entry = import(self.module_name, globals(),globals(), ['name'])
File "C:\Python27\lib\site-packages\pip-9.0.1-py2.7.egg\pip_init_.py", line 28, in
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "C:\Python27\lib\site-packages\pip-9.0.1-py2.7.egg\pip\vcs\mercurial.py", line 9, in
from pip.download import path_to_url
File "C:\Python27\lib\site-packages\pip-9.0.1-py2.7.egg\pip\download.py", line 37, in
from pip.utils.ui import DownloadProgressBar, DownloadProgressSpinner
File "C:\Python27\lib\site-packages\pip-9.0.1-py2.7.egg\pip\utils\ui.py", line 57, in
_BaseBar = _select_progress_class(IncrementalBar, Bar)
File "C:\Python27\lib\site-packages\pip-9.0.1-py2.7.egg\pip\utils\ui.py", line 50, in _select_progress_class
six.text_type().join(characters).encode(encoding)
LookupError: unknown encoding: cp65001
总之想用这个,至少涉及到pip easy_install pkg_resources等一大堆东西(目前记得这几个词),如果想解决新的报错,又要安装新的依赖,而且无法预期到底还需要多少... ...
coscmd delete -rf backup/2018-08-28
Delete the remaining files again
The directory does not exist
删除不了
[root@VM_0_5_centos ~]# pip -V
pip 9.0.0 from /usr/lib/python2.7/site-packages (python 2.7)
error message:
[root@VM_0_5_centos ~]# pip install coscmd
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 272, in run
with self._build_session(options) as session:
File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 72, in _build_session
insecure_hosts=options.trusted_hosts,
File "/usr/lib/python2.7/site-packages/pip/download.py", line 329, in __init__
self.headers["User-Agent"] = user_agent()
File "/usr/lib/python2.7/site-packages/pip/download.py", line 93, in user_agent
from pip._vendor import distro
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 1051, in <module>
_distro = LinuxDistribution()
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 594, in __init__
if include_lsb else {}
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 922, in _lsb_release_info
stdout, stderr = stdout.decode('ascii'), stderr.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 22: ordinal not in range(128)
Traceback (most recent call last):
File "/usr/bin/pip", line 8, in <module>
sys.exit(main())
File "/usr/lib/python2.7/site-packages/pip/__init__.py", line 233, in main
return command.main(cmd_args)
File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 251, in main
timeout=min(5, options.timeout)) as session:
File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 72, in _build_session
insecure_hosts=options.trusted_hosts,
File "/usr/lib/python2.7/site-packages/pip/download.py", line 329, in __init__
self.headers["User-Agent"] = user_agent()
File "/usr/lib/python2.7/site-packages/pip/download.py", line 93, in user_agent
from pip._vendor import distro
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 1051, in <module>
_distro = LinuxDistribution()
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 594, in __init__
if include_lsb else {}
File "/usr/lib/python2.7/site-packages/pip/_vendor/distro.py", line 922, in _lsb_release_info
stdout, stderr = stdout.decode('ascii'), stderr.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 22: ordinal not in range(128)
方便在 Debian 系发行版下直接安装
比如
Download cos://backup/cdb/bbs20180807_v2.utf8 => .
Downloading .
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.38G/3.38G [01:15<00:00, 44.7MB/s]
Completing mget
[Errno 21] Is a directory: u'.'
Mget file failure
这样一个命令, 我都不知道错在哪了。
据文档:
使用-s或者--sync参数,可以在下载文件夹时跳过本地已存在的相同文件(前提是下载文件夹是通过COSCMD的upload接口上传的,文件携带有 x-cos-meta-md5头部)
那么我是否可以,使用SDK上传文件,手动带上 x-cos-meta-md5 的头部。这样的话,再使用 coscmd 工具时, 能跳过本地已下载的文件么?
google/leveldb
如题:coscmd能否支持python3+环境?
cos golang版本也可以有?
[root@217 home]# coscmd upload /home/123.mp4 123.mp4
Upload /home/123.mp4 => cos:///123.mp4
[root@217 home]#
也没有报错
Executing coscmd download -r exp_log /tmp
downloads the COS folder exp_log
into the relative folder tmp
rather than the absolute folder /tmp
.
你好,我想使用coscmd工具上传多个文件到指定桶的根目录下,请问是否支持?
现在使用 -r 指令上传文件夹,但会在根目录下生成同名文件夹,不符合我的需求。
coscmd upload 想忽略一些目录应该如何实现?
文档页面 中,有一些使用示例:
coscmd download -r <cospath> <localpath> //命令格式
coscmd download -r /home/aaa/ bbb/aaa //操作示例
coscmd download -r /home/aaa/ bbb/ //操作示例
coscmd download -rf / bbb/aaa //覆盖下载当前bucket根目录下所有的文件
coscmd download -rs / bbb/aaa //同步下载当前bucket根目录下所有的文件,跳过md5校验相同的文件
coscmd download -rs / bbb/aaa --ignore *.txt,*.doc //忽略.txt和.doc的后缀文件
建议不要用 //
作为注释符,因为 bash 里面是 #
为注释符,用户复制整行到终端运行会报错,也显得不够专业。同时建议另起一行表示注释:
# 命令格式
coscmd download -r <cospath> <localpath>
# 覆盖下载当前bucket根目录下所有的文件
coscmd download -rf / bbb/aaa
mc 和 s3cmd 都支持目录不带后缀 /,加很别扭
我们的应用场景是,使用coscmd下载大量的文件,难免在个别时候会下载到残损文件。如果每个文件都去自己校验,会非常耗时。
wget https://github.com/tencentyun/coscmd/releases/download/V1.8.4.17/coscmd-v1.8.4.17-py2.tar.gz
tar xzf coscmd-v1.8.4.17-py2.tar.gz
cd coscmd-1.8.4.17
[root@217 coscmd-1.8.4.17]# ./setup.py
-bash: ./setup.py: 权限不够
[root@217 coscmd-1.8.4.17]# chmod +x setup.py
[root@217 coscmd-1.8.4.17]# ls
coscmd coscmd.egg-info MANIFEST.in PKG-INFO README.md requirements.txt setup.cfg setup.py
[root@217 coscmd-1.8.4.17]# ./setup.py
./setup.py: line 1: from: command not found
./setup.py: line 2: from: command not found
./setup.py: line 3: from: command not found
: command not found
: command not found
./setup.py: line 6: syntax error near unexpected token (' '/setup.py: line 6:
def requirements():
[root@217 coscmd-1.8.4.17]#
使用--skipmd5参数跳过本地验证直接上传,服务器还会记录x-cos-meta-md5头部值吗?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.