yushu9 / kosmosfs Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/kosmosfs
License: Apache License 2.0
Automatically exported from code.google.com/p/kosmosfs
License: Apache License 2.0
operation system: centos 6.2 64-bit
kfs version: kfs-0.5
[root@gifer scripts]# python kfssetup.py -f machines.cfg -b ../build -w
../webui -s
kfssetup.py:28: DeprecationWarning: The popen2 module is deprecated. Use the
subprocess module.
import socket,threading,popen2
kfssetup.py:29: DeprecationWarning: the md5 module is deprecated; use hashlib
instead
import md5
cp: 略过目录"./scripts"
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: metaserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping
gtar: 从成员名中删除开头的“/”
[localhost]: chunkserver exists...skipping
[root@gifer tools]# ./kfsping -c -s localhost -p 20000
Meta-server: 0
Total-space: 0 (MB)
Used-space: 0 (MB)
is setup success?
if no what should i do ?
thank you!
Original issue reported on code.google.com by [email protected]
on 21 May 2012 at 8:38
What steps will reproduce the problem?
1. kfs metaserver chunkserver deployed on a machine
2. when I called the function Write() on another machine this occured.
3. AllocChunk()failed sizechunk(fd)<0
What version of the product are you using? On what operating system?
kfs-0.5 centos
Please provide any additional information below.
I do not know the reason
Thanks advance!
Original issue reported on code.google.com by [email protected]
on 7 Jan 2014 at 10:03
I am initiating a new project which is likely fit into KFS. We have a huge
number of distributed chunk servers. I am wondering how KFS to locate a chunk.
Does it have an interface to memcache or database? Please let me know a little
bit more details. Thank you.
Original issue reported on code.google.com by [email protected]
on 4 Dec 2012 at 2:22
operation system: centos 6.2 64-bit
kfs version: kfs-0.5
when I compile the KfsSample_main.cc file, it has same erros:
make all
Building file: ../kfstest.cpp
Invoking: GCC C++ Compiler
g++ -I/home/altair/code/kfs/include/kfs -O0 -g3 -Wall -c -fmessage-length=0
-MMD -MP -MF"kfstest.d" -MT"kfstest.d" -o"kfstest.o" "../kfstest.cpp"
Finished building: ../kfstest.cpp
Building target: KfsTestSample
Invoking: GCC C++ Linker
g++ -o"KfsTestSample" ./kfstest.o
/home/altair/code/kfs/lib/static/libkfsClient.a
/home/altair/code/kfs/lib/static/libkfsCommon.a
/home/altair/code/kfs/lib/static/libkfsEmulator.a
/home/altair/code/kfs/lib/static/libkfsIO.a
/home/altair/code/kfs/lib/static/libkfsMeta.a
/home/altair/code/kfs/lib/static/libqcdio.a
/home/altair/code/kfs/lib/static/libtools.a -lpthread -lcrypto -lrt
./kfstest.o: In function `main':
/home/altair/workspace/KfsTestSample/Debug/../kfstest.cpp:193: undefined
reference to `KFS::KfsClient::Seek(int, long)'
collect2: ld returned 1 exit status
make: *** [KfsTestSample] Error 1
the code that I use seek() is:
gKfsClient->Seek(fd, (off_t)40);
the complier force (off_t)40 convert to long int;and can't find
Seek(int,off_t) define.Has I done something wrong?Hope to get some help,thank!
Original issue reported on code.google.com by [email protected]
on 29 Jun 2012 at 6:16
What steps will reproduce the problem?
1. open file in O_RDWR
2. Write file
3. Read file (error)
What is the expected output? What do you see instead?
INFO - (KfsClient.cc:3317) Server says lease on chunk ** is busy...waiting
What version of the product are you using? On what operating system?
KFS-0.5, linux (ubuntu)
Please provide any additional information below.
why will this error happen?
how to deal with it? see the attachment, modified from the kfsSample.cpp
thanks vary much...
Original issue reported on code.google.com by [email protected]
on 14 Aug 2012 at 10:21
Attachments:
KFS 0.5/Debian squeeze/mipsel/2.6.32-5-4kc-malta/PAGESIZE=4096
metaserver and kfsping exited with the error message as follows:
TryToStart Invalid argument 22Aborted
In src/cc/qcdio/qcthread.cpp:106
if (inStackSize > 0 && (theErr = pthread_attr_setstacksize(
&theStackSizeAttr, inStackSize)) != 0)
pthread_attr_setstacksize failed when inStackSize is 32KB. I wrote a small test
program:
pthread_attr_getstacksize(&a, &n);
printf("%d\n", n);
for (n = 16384; n <= 256 * 1024; n *= 2)
printf("%d\t%d\n", n, pthread_attr_setstacksize(&a, n));
On x86:
8388608
16384 0
32768 0
65536 0
131072 0
262144 0
On the machine:
8388608
16384 22
32768 22
65536 22
131072 0
262144 0
I knew that's not KFS' fault, but 128KB seems to be more adaptable.
`man pthread_attr_setstacksize`:
pthread_attr_setstacksize() can fail with the following error:
EINVAL The stack size is less than PTHREAD_STACK_MIN (16384) bytes.
On some systems, pthread_attr_setstacksize() can fail with the error EINVAL if
stacksize is not a multiple of the system page size.
Original issue reported on code.google.com by [email protected]
on 25 Jul 2012 at 10:45
I tried to use KFS-0.5 with Hadoop-1.0.3. When I ran a job,
I got the error information:
ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop
cause:java.io.IOException: The ownership/permissions on the staging directory
kfs://mars:20000/tmp/hadoop-hadoop/mapred/staging/hadoop/.staging is not as
expected. It is owned by and permissions are rwxrwxrwx. The directory must be
owned by the submitter hadoop or by hadoop and permissions must be rwx------
`bin/hadoop fs -ls` shows all files have the rwxrwxrwx permission.
kfsshell doesn't list permission.
class KfsClient doesn't seem to have something like chmod or chown.
Does KFS not support file permission?
Thanks,
Gary
Original issue reported on code.google.com by [email protected]
on 31 Jul 2012 at 1:49
Great project first of all! Respect to you kosmos ppl.
But any chance that we(users) can have more broad and comprehensible
documenation on binaries and scripts?
For example:
$ ./telemetry_server --help
./telemetry_server: invalid option -- '-'
./telemetry_server: invalid option -- 'e'
Usage: ./telemetry_server -m <machines file> -l <log fn>
Program tells me what usage should I use, but it doesn't help user to
understood, actually, what is "telemetry_server" by itself :)
This kind of problem affects all binaries - lack of description.
Or this one:
$ ./rebalancer --help
Usage: ./rebalancer <network defn> <data dir> <outputFn>
Any chance to know what is <network defn> ??
Thank you in advance.
In anycase, great job!
Original issue reported on code.google.com by [email protected]
on 7 Jun 2012 at 10:50
1.there are bugs In Kfs_fuse_main.cc
2.when client call write(nfd,buf,nwrite) function to write data into dfs,
why the return size is even bigger than nwrite size .Lt is absurd. l print the
log
and find that it returns the total size that have been written to dfs.
cause l call {int res=write(nfd,buf,4096)} for serval times.
and res is 4096,8192...
l thought maybe it is because of the default buffer in client.
so l call function SetIoBufferSize(nfd,0),but it doesn't work.
what a fun it is .the write function returns some size bigger than 4096,and the
file is even closed in client,but the file size in dfs is still 4096 big.
so many bugs in the project
Original issue reported on code.google.com by [email protected]
on 13 May 2012 at 12:08
I finished compiling and deploying KFS. And made the servers start. I want to
have a test on client programming. But I get link errors like these:
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::re_detail::cpp_regex_traits_implementation<char>::transform_primary(char
const*, char const*) const'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::re_detail::get_default_error_string(boost::regex_constants::error_type)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::re_detail::cpp_regex_traits_implementation<char>::transform(char
const*, char const*) const'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::re_detail::put_mem_block(void*)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::basic_regex<char, boost::regex_traits<char,
boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*, unsigned
int)'
/home/ggicci/kfs/build/lib/../lib/libkfsIO.so: undefined reference to
`boost::re_detail::raise_runtime_error(std::runtime_error const&)'
I give a snapshot of my code and make output
[here](https://gist.github.com/ggicci/10342928).
Original issue reported on code.google.com by [email protected]
on 10 Apr 2014 at 12:32
After deploying the kfs, i can' use the tools. I tried the kfsping command
(kfsping -m -s kfs1 -p 20000) and he drops the following message:
-bash: kfsping: command not found
Same with cptokfs...
I'm using KFS 0.5 version on Linux Debian Squeeze 32-bit(kernel version
2.6.32-5-686)
Please help.
Original issue reported on code.google.com by [email protected]
on 5 Apr 2012 at 4:53
What steps will reproduce the problem?
1. After doing gmake when I'm building C++ code
What is the expected output? What do you see instead?
Many errors during gmake commmand. When files treat the contents of a
xfs/linux.h (with an include command) some function in this file don't work
correctly (cannot convert, was not declared in this scope).
What version of the product are you using? On what operating system?
I'm trying to build kfs-0.5 on Centos6.3 64bit with xfsprogs.x86_64 and
xfsprogs-devel.x86_64 packs
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 12 Jul 2012 at 1:56
Attachments:
Are there any design documents for the purpose of code analyzing
such as uml graph,anything will be fine
Original issue reported on code.google.com by [email protected]
on 11 Aug 2011 at 10:54
Complication problems occured when Linking CXX executable filelister:
CMakeFiles/filelister.dir/filelister_main.o: In function
'__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to
'boost::system:;generic_category()'
CMakeFiles/filelister.dir/filelister_main.o: In function 'thread_exception':
/usr/local/include/boost/thread/exceptions.hpp:51 undefined reference to
'boost::system:;generic_category()'
libkfsMeta.a(checkpoint.o):In function
'__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to
'boost::system:;generic_category()'
libkfsMeta.a(kfsops.o):In function '__static_initialization_and_destruction_0':
/usr/local/include/boost/system/error_code.hpp:214: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:215: undefined reference to
'boost::system:;generic_category()'
/usr/local/include/boost/system/error_code.hpp:216: undefined reference to
'boost::system:;generic_category()'
....
....
....
....
collect2: ld 返回1
gmake[2]: *** [src/cc/meta/filelister] 错误 1
gmake[1]: *** [src/cc/meta/CMakeFiles/cpfromkfs.dir/all] 错误 2
gmake: *** [all] 错误 2
Original issue reported on code.google.com by [email protected]
on 25 May 2013 at 9:31
Error while mounting KFS directory using FUSE ?
./kfs_fuse /media/hugedisk/KFS/mountkfs -f
kfs_fuse: /usr/include/boost/smart_ptr/shared_ptr.hpp:418: T*
boost::shared_ptr< <template-parameter-1-1> >::operator->() const [with T =
KFS::KfsClient]: Assertion `px != 0' failed.
Aborted
What version of the product are you using? On what operating system?
Im using kosmosfs .5 version on Ubuntu 10.04 64 bit ..
Original issue reported on code.google.com by [email protected]
on 10 Jun 2011 at 12:33
bug in file
kfs-0.5/src/cc/libkfsClient/WriteAppender.cc
http://code.google.com/p/kosmosfs/source/browse/trunk/src/cc/libkfsClient/WriteA
ppender.cc
void SpaceRelease()
{
if (mSpaceAvailable <= 0) {
StartAppend();
return;
}
Reset(mSpaceReleaseOp);
mSpaceReleaseOp.chunkId = mAllocOp.chunkId;
//! bug: should mAllocOp.chunkVersion; NOT ,
mSpaceReleaseOp.chunkVersion = mAllocOp.chunkVersion; //! , --> ;
mSpaceReleaseOp.writeInfo = mWriteIds;
mSpaceReleaseOp.numBytes = size_t(mSpaceAvailable);
Enqueue(mSpaceReleaseOp);
}
chenbk
Original issue reported on code.google.com by [email protected]
on 30 Nov 2011 at 3:10
Linking CXX executable cpfromkfs
../qcdio/libqcdio.a(qcmutex.o): In function `GetAbsTimeout':
/home/septem/kfs/kfs-0.5/src/cc/qcdio/qcmutex.cpp:40: undefined reference to
`clock_gettime'
/home/septem/kfs/kfs-0.5/src/cc/qcdio/qcmutex.cpp:40: undefined reference to
`clock_gettime'
collect2: ld returned 1 exit status
make[2]: *** [src/cc/tools/cpfromkfs] Error 1
make[1]: *** [src/cc/tools/CMakeFiles/cpfromkfs.dir/all] Error 2
make: *** [all] Error 2
Original issue reported on code.google.com by [email protected]
on 29 Dec 2011 at 5:55
after, command "python kfssetup.py -f machines.cfg -m machines.txt -b ../build
-w ../webui" and command "python kfslaunch.py -f machines.cfg -m machines.txt
-s", I tend to run command "kfsping -m -s 192.168.1.107 -p 20000", but can not
connect.
It display the following info
[root@bogon tools]# ./kfsping -m -s 192.168.1.107 -p 20000
Connect: : Connection refused
02-19-2012 14:23:40.960 ERROR - (kfsping_main.cc:127) Unable to connect to
192.168.1.107 20000
actually, Meta Server don't run.
So, I use cmd "./metaserver MetaServer.prp", shell tell me that
Loading key metaServer.clientPort with value 20000
Loading key metaServer.chunkServerPort with value 30000
Loading key metaServer.logDir with value ./kfslog
Loading key metaServer.cpDir with value ./kfscp
Using meta server client port: 20000
Using meta server chunk server port: 30000
Setting cluster key to:
Setting md5sumFilename to:
min. # of chunkserver that should connect: 1
min. # of replicas per file: 1
max. # of replicas per file: 64
Setting the # of open files to: 1024
restoring from checkpoint of Tue Feb 14 23:06:31 2012
log file is ./kfslog/log.0
Log time: Tue Feb 14 23:06:31 2012
kfs_startup() end
02-20-2012 16:39:45.170 INFO - (metaserver_main.cc:88) Starting metaserver...
02-20-2012 16:39:45.203 INFO - (replay.cc:57) Doing log replay from file:
./kfslog/log.0
02-20-2012 16:39:45.204 INFO - (replay.cc:57) Doing log replay from file:
./kfslog/log.0
02-20-2012 16:39:45.231 DEBUG - (logger.cc:122) Opening ./kfslog/log.0 in
append mode
and no response.
why I can't start meta server and chunk server??
OS: Centos 5.7 x86_64
KFS Code: kfs-0.5
Thank you very much!
Original issue reported on code.google.com by [email protected]
on 20 Feb 2012 at 8:48
What steps will reproduce the problem?
1. kfs_fuse mountpoint
2. try to cp a small file
Gives error on cp: Numerical result out of range
ChunkServer:
12-12-2011 02:45:00.469 DEBUG - (MetaServerSM.cc:411) recv meta reply: seq:
2516678953956645717 status: 0 corrupt chunk: fileid = 8 chunkid = 139
12-12-2011 02:45:06.980 DEBUG - (MetaServerSM.cc:479) recv meta cmd: seq:
335567769309155733 alloc-chunk: seq: 335567769309155733 fileid: 8 chunkid: 140
chunkvers: 1 leaseid: -1 append: 0
Can't make it to work even on basic files like an image.
Ubuntu 11.04, kfs 0.5
Original issue reported on code.google.com by [email protected]
on 12 Dec 2011 at 12:55
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.