nebula-contrib / nebulagraph-lite Goto Github PK
View Code? Open in Web Editor NEWThe minimal, ad-hoc way of plug and play NebulaGraph with pip install, even inside Colab Notebook!
License: Apache License 2.0
The minimal, ad-hoc way of plug and play NebulaGraph with pip install, even inside Colab Notebook!
License: Apache License 2.0
It seems now the stop of service will cause metad corruption for always, cleanup is needed :(
❯ tail ~/.nebulagraph/lite/logs/meta0/*
==> /home/w/.nebulagraph/lite/logs/meta0/metad-stderr.log <==
/usr/local/nebula/bin/nebula-metad(event_base_loop+0x36e)[0x2648e8e]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase8loopBodyEib+0x46d)[0x2586efd]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase4loopEv+0x3d)[0x25877ed]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase11loopForeverEv+0x17)[0x258a427]
/usr/local/nebula/bin/nebula-metad(_ZN5folly20IOThreadPoolExecutor9threadRunESt10shared_ptrINS_18ThreadPoolExecutor6ThreadEE+0x338)[0x2514578]
/usr/local/nebula/bin/nebula-metad(_ZN5folly6detail8function14FunctionTraitsIFvvEE7callBigISt5_BindIFMNS_18ThreadPoolExecutorEFvSt10shared_ptrINS7_6ThreadEEEPS7_SA_EEEEvRNS1_4DataE+0x46)[0x2522ae6]
/usr/local/nebula/bin/nebula-metad[0x2b4d5ef]
/lib64/libpthread.so.0(+0x7ea4)[0x7c7760807ea4]
/lib64/libc.so.6(clone+0x6c)[0x7c77604fe96c]
(safe mode, symbolizer not available)
==> /home/w/.nebulagraph/lite/logs/meta0/metad-stdout.log <==
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.ERROR <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.FATAL <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.INFO <==
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20240114 10:52:40.028664 6557 MetaDaemon.cpp:137] localhost = "127.0.0.1":9559
I20240114 10:52:40.337798 6557 NebulaStore.cpp:48] Start the raft service...
I20240114 10:52:40.343425 6557 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 10485760 for each part by default
I20240114 10:52:40.637010 6557 RaftexService.cpp:46] Start raft service on 9560
I20240114 10:52:40.647907 6557 NebulaStore.cpp:82] Scan the local path, and init the spaces_
I20240114 10:52:40.648157 6557 NebulaStore.cpp:90] Scan path "/data/meta/nebula/0"
I20240114 10:52:40.648191 6557 NebulaStore.cpp:292] Init data from partManager for "127.0.0.1":9559
I20240114 10:52:40.648216 6557 NebulaStore.cpp:417] Create data space 0
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.ERROR.20240114-104504.3898 <==
Log file created at: 2024/01/14 10:45:04
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20240114 10:45:04.757242 3898 FileUtils.cpp:377] Failed to read the directory "/data/meta/nebula" (2): No such file or directory
E20240114 10:45:06.707832 3898 FileUtils.cpp:76] Failed to remove the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06/data" (39): Directory not empty
E20240114 10:45:06.707902 3898 FileUtils.cpp:49] Failed to remove "data" in "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
E20240114 10:45:06.707931 3898 FileUtils.cpp:70] Failed to remove the content of the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.ERROR.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.FATAL.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.INFO.20240114-104504.3898 <==
I20240114 10:51:34.553622 3898 RaftexService.cpp:67] Stopping the raftex service on port 9560
I20240114 10:51:34.553655 3898 RaftexService.cpp:77] All partitions have stopped
I20240114 10:51:34.553689 3898 NebulaStore.cpp:346] Stop kv engine...
I20240114 10:51:34.553740 3898 NebulaStore.cpp:36] Cut off the relationship with meta client
I20240114 10:51:34.570483 3898 Part.h:59] [Port: 9560, Space: 0, Part: 0] ~Part()
I20240114 10:51:34.579130 3898 RocksEngine.h:247] Release rocksdb on /data/meta/nebula/0
I20240114 10:51:34.580722 3898 NebulaStore.cpp:44] ~NebulaStore()
I20240114 10:51:34.580850 3898 MetaDaemon.cpp:215] The meta Daemon stopped
I20240114 10:51:34.593420 3898 JobManager.cpp:138] JobManager::shutDown() begin
I20240114 10:51:34.593461 3898 JobManager.cpp:141] JobManager not running, exit
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.INFO.20240114-105240.6557 <==
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20240114 10:52:40.028664 6557 MetaDaemon.cpp:137] localhost = "127.0.0.1":9559
I20240114 10:52:40.337798 6557 NebulaStore.cpp:48] Start the raft service...
I20240114 10:52:40.343425 6557 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 10485760 for each part by default
I20240114 10:52:40.637010 6557 RaftexService.cpp:46] Start raft service on 9560
I20240114 10:52:40.647907 6557 NebulaStore.cpp:82] Scan the local path, and init the spaces_
I20240114 10:52:40.648157 6557 NebulaStore.cpp:90] Scan path "/data/meta/nebula/0"
I20240114 10:52:40.648191 6557 NebulaStore.cpp:292] Init data from partManager for "127.0.0.1":9559
I20240114 10:52:40.648216 6557 NebulaStore.cpp:417] Create data space 0
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.WARNING.20240114-104504.3898 <==
Log file created at: 2024/01/14 10:45:04
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20240114 10:45:04.757242 3898 FileUtils.cpp:377] Failed to read the directory "/data/meta/nebula" (2): No such file or directory
E20240114 10:45:06.707832 3898 FileUtils.cpp:76] Failed to remove the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06/data" (39): Directory not empty
E20240114 10:45:06.707902 3898 FileUtils.cpp:49] Failed to remove "data" in "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
E20240114 10:45:06.707931 3898 FileUtils.cpp:70] Failed to remove the content of the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.WARNING.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.WARNING <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972 6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005
We could add CLI support
python3 -m nebulagraph_lite.cli
, via #21there some works to be done.
Target system:
Dependencies:
Usage:
pip install nebulagraph-lite
from nebulagraph_lite import nebulagraph_let as ng_let
n = ng_let()
n.start()
Design
~/.nebulagraph/lite/
Steps
! pip3 install nebulagraph-lite
from nebulagraph_lite import nebulagraph_let as ng_let
n = ng_let()
n.start()
Collecting nebulagraph-lite
Downloading nebulagraph_lite-0.2.2-py3-none-any.whl.metadata (7.5 kB)
Collecting udocker==1.3.13 (from nebulagraph-lite)
Downloading udocker-1.3.13-py2.py3-none-any.whl.metadata (36 kB)
Collecting psutil>=5.9.6 (from nebulagraph-lite)
Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Collecting nebula3-python>=3.5.0 (from nebulagraph-lite)
Using cached nebula3_python-3.5.0-py3-none-any.whl.metadata (8.4 kB)
Requirement already satisfied: future>=0.18.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (1.0.0)
Requirement already satisfied: httplib2>=0.20.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (0.21.0)
Requirement already satisfied: pytz>=2021.1 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (2023.3.post1)
Requirement already satisfied: six>=1.16.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (1.16.0)
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /opt/conda/lib/python3.10/site-packages (from httplib2>=0.20.0->nebula3-python>=3.5.0->nebulagraph-lite) (3.1.1)
Downloading nebulagraph_lite-0.2.2-py3-none-any.whl (16 kB)
Downloading udocker-1.3.13-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.6/118.6 kB 4.8 MB/s eta 0:00:00
Using cached nebula3_python-3.5.0-py3-none-any.whl (326 kB)
Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.2/288.2 kB 12.3 MB/s eta 0:00:00
Installing collected packages: udocker, psutil, nebula3-python, nebulagraph-lite
Attempting uninstall: psutil
Found existing installation: psutil 5.9.3
Uninstalling psutil-5.9.3:
Successfully uninstalled psutil-5.9.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
beatrix-jupyterlab 2023.128.151533 requires jupyterlab~=3.6.0, but you have jupyterlab 4.1.2 which is incompatible.
distributed 2023.7.1 requires dask==2023.7.1, but you have dask 2024.2.0 which is incompatible.
jupyterlab 4.1.2 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.
jupyterlab-lsp 5.0.3 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.
raft-dask 23.8.0 requires dask==2023.7.1, but you have dask 2024.2.0 which is incompatible.
Successfully installed nebula3-python-3.5.0 nebulagraph-lite-0.2.2 psutil-5.9.7 udocker-1.3.13
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.