Comments (1)
LiSu
commented on May 26, 2024
GraphScope基于分布式GNN训练框架graphlearn-for-pytorch (GLTorch)支持大规模分布式GNN训练。GLTorch在模型层和PyG兼容，支持将PyG GNN训练扩展到分布式大图。为了支持在大于机器可用内存大小的图上训练GNN，PyG引入了一套可插拔的Remote Backend机制，即通过FeatureStore 和 GraphStore等抽象，支持第三方图存储引擎和PyG的对接。其中FeatureStore允许用户利用存储在远程的节点/边特征，GraphStore允许用户利用存储在远程的图结构信息，两者结合支持基于远端存储的GNN训练扩展。本项目旨在通过实现基于Graphscope的PyG Remote Backend，更进一步简化GraphScope和PyG的对接方式，提供对PyG用户友好的基于GraphScope进行分布式GNN训练的产品使用体验。
产出：
- 基于GLTorch当前架构，设计FeatureStore 和 GraphStore在GraphScope上的实现方案
- 完成Remote Backend整体实现，在GraphScope上提供基于PyG Remote Backend的分布式训练示例
难度： 初级
技术要求：熟练使用Python语言，熟悉C++
from graphscope.
Related Issues (20)
- [BUG] 利用Cypher查询节点，不返回节点属性，节点属性必须指定，并且返回的结果不稳定？ HOT 2
- Integrate Interactive with `gsctl` HOT 1
- [BUG] [Graphscope on yarn] When generating Fragments, there is a partition issue.
- feature(interactive): Support `properties()` to Fetch All Properties of A Vertex in Cypher
- [BUG] Missing pattern sentences in Pattern Matching in GOpt-based compilation
- Why does GraphScope Vineyard require so much memory when running on Yarn
- [BUG] [GAE] The triangles algorithm results cannot be consistent for the same data set
- [BUG] Interactive unexpected behavior: delete serving graph successfully by openAPI generated code
- In pregel programming model, how does the static method Combine work HOT 1
- feat(interactive): Support retrieving current running schema of the query service
- feat(interactive): Add a API which will return the statistic of a graph.
- feat(interactive): Adapt to the GOpt optimizer
- [BUG] The returned `path` can not be displayed by neo4j driver
- [BUG] Unexpected when dealing with filters in `match` in the GOpt-based compilation
- [BUG] Unexpected alias maintenance in the new GOpt-based compilation stack
- [BUG] The absence of `bulk_loader` in system `PATH` cause engine down.
- refactor(interactive): Refine Stored procedure stack
- Miss pk identify in some cases in GOpt
- Deal with the cases when type infer is not precise in Gremlin queries in GOpt HOT 1
- Reorganize the Coordinator and gsctl to adapt to the unified schema
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from graphscope.