Giter VIP home page Giter VIP logo

sofa-rpc-node's Introduction

sofa-rpc-node

SOFARPC Nodejs 实现版本

NPM version Node.js CI Known Vulnerabilities npm download

一、SOFARPC Node 简介

简单说它是 SOFARPC 的 Nodejs 版实现,但本质上它是一个通用的 Nodejs RPC 解决方案。Nodejs RPC 在阿里和蚂蚁内部已经发展了四五年时间,如今广泛应用于各类业务场景,并经历了多次双 11 大促的考验。功能方面从基本的服务发布、寻址、点对点远程调用能力;到各种路由、负载均衡策略;再到故障隔离、熔断等高级功能,已逐渐发展成一个高可扩展性、高性能、生产级的 RPC 框架。

二、模块划分

SOFARPC Node 主要包含了四个子模块,分别是:

  • client: RPC 的客户端实现
  • server: RPC 的服务端实现
  • registry: 服务注册中心抽象及实现(目前提供 zookeeper 实现)
  • test: RPC 测试工具类
.
└── lib
    ├── client
    ├── registry
    ├── server
    └── test

三、快速上手

安装

npm install sofa-rpc-node --save

安装并启动 zookeeper

sofa-rpc-node 默认的注册中心实现基于 zookeeper,所以需要先启动一个 zookeeper 实例

从 Homebrew 安装(macOs)

brew install zookeeper

启动 zk server(默认端口为 2181)

zkServer start

# ZooKeeper JMX enabled by default
# Using config: /usr/local/etc/zookeeper/zoo.cfg
# Starting zookeeper ... STARTED

代码示例

  • 暴露一个 RPC 服务,并发布到注册中心
const { RpcServer } = require('sofa-rpc-node').server;
const { ZookeeperRegistry } = require('sofa-rpc-node').registry;
const logger = console;

// 1. 创建 zk 注册中心客户端
const registry = new ZookeeperRegistry({
  logger,
  address: '127.0.0.1:2181', // 需要本地启动一个 zkServer
});

// 2. 创建 RPC Server 实例
const server = new RpcServer({
  logger,
  registry, // 传入注册中心客户端
  port: 12200,
});

// 3. 添加服务
server.addService({
  interfaceName: 'com.nodejs.test.TestService',
}, {
  async plus(a, b) {
    return a + b;
  },
});

// 4. 启动 Server 并发布服务
server.start()
  .then(() => {
    server.publish();
  });
  • 调用 RPC 服务(从注册中心获取服务列表)
const { RpcClient } = require('sofa-rpc-node').client;
const { ZookeeperRegistry } = require('sofa-rpc-node').registry;
const logger = console;

// 1. 创建 zk 注册中心客户端
const registry = new ZookeeperRegistry({
  logger,
  address: '127.0.0.1:2181',
});

async function invoke() {
  // 2. 创建 RPC Client 实例
  const client = new RpcClient({
    logger,
    registry,
  });
  // 3. 创建服务的 consumer
  const consumer = client.createConsumer({
    interfaceName: 'com.nodejs.test.TestService',
  });
  // 4. 等待 consumer ready(从注册中心订阅服务列表...)
  await consumer.ready();

  // 5. 执行泛化调用
  const result = await consumer.invoke('plus', [ 1, 2 ], { responseTimeout: 3000 });
  console.log('1 + 2 = ' + result);
}

invoke().catch(console.error);
  • 调用 RPC 服务(直连模式)
const { RpcClient } = require('sofa-rpc-node').client;
const logger = console;

async function invoke() {
  // 不需要传入 registry 实例了
  const client = new RpcClient({
    logger,
  });
  const consumer = client.createConsumer({
    interfaceName: 'com.nodejs.test.TestService',
    serverHost: '127.0.0.1:12200', // 直接指定服务地址
  });
  await consumer.ready();

  const result = await consumer.invoke('plus', [ 1, 2 ], { responseTimeout: 3000 });
  console.log('1 + 2 = ' + result);
}

invoke().catch(console.error);
  • 测试 RPC Server 的方法(用于单元测试)
const request = require('sofa-rpc-node').test;
const { RpcServer } = require('sofa-rpc-node').server;
const logger = console;

describe('test/server.test.js', () => {
  let server;
  before(async function() {
    server = new RpcServer({
      logger,
      port: 12200,
    });
    server.addService({
      interfaceName: 'com.nodejs.test.TestService',
    }, {
      async plus(a, b) {
        return a + b;
      },
    });
    await server.start();
  });
  after(async function() {
    await server.close();
  });

  it('should call plus ok', async function() {
    await request(server)
      .service('com.nodejs.test.TestService')
      .invoke('plus')
      .send([ 1, 2 ])
      .expect(3);
  });
});
  • 暴露和调用 protobuf 接口

通过 *.proto 来定义接口

syntax = "proto3";

package com.alipay.sofa.rpc.test;

// 可选
option java_multiple_files = false;

service ProtoService {
  rpc echoObj (EchoRequest) returns (EchoResponse) {}
}

message EchoRequest {
  string name = 1;
  Group group = 2;
}

message EchoResponse {
  int32 code = 1;
  string message = 2;
}

enum Group {
  A = 0;
  B = 1;
}

服务端代码

const antpb = require('antpb');
const protocol = require('sofa-bolt-node');
const { RpcServer } = require('sofa-rpc-node').server;
const { ZookeeperRegistry } = require('sofa-rpc-node').registry;
const logger = console;

// 传入 *.proto 文件存放的目录,加载接口定义
const proto = antpb.loadAll('/path/proto');
// 将 proto 设置到协议中
protocol.setOptions({ proto });

const registry = new ZookeeperRegistry({
  logger,
  address: '127.0.0.1:2181',
});

const server = new RpcServer({
  logger,
  protocol, // 覆盖协议
  registry,
  codecType: 'protobuf', // 设置默认的序列化方式为 protobuf
  port: 12200,
});

server.addService({
  interfaceName: 'com.alipay.sofa.rpc.test.ProtoService',
}, {
  async echoObj(req) {
    req = req.toObject({ enums: String });
    return {
      code: 200,
      message: 'hello ' + req.name + ', you are in ' + req.group,
    };
  },
});
server.start()
  .then(() => {
    server.publish();
  });

客户端代码

const antpb = require('antpb');
const protocol = require('sofa-bolt-node');
const { RpcClient } = require('sofa-rpc-node').client;
const { ZookeeperRegistry } = require('sofa-rpc-node').registry;
const logger = console;

// 传入 *.proto 文件存放的目录,加载接口定义
const proto = antpb.loadAll('/path/proto');
// 将 proto 设置到协议中
protocol.setOptions({ proto });

const registry = new ZookeeperRegistry({
  logger,
  address: '127.0.0.1:2181',
});

async function invoke() {
  const client = new RpcClient({
    logger,
    protocol,
    registry,
  });
  const consumer = client.createConsumer({
    interfaceName: 'com.alipay.sofa.rpc.test.ProtoService',
  });
  await consumer.ready();

  const result = await consumer.invoke('echoObj', [{
    name: 'gxcsoccer',
    group: 'B',
  }], { responseTimeout: 3000 });
  console.log(result);
}

invoke().catch(console.error);

最佳实践

虽然上面我们提供了示例代码,但是我们并不推荐您直接使用该模块,因为它的定位是 RPC 基础模块,只提供基本的 API,对于业务开发者可能并不是非常友好。我们的最佳实践是通过插件将 RPC 能力集成到 Egg.js 框架里,提供更加直观友好的用户接口,让您就像使用本地方法一样使用 RPC。这块也会在近期开放,敬请期待!

三、相关文档

四、如何贡献

请告知我们可以为你做些什么,不过在此之前,请检查一下是否有已经存在的Bug或者意见

如果你是一个代码贡献者,请参考代码贡献规范

五、开源协议

MIT

Contributors


gxcsoccer


fengmk2


killagu


mansonchor


semantic-release-bot


ChangedenCZD


KenyeeC


RabbitSion


hyj1991


mytEcust


smile21


onlylovermb


chenfengjw163

This project follows the git-contributor spec, auto updated at Mon May 22 2023 14:22:46 GMT+0800.

sofa-rpc-node's People

Contributors

changedenczd avatar chenfengjw163 avatar duan-0916 avatar fengmk2 avatar gxcsoccer avatar hyj1991 avatar kenyeec avatar killagu avatar mansonchor avatar mytecust avatar onlylovermb avatar rabbitsion avatar semantic-release-bot avatar smile21 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sofa-rpc-node's Issues

invoke时 增加对中间件返回值处理 方便做请求拦截 请求缓存

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

重试机制

invoke 时,如果目标服务器出现网络不可达,或是短暂的繁忙造成请求响应超时时,有没有需要加入重试机制

how to re-publish after reconnected to the zookeeper ?

After the connection expired , it will auto reconnect , but how can we re-publish the provider ?
is there any way to do sth like this :

server.start()
    .then(async () => {
        server.publish();
        server.on('server:event', async (e) => {
            if (e.type === 'zookeeper' && e.name === 'connected') {
                server.services.get(e.id).rePublish();
            }
        });
    });

支持Nacos

支持Nacos作为注册中心,目前仅支持zk,局限性太大。不知道仓库是否还在维护?

Docker swarm, 通过zookeeper得到rpc服务器地址后无法连接

container 的网络
eth0 Link encap:Ethernet HWaddr 02:42:0A:FF:01:96
inet addr:10.255.1.150 Bcast:10.255.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:164 errors:0 dropped:0 overruns:0 frame:0
TX packets:118 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:18204 (17.7 KiB) TX bytes:16597 (16.2 KiB)

eth1 Link encap:Ethernet HWaddr 02:42:0A:00:2B:81
inet addr:10.0.43.129 Bcast:10.0.43.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:994842 errors:0 dropped:0 overruns:0 frame:0
TX packets:1557577 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:88179528 (84.0 MiB) TX bytes:124562996 (118.7 MiB)

eth0 被自动注册到zookeeper去了, 但是只有eth1 才能够顺利访问。
如何在注册的时候选择注册哪个网络啊?

Can't find remote invoke method

Describe the bug
Trying to connect to JAVA server using SOFA boot, the service was connected but when I invoke the method, got error

RPC-02412: Cannot find invoke method

I have tried every combination of the method on the server, but doesn't work. Any Idea what to put in the method?

Thanks!

To Reproduce
Steps to reproduce the behavior:

  1. Start JAVA SOFA Boot server
  2. Create a client with direct connect to the server
  3. Run invoke call
  4. See error

Expected behavior
Expected response in server logs or even other errors

sofa node作为客户端,调用使用hessian4的版本的服务端,出现异常

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

服务端使用默认hessian3序列化版本,用node客户端去调用服务端,服务端可以正常接收请求,并返回,
但是当服务端hessian版本换成4的话,使用原来的node代码去调用的时候,出现异常,服务端未收到请求,我猜是序列化的问题,以下是错误日志:
image,寻求专业解答。

本项目还在维护吗

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

rpcServer是否可以手动指定服务ip?

gxcsoccer 你好

我这边在实际使用的时候有一个小问题,我的服务器有多块网卡,并不能实际的准确指定服务的ip,
能否在rpcServer初始化的时候指定服务ip呢?

类似这样,

// 2. 创建 RPC Server 实例
const server = new RpcServer({
  logger,
  registry, // 传入注册中心客户端
  port: 12200,
  // 传入本机ip
  ip?: 192.168.1.1
});

谢谢

支持motan协议吗

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

hessain2 HashMap cannot be cast to xx.xx.entity

node ts 描述参数类

export interface AppPermissionListVo {
   /**
    * 路由LIST
    */
   permissions: Array<AppPlatformPermissionVo>;
}

参数只包装了父级 cn.bhbapp.vo.platform.AppPermissionListVo
于是调用时,AppPlatformPermissionVo解析报错,转换异常。

能否增加一个Nacos的NacosRegister

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

能否增加一个Nacos的NacosRegister

Describe the solution you'd like
A clear and concise description of what you want to happen.

因为注册中心用的Nacos,如果要用Zookeeper还得单独部署zookeeper,或者自己用直连模式,根据Nacos注册的情况,手动负载均衡,如果提供个Nacos的NacosRegister,少写很多代码

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

集成 dubbo 时,直连服务是可以的,但是通过注册中心就不成功了

  • 集成 dubbo 时,AddressGroup 的 addressList 为空
// sofa-rpc-node/lib/client/consumer.js 中

  async _init() {
    this._addressGroup = this.createAddressGroup(this.id + '@' + this.group);
    if (this.options.serverHost) {
      const addressList = this.options.serverHost.split(',').map(url => this.parseUrl(url));
      setImmediate(() => { this._addressGroup.addressList = addressList; });
    } else {
      await this.registry.ready();
      this._addressGroup = this.createAddressGroup(this.id + '@' + this.group);
      this._addressListener = addressList => {
// addressList 为 []
        this._addressGroup.addressList = addressList.map(url => this.parseUrl(url));
      };
      this.registry.subscribe(this.registryConfig, this._addressListener);
    }
    await this._addressGroup.ready();
  }

  • 上述原因导致 invoke 时 执行
async getConnection(req) {
    return await this._addressGroup.getConnection(req);
  }

时返回 null

想知道为什么 addressList 是空的?

集成 dubbo 时,直连服务是可以的,但是通过注册中心就不成功了

还有 group 的设置貌似不成功,直连的时候我需要将 group 名字写在 interfaceName 里面才行,单独配置 group 或者在初始化 client 时设置 group 都不成功

consumer invoke 无法 ready 时可能会造成内存泄露

问题一:consumer invoke 无法 ready 时,因为会不断 push 回调函数到数组中,所以会造成内存泄露,ready 的执行在这里
问题二:客户端可能会设置 responseTimeout,但如果无法 ready 时就会超过这个 timeout 的时间,客户端不好处理一些超时问题

zk断网或者本地断网超过N分钟后,死循环导致内存严重泄漏

data_client文件中 zk 重连函数,for循环,遍历 map的keys,并且会触发 zookeeper-client:watcher事件
image

api_client中监听zookeeper-client:watcher 最终会触发data_client的invokeWithWatcher,因为是操作的同一个map _watchers 会导致一个死循环,map永远无法清空,所以需将data_client中的 for (const key of this._watchers.keys()) 替换为
const keys = [...this._watchers.keys()];
for(const key of keys)
image

参照demo ,一直报一个错。

2018-11-25 11:23:47,345 WARN 8708 nodejs.ResponseTimeoutError: register to channel: [email protected]:2181 failed, will retry after 3s, Server no response in 3000ms, address#127.0.0.1:7777

直连模式可以调用,但是使用 ZookeeperRegistry 就一直报上面的错

启动两个RpcServer,再关闭其中一个,另一个也无法使用

1、按照demo创建两个进程,一个RpcServer使用12200端口,另一个使用12201端口

'use strict';

const { RpcServer } = require('sofa-rpc-node').server;
const { ZookeeperRegistry } = require('sofa-rpc-node').registry;
const logger = console;

// 1. 创建 zk 注册中心客户端
const registry = new ZookeeperRegistry({
  logger,
  address: '127.0.0.1:2181', // 需要本地启动一个 zkServer
});

// 2. 创建 RPC Server 实例
const server = new RpcServer({
  logger,
  registry, // 传入注册中心客户端
  port: 12200, // 另一个使用12201
});

// 3. 添加服务
server.addService({
  interfaceName: 'com.nodejs.test.TestService',
}, {
  async plus(a, b) {
    console.log(arguments)
    return a + b;
  },
});

// 4. 启动 Server 并发布服务
server.start()
  .then(() => {
    server.publish();
  });

2、ZK显示/sofa-rpc/com.nodejs.test.TestService/providers有两个节点

[bolt%3A%2F%2F10.39.190.174%3A12200%3FstartTime%3D1550489950969%26pid%3D41913%26uniqueId%3D%26dynamic%3Dtrue%26appName%3D%26timeout%3D3000%26serialization%3Dhessian2%26weight%3D100%26accepts%3D100000%26language%3Dnodejs%26rpcVer%3D50400%26protocol%3D%26interface%3Dcom.nodejs.test.TestService%26version%3D1.0%26group%3DSOFA, bolt%3A%2F%2F10.39.190.174%3A12201%3FstartTime%3D1550489964826%26pid%3D41915%26uniqueId%3D%26dynamic%3Dtrue%26appName%3D%26timeout%3D3000%26serialization%3Dhessian2%26weight%3D100%26accepts%3D100000%26language%3Dnodejs%26rpcVer%3D50400%26protocol%3D%26interface%3Dcom.nodejs.test.TestService%26version%3D1.0%26group%3DSOFA]

2、关闭其中一个进程
3、过一段时间后,ZK显示/sofa-rpc/com.nodejs.test.TestService/providers下为空

[]

期望另一个进程应该继续提供服务才对

`npm i` 后会删除`.travis.yml`的`before_install`指令

执行npm i后会删除 .travis.ymlbefore_install指令

-before_install:
-  - 'wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz'
-  - 'tar xf zookeeper-3.4.6.tar.gz'
-  - 'mv zookeeper-3.4.6/conf/zoo_sample.cfg zookeeper-3.4.6/conf/zoo.cfg'
-  - './zookeeper-3.4.6/bin/zkServer.sh start'

可以和java dubbo框架互相调用么

想请教下,使用example里dubbo的server提供provider,在java里使用dubbo框架,能调用么。不知道是不是方式不对,我试了不行,还是说java也需要sofa框架的dubbo协议?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.