Giter VIP home page Giter VIP logo

orleans.clustering.redis's Introduction

Orleans.Clustering.Redis

Orleans membership provider for Redis

Build Status

Orleans is a framework that provides a straight-forward approach to building distributed high-scale computing applications, without the need to learn and apply complex concurrency or other scaling patterns.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

StackExchange.Redis StackExchange.Redis is a high performance general purpose redis client for .NET languages (C# etc).

Orleans.Clustering.Redis is a package that uses Redis as a backend for cluster membership, making it easy to run Orleans clusters dynamically.

TL;DR

If you want to quickly test it, clone this repo and go to the Samples Directory for instructions on how to run a sample cluster.

Overview

Redis is a straight key/value store. Membership data is stored into the key [clusterid.serviceid] as a hash.

Installation

Installation is performed via NuGet

From Package Manager:

PS> Install-Package Orleans.Clustering.Redis

.Net CLI:

# dotnet add package Orleans.Clustering.Redis

Configuration

A functional Redis database is required for this provider to work.

Silo

Tell Orleans runtime that we are going to use Redis as our Cluster Membership Provider:

var silo = new SiloHostBuilder()
        ...
        .UseRedisMembership(opt =>
        {
            opt.ConnectionString = "host:port";
            opt.Database = 0;
        })
        ...
        .Build();

ConnectionString tells the connector where to find the Redis database.

Database is an integer which tells the membership table which database to get after connecting to the Redis service.

More information on connection string configuration can be found at on the StackExchange.Redis driver site (https://stackexchange.github.io/StackExchange.Redis/Configuration.html).

Client

Now that our silo is up and running, the Orleans client needs to connect to the Redis database to look for Orleans gateways.

var client = new ClientBuilder()
        ...
        .UseRedisMembership(opt =>
        {
            opt.ConnectionString = "host:port";
            opt.Database = 0;
        })
        ...
        .Build();

At the moment the gateway list is provided by the underlying membership provider directly.

Enjoy your Orleans application running without need to specify membership manually!

orleans.clustering.redis's People

Contributors

arleyschrock avatar reubenbond avatar sagron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

orleans.clustering.redis's Issues

Build and release a new package

Arrived here, trying to use this package for my project.

According to Nuget, the last release was 2018, but the last commit was a couple of months ago.

image

What is the status of this project? The readme doesn't mention any deprecation.

Is it possible to release an updated package to Nuget? :)

Thank you kindly!
Fábio

Unable to have more than one silo in the cluster

I am currently attempting to use your Redis membership provider for a development environment on my local machine, where I would essentially start silos at

  • 127.0.0.1:11101 and 127.0.0.1:30001
  • 127.0.2.1:11102 and 127.0.2.1:30002
  • 127.0.3.1:11103 and 127.0.3.1:30003

as well as a client connecting to the same Redis instance.

Now whichever silo hits Redis first creates an entry that looks like this:

127.0.0.1:6379> HGETALL dev:MyCluster
1) "S127.0.0.1:11101:277414322"
2) "{\"Entry\":{\"SiloAddress\":{\"Generation\":277414322,\"Endpoint\":{\"Address\":\"127.0.0.1\",\"Port\":11101}},\"Status\":3,\"SuspectTimes\":[],\"ProxyPort\":30001,\"HostName\":\"chillkroete\",\"SiloName\":\"Silo_69578\",\"RoleName\":\"TrackerService\",\"UpdateZone\":0,\"FaultZone\":0,\"StartTime\":\"2018-10-16T19:32:02.8708864Z\",\"IAmAliveTime\":\"2018-10-16T19:32:07.7225187Z\"},\"TableVersion\":{\"Version\":2,\"VersionEtag\":\"1\"},\"ResourceVersion\":\"1\"}"

That silo then logs information like these:

info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
info: Orleans.Runtime.SiloControl[0]
      GetSimpleGrainStatistics
info: Orleans.Runtime.Management.ManagementGrain[0]
      GetDetailedHosts onlyActive=True
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]

Interestingly, the other silos attempt to InsertRow and then still only find the first entry:

dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.2.1:11102:277414326
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414326,"Endpoint":{"Address":"127.0.2.1","Port":11102}},"Status":2,"SuspectTimes":[],"ProxyPort":30002,"HostName":"chillkroete","SiloName":"Silo_ace4d","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:06.6316528Z","IAmAliveTime":"2018-10-16T19:32:11.0376885Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.2.1:11102:277414326
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414326,"Endpoint":{"Address":"127.0.2.1","Port":11102}},"Status":2,"SuspectTimes":[],"ProxyPort":30002,"HostName":"chillkroete","SiloName":"Silo_ace4d","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:06.6316528Z","IAmAliveTime":"2018-10-16T19:32:18.0667248Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.2.1:11102:277414326
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414326,"Endpoint":{"Address":"127.0.2.1","Port":11102}},"Status":2,"SuspectTimes":[],"ProxyPort":30002,"HostName":"chillkroete","SiloName":"Silo_ace4d","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:06.6316528Z","IAmAliveTime":"2018-10-16T19:32:32.9006762Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.2.1:11102:277414326
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414326,"Endpoint":{"Address":"127.0.2.1","Port":11102}},"Status":2,"SuspectTimes":[],"ProxyPort":30002,"HostName":"chillkroete","SiloName":"Silo_ace4d","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:06.6316528Z","IAmAliveTime":"2018-10-16T19:32:39.0281563Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]

and

dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.3.1:11103:277414328
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414328,"Endpoint":{"Address":"127.0.3.1","Port":11103}},"Status":2,"SuspectTimes":[],"ProxyPort":30003,"HostName":"chillkroete","SiloName":"Silo_14d0e","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:08.2988266Z","IAmAliveTime":"2018-10-16T19:32:16.2606885Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.3.1:11103:277414328
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414328,"Endpoint":{"Address":"127.0.3.1","Port":11103}},"Status":2,"SuspectTimes":[],"ProxyPort":30003,"HostName":"chillkroete","SiloName":"Silo_14d0e","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:08.2988266Z","IAmAliveTime":"2018-10-16T19:32:31.1208846Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadRow: S127.0.3.1:11103:277414328
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      InsertRow: {"SiloAddress":{"Generation":277414328,"Endpoint":{"Address":"127.0.3.1","Port":11103}},"Status":2,"SuspectTimes":[],"ProxyPort":30003,"HostName":"chillkroete","SiloName":"Silo_14d0e","RoleName":"TrackerService","UpdateZone":0,"FaultZone":0,"StartTime":"2018-10-16T19:32:08.2988266Z","IAmAliveTime":"2018-10-16T19:32:34.6838954Z"}, {"Version":1,"VersionEtag":"0"}
dbug: Orleans.Clustering.Redis.RedisMembershipTable[0]
      ReadAll
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S127.0.0.1:11101:277414322 SiloName=Silo_69578 Status=Active]

Moreover, stopping one silo doesn't make the others fail over. I can verify in the OrleansDashboard that exactly one silo is registered in the cluster.

I feel like I'm doing something wrong, but I can provoke the same behavior with the example you provided:

127.0.0.1:6379> HGETALL testcluster:testcluster
1) "S172.17.0.1:10025:277414741"
2) "{\"Entry\":{\"SiloAddress\":{\"Generation\":277414741,\"Endpoint\":{\"Address\":\"172.17.0.1\",\"Port\":10025}},\"Status\":3,\"SuspectTimes\":[],\"ProxyPort\":20025,\"HostName\":\"chillkroete\",\"SiloName\":\"Silo_d2d31\",\"RoleName\":\"RedisSiloHost\",\"UpdateZone\":0,\"FaultZone\":0,\"StartTime\":\"2018-10-16T19:39:01.4353724Z\",\"IAmAliveTime\":\"2018-10-16T19:39:02.9488986Z\"},\"TableVersion\":{\"Version\":2,\"VersionEtag\":\"1\"},\"ResourceVersion\":\"1\"}"

SiloHost is saying

info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 0 are Active, 0 are Dead, 1 are Joining, Version=<1, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Joining]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Runtime.MembershipService.MembershipOracleData[100634]
      -ReadAll (called from BecomeActive) Membership table 1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Runtime.MembershipService.MembershipOracleData[100605]
      -Finished BecomeActive.

and RedisGatewayHost is just repeating

info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Runtime.Catalog[100507]
      Before collection#2: memory=9MB, #activations=0, collector=<#Activations=0, #Buckets=0, buckets=[]>.
info: Orleans.Runtime.Catalog[100508]
      After collection#2: memory=9MB, #activations=0, collected 0 activations, collector=<#Activations=0, #Buckets=0, buckets=[]>, collection time=00:00:00.0255795.
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]
info: Orleans.Clustering.Redis.RedisMembershipTable[0]
      1 silos, 1 are Active, 0 are Dead, Version=<2, 1>. All silos: [SiloAddress=S172.17.0.1:10025:277414741 SiloName=Silo_d2d31 Status=Active]

Do you have any idea where the issue could be? I'm on .NET Core 2.1, Orleans 2.1.1 and Redis 4.0.11.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.