Giter VIP home page Giter VIP logo

Comments (2)

maxwong avatar maxwong commented on May 14, 2024

Thanks for the question, minfree123.

What is Host / Guest / Arbiter?

Host / Guest / Arbiter are roles. A role is a logical representation of one Federated Learning (hereinafter referred to as FL) participant in a FL algorithm.

Let's take Vertical (i.e. 'Hetero' or 'Heterogeneous') LR algorithm as an example.
The complete dataset is a matrix X ∈ Rn×d. This matrix does not exist in one place but is composed of the columns of the datasets A and B held respectively by A and B . B also holds the label vector Y.
Then we can decompose W⊤x as: W⊤x = W⊤aXa + W⊤bXb. Party A and Party B is supposed to cooperate to get the federated W⊤x and update model parameters W further.

In our algorithm implementation, the party in possess of Y is named 'Guest' in algorithm, and this party has 'Guest' as its role name. Similarly, 'Host' is the party with Xa but WITHOUT Y, 'Arbiter' is the party to perform decryptions for Guest and Host.

To sum up, Host / Guest / Arbiter are roles, and roles are symbols of different FL algorithm parties.

What is a proxy?

Proxy is the networking communication endpoint for one FL party. Generally you can think of it as an application-layer router.
Proxy is connected to Federation Service only at current version.

How to configure a route_table?

route_table.json defines communication rules among FL parties. For now, it is a file-based configuration. If a guest needs to send data to a Host, it must find the corresponding rule in route_table. Followings are the example route_table you can find in this project:

{
  "route_table": {               # route_table section
    "default": {                 # global default party, i.e. default rule
      "default": [               # global default rules
        {
          "ip": "127.0.0.1",
          "port": 9999
        }
      ]
    },
    "10000": {                   # rules for party id 10000
      "default": [               # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "127.0.0.1",		
          "port": 8889
        }
      ]
    },
    "9999": {                    # rules for party id 9999
      "default": [
        {
        "ip": "127.0.0.1",
        "port": 8890
        }
      ]
    }
  },
  "permission": {                # permission section
    "default_allow": true        # default permission rule. This will evolve in near future release so let's just put a true here for now
  }
}

If party 10000 is running FATE, then the section for party 10000 in party 10000 can be configured as following:

"10000": {                  # rules for party id 10000
  "fate": [                 # federation endpoints this party, in json array
    {                       # one json object represents one federation entry, use one guest entry for now
      "ip": "192.168.0.1",        
      "port": 8088
    }
  ]
}

Route rule has the following priority: party id > role. More specifically:

  1. Exact match: both party id and role match. Corresponding rule will be used.
  2. Party id match only: party id matches, but role does not match. Default rule for that specific party will be used in this occasion.
  3. Nothing match: neither party id nor role matches. Global default rule will be used.

Vertical LR Example

Let's assume 10000 is Guest (the party in possess of Y), 9999 is Host (the party without Y), 10001 is Arbiter (central decryption party), and we use a central endpoint to deploy them as a star model.

role party id proxy ip:port federation ip:port
guest 10000 172.16.0.1:8443 192.168.1.1:9394
host 9999 172.16.0.2:8443 192.168.10.10:9394
arbiter 10001 172.16.0.3:8443 192.168.20.20:9394
<central> <n/a> 172.16.0.4:8443 <n/a>

In 10000, the configuration looks like the following:

{
  "route_table": {               # route_table section
    "default": {                 # global default party, i.e. default rule
      "default": [               # global default rules
        {
          "ip": "172.16.0.4",
          "port": 8443
        }
      ]
    },
    "10000": {                   # rules for party id 10000
      "fate": [                  # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "192.168.1.1",        
          "port": 9394
        }
      ]
    }
  },
  "permission": {                # permission section
    "default_allow": true        # default permission rule. This will evolve in near future release so let's just put a true here for now
  }
}

In 9999, the configuration looks like the following:

{
  "route_table": {               # route_table section
    "default": {                 # global default party, i.e. default rule
      "default": [               # global default rules
        {
          "ip": "172.16.0.4",
          "port": 8443
        }
      ]
    },
    "9999": {                    # rules for party id 9999
      "fate": [                  # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "192.168.10.10",        
          "port": 9394
        }
      ]
    }
  },
  "permission": {                # permission section
    "default_allow": true        # default permission rule. This will evolve in near future release so let's just put a true here for now
  }
}

In 10001, the configuration looks like the following:

{
  "route_table": {               # route_table section
    "default": {                 # global default party, i.e. default rule
      "default": [               # global default rules
        {
          "ip": "172.16.0.4",
          "port": 8443
        }
      ]
    },
    "10001": {                   # rules for party id 10000
      "fate": [                  # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "192.168.20.20",        
          "port": 9394
        }
      ]
    }
  },
  "permission": {                # permission section
    "default_allow": true        # default permission rule. This will evolve in near future release so let's just put a true here for now
  }
}

In central point, the configuation looks like the following:

{
  "route_table": {               # route_table section
    "10000": {                   # rules for party id 10000
      "default": [               # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "172.16.0.1",        
          "port": 8443
        }
      ]
    },
    "9999": {                    # rules for party id 9999
      "default": [               # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "172.16.0.2",        
          "port": 8443
        }
      ]
    },
    "10001": {                   # rules for party id 10001
      "default": [               # default rule for this party, in json array
        {                        # one json object represents one rule entry
          "ip": "172.16.0.3",        
          "port": 8443
        }
      ]
    }
  },
  "permission": {                # permission section
    "default_allow": true        # default permission rule. This will evolve in near future release so let's just put a true here for now
  }
}

Ending

In fact, we support deploying more than one role in one party.

We plan to provide more mechanisms for route table configurations and permission controls.

If above resolves your issue, we will create a document for proxy configuration.

from fate.

minfree123 avatar minfree123 commented on May 14, 2024

maxwong, Thanks for your detailed answer.
follow the conf rule you explained, it works now.
i put the same conf file in guest,host,arbiter nodes, as follows

{
  "route_table": {
    "default": {
      "default": [
        {
          "ip": "172.16.0.1",
          "port": 8443
        }
      ]
    },
    "10000": {
      "default": [
        {
          "ip": "172.16.0.1",
          "port": 8443
        }
      ],
      "fate": [
        {
          "ip": "172.16.0.1",        
          "port": 9394
        }
      ]
    },
    "9999": {
      "default": [
        {
          "ip": "172.16.0.2",
          "port": 8443
        }
      ],
      "fate": [
        {
          "ip": "172.16.0.2",        
          "port": 9394
        }
      ]
    },
    "10001": {
      "default": [
        {
          "ip": "172.16.0.3",
          "port": 8443
        }
      ],
      "fate": [
        {
          "ip": "172.16.0.3",        
          "port": 9394
        }
      ]
    }
  },
  "permission": {
    "default_allow": true
  }
}

from fate.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.