Giter VIP home page Giter VIP logo

Comments (5)

R-omk avatar R-omk commented on August 18, 2024 1

This library is not really about "Consistent hash", read more https://en.wikipedia.org/wiki/Rendezvous_hashing

from flexihash.

pda avatar pda commented on August 18, 2024

Here's the relevant code:

// hash the target into multiple positions
for ($i = 0; $i < round($this->replicas * $weight); ++$i) {
$position = $this->hasher->hash($target.$i);
$this->positionToTarget[$position] = $target; // lookup
$this->targetToPositions[$target] [] = $position; // target removal
}

Based on that, and some very hazy memory:

For a given weight w, that target is w times more likely to be picked compared to w = 1.
So in your example, the regular targets have 1 / 12 = 8.3% chance, and the boosted one has 3/12 = 25% chance.
(The denominator is 12 because 9 targets of weight=1 plus 1 target of weight=3).
You can use non-integer weight for more subtle distribution, like 1.5.

from flexihash.

dliebner avatar dliebner commented on August 18, 2024

Thanks! So a higher weight => more distributions on the ring, as chosen by $target.$i, so the original placements will remain the same, only new ones will be added, "stealing" keys from other targets in a fairly distributed manner. Am I understanding that correctly?

from flexihash.

pda avatar pda commented on August 18, 2024

FYI in case this is helpful, this was the original inspiration / reference material for the library 14 years ago:

https://web.archive.org/web/20080331024845/http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/

This is the part that translated to $weight in flexihash:

You can tune the amount of load you send to each server based on that serverโ€™s capacity. Imagine this spatially โ€“ more points for a server means it covers more of the ring and is more likely to get more resources assigned to it.

Mostly I found the diagrams in that blog post really helpful in forming a mental model.

from flexihash.

dliebner avatar dliebner commented on August 18, 2024

Thanks for that! I plan to implement a bounded-load layer on top of this once it becomes necessary. For anyone else out there, here are the resources I drew from when learning about consistent hashing:
https://www.toptal.com/big-data/consistent-hashing
https://medium.com/vimeo-engineering-blog/improving-load-balancing-with-a-new-consistent-hashing-algorithm-9f1bd75709ed
https://ai.googleblog.com/2017/04/consistent-hashing-with-bounded-loads.html

from flexihash.

Related Issues (9)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.