Comments (8)
jamesdfrost
commented on May 10, 2024
2
But what about the ethical implications? I mean, a lot of these AI's are young and inexperienced, and I'd be worried about them being introduced to gambling from birth - don't you remember what happened in Superman III? ;)
from open_spiel.
elkhrt
commented on May 10, 2024
I think this is a really interesting area, and would be excited to see work
in this direction.
I'd recommend starting with money games, which are conceptually much
simpler, but have similar dynamics.
…On Sun, 22 Sep 2019, 14:49 jamesdfrost, ***@***.***> wrote:
I'd like to take a look at implementation of doubling cube and match logic
for Backgammon.
The doubling cube is critical to backgammon strategy, and previous
Backgammon value networks such as TDGammon have not implemented this within
the neural network - it's been traditionally bolted on via a separate
doubling algorithm. I think this has the potential to be an area where this
framework could offer significant improvements over previous works.
The way I would see it working would be:
- Players with an available double would have a move choice of
offering the double or rolling the dice.
- If double is offered, opponent would have the move choices of accept
the double or resign.
- If the double is accepted the doubling dice would then be owned by
the opposing player.
For this to work, the backgammon State will need to include:
- Match Player 0 score
- Match Player 1 score
- Match SetPoints (the number of points a player needs to get to)
- Match isCrawford (is the Crawford rule applicable?)
- Doubling cube value
- Doubling cube owner (0/1 for players, -1 for no owner)
Would also need to have a way of passing the following options to
backgammon.cc :
- UseDoublingCube
- MatchSetPoints (1 for single game)
- UseCrawfordRule
Would be interested in views on this - probably won't be looking at this
for a few weeks (assuming you're happy to support this), but keen to get
initial thoughts down.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#67>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AHAF7TDPUUEK3LAZME6VCI3QK5ZWZANCNFSM4IZCU2DA>
.
from open_spiel.
elkhrt
commented on May 10, 2024
🤣 - we've already started them on poker!
The problem with tournament play is that there's no objective measure of
utility until the end of the match. So an episode (for learning purposes)
needs to be the whole match rather than a single game.
Whereas a cash game has a clear utility function at the end of a single
game, which is much simpler.
…On Sun, 22 Sep 2019, 19:10 jamesdfrost, ***@***.***> wrote:
But what about the ethical implications? I mean, a lot of these AI's are
young and inexperienced, and I'd be worried about them being introduced to
gambling from birth - don't you remember what happened in Superman III? ;)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#67>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AHAF7TGS4A2T6JZ7YTYDNJLQK6YH7ANCNFSM4IZCU2DA>
.
from open_spiel.
jamesdfrost
commented on May 10, 2024
OK, will let them gamble, but only before 9:00pm at the weekends.! :)
Could you still not have a reward function based on the number of points won on an individual game, which is capped to the number of points which contribute to the series win, and then perhaps some form of bonus (e.g. +100) for winning the series?
So there is still an immediate reward for winning points, but also the series win becomes the most important carrot, and the bot should eventually learn not to get too risky when it doesn't need to.
I can't see a situation where you would deliberately lose a game in order to win a series, the key is making sure it doesn't risk doubles which it doesn't need to.
But agree it is going to be more difficult than money games, although there are a few extra options / rules which would need implemented for these (Beaver / Jacoby)
from open_spiel.
elkhrt
commented on May 10, 2024
Great!
Changing the eventual total reward will result in the agent learning the
wrong thing, although it's possible this isn't a big effect. I wouldn't do
it.
You could possibly give intermediate rewards to help learning, and then
subtract them from the final reward. That means the overall incentives are
unchanged, but the game is easier to learn. This could definitely help some
RL techniques.
But reward fiddling doesn't change the fundamental problem that in a match,
we don't know how much better 2-0 is than 1-0 (or 0-2), whereas in the
analogous cash game we do.
…On Sun, 22 Sep 2019, 20:07 jamesdfrost, ***@***.***> wrote:
OK, will let them gamble, but only before 9:00pm at the weekends.! :)
Could you still not have a reward function based on the number of points
won on an individual game, which is capped to the number of points which
contribute to the series win, and then perhaps some form of bonus (e.g.
+100) for winning the series?
So there is still an immediate reward for winning points, but also the
series win becomes the most important carrot, and the bot should eventually
learn not to get too risky when it doesn't need to.
I can't see a situation where you would deliberately lose a game in order
to win a series, the key is making sure it doesn't risk doubles which it
doesn't need to.
But agree it is going to be more difficult than money games, although
there are a few extra options / rules which would need implemented for
these (Beaver / Jacoby)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#67>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AHAF7TBL7IPOOFTAYLYXPRDQK666PANCNFSM4IZCU2DA>
.
from open_spiel.
lanctot
commented on May 10, 2024
Hi @jamesdfrost , love it. Yes, let's do this! Would be fantastic to get the full game supported.
Agree we should do step-by-step. Let's get the doubling cube in there first (btw, I assume "money games" just means that the points get multiplied by the value on the die? And we can move on the matches after we know that's working.
from open_spiel.
jamesdfrost
commented on May 10, 2024
Lanctot - yes, thats correct.
Interestingly although backgammon is often quoted as the worlds oldest board game, the introduction of doubling didn't happen till the 1920s, and led to a huge revival in the game due to the ability to really raise the stakes - before that, backgammon was in serious decline. Initially it was quite misunderstood - for example in Vanity Fair's Backgammon to Win they said"if two absolutely perfect players engaged in a match, there would never be an accepted double."
from open_spiel.
lanctot
commented on May 10, 2024
Hi @jamesdfrost , closing this one now as it's been open for a while and I want to use the Issues for actual outstanding issues. But would still love to have doubling cubes and matches eventually!
from open_spiel.
Related Issues (20)
- Is the comment example in RNaD EntropySchedule wrong? HOT 1
- About Predictive CFR+ HOT 2
- Some questions about population-based algorithms HOT 4
- Multiple policy heads in RNaD HOT 4
- Enhance Bridge State HOT 5
- python/examples/alpha_zero.py crashes with `CUDA_ERROR_NOT_INITIALIZED` HOT 3
- Bug report - wrong castling in chess HOT 3
- Support for newer CUDA drivers? HOT 6
- flag.h do not include HOT 5
- Preprocessor error when compiling with torchlib HOT 1
- Python 3.12 installation HOT 1
- Cannot resume Alphazero training with torchlib HOT 6
- dqn.cc build error HOT 2
- Policy Gradient based Self Play HOT 23
- Recommended Alphazero training config parameters HOT 6
- Question: how to evaluate rnad algorithm HOT 3
- RNaD: Possible Error in calculation of Neurd Loss HOT 7
- Quoridor Movement Action IDs keep changing HOT 16
- Implementing alphamu algorithm? HOT 1
- bug in chess terminal determination? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open_spiel.