It's me chiming again with what I've been doing today.
Basically I spent about 6 hours of today thinking about this, in the process coming up with 4 formulae, 2 of which I ditched because they worked badly.
If you remember, I suggested that the rating system have a constant p that makes the rating change less drastically because of luck-based wins, and asked the community about such a value of p. I managed to incorporate this p in the formula for the rating change. The new rating with p will be called Glicko with Constant Weighting.
Later, TAY raised the point that if a player beats another player that is much higher-ranked than him, it is more probable that his win was due to luck. Hence I tried my hand at making Glicko with Variable Weighting. The problem was about how to make the weighting change, and that's why I came up with 4 formulae: each formula corresponds to a different method of varying the weighting. After testing them, it was apparent that 2 of them were counter-intuitive so I'm left with two.
One of them, which I'll call Glicko with Linear Weighting, assumes the following. Say that PW is the probability of Player 1 winning against Player 2 on merit. This can be calculated given both players' rating and deviation by a formula invented by Glickman. Also suppose W is 1 if Player 1 won, and 0 if Player 2 won. Then the unsigned value of (PW-W), which we shall call x, is a value corresponding to how wrong we were in our PW prediction. If x = 0, then the player was either sure to win according to PW and won, or sure to lose according to PW and lost. If x = 1, the player was either sure to win according to PW and lost, or sure to lose according to PW and won. Needless to say, in practice x is always a number between 0 and 1 exclusive.
Given this x, Glicko with Linear Weighting assumes that
If x = 0 then P = 1
If x = 0.5 then P = p
If x = 1 then P = 2p-1
where p is the constant discussed before.
The other formula, which I'll call Glicko with Quadratic Weighting, assumes that
If x = 0 then P = 1
If x = 0.5 then P = p
If x = 1 then P = 3p-2
i.e. it makes the rating change even less in the case of an unexpected result than for the Linear Weighting one.
I then tested a hypothetical Player 1 having rating 1500 and deviation 100 playing against Player 2 having deviation 100 and the following ratings, assuming that he wins in all cases. Here are the updated ratings of Player 1 using the original Glicko, Glicko with Constant Weighting, Glicko with Linear Weighting and Glicko with Quadratic Weighting, assuming p = 0.9375:
Code:
Player 2 Rating New Player 1 Rating using
Normal Glicko Constant Weighting Linear Weighting Quadratic Weighting
---------------------------------------------------------------------------------------------
600 1500 1500 (=) 1500 (=) 1500 (=)
650 1501 1500 (-1) 1501 (=) 1501 (=)
700 1501 1501 (=) 1501 (=) 1501 (=)
750 1501 1501 (=) 1501 (=) 1501 (=)
800 1501 1501 (=) 1501 (=) 1501 (=)
850 1501 1501 (=) 1501 (=) 1501 (=)
900 1502 1502 (=) 1502 (=) 1502 (=)
950 1503 1502 (-1) 1502 (-1) 1503 (=)
1000 1503 1503 (=) 1503 (=) 1503 (=)
1050 1504 1504 (=) 1504 (=) 1504 (=)
1100 1505 1505 (=) 1505 (=) 1505 (=)
1150 1507 1506 (-1) 1507 (=) 1507 (=)
1200 1509 1507 (-2) 1508 (-1) 1508 (-1)
1250 1511 1509 (-2) 1510 (-1) 1510 (-1)
1300 1513 1511 (-2) 1512 (-1) 1512 (-1)
1350 1516 1514 (-2) 1515 (-1) 1515 (-1)
1400 1519 1516 (-3) 1517 (-2) 1517 (-2)
1450 1522 1519 (-3) 1520 (-2) 1520 (-2)
1500 1526 1522 (-4) 1522 (-4) 1522 (-4)
1550 1529 1525 (-4) 1525 (-4) 1525 (-4)
1600 1533 1528 (-5) 1527 (-6) 1527 (-6)
1650 1536 1531 (-5) 1530 (-6) 1529 (-7)
1700 1539 1534 (-5) 1532 (-7) 1530 (-9)
1750 1542 1537 (-5) 1534 (-8) 1531 (-11)
1800 1544 1539 (-5) 1535 (-9) 1532 (-12)
1850 1546 1541 (-5) 1536 (-10) 1533 (-13)
1900 1548 1542 (-6) 1537 (-11) 1533 (-14)
1950 1550 1543 (-7) 1538 (-12) 1534 (-16)
2000 1551 1544 (-7) 1539 (-12) 1534 (-17)
2050 1552 1545 (-7) 1539 (-13) 1534 (-18)
2100 1552 1546 (-6) 1540 (-12) 1534 (-18)
2150 1553 1546 (-7) 1540 (-13) 1534 (-19)
2200 1553 1547 (-6) 1540 (-13) 1534 (-19)
2250 1554 1547 (-7) 1541 (-13) 1534 (-20)
2300 1554 1547 (-7) 1541 (-13) 1534 (-20)
2350 1554 1547 (-7) 1541 (-13) 1534 (-20)
2400 1554 1548 (-6) 1541 (-13) 1534 (-20)
(Glicko with Constant Weighting makes P = p no matter what x is, while normal Glicko makes P = 1 no matter what x is.)
If you notice, when Player 1 beats a player having a rating lower than his, all three weightings don't change the original Glicko rating by much. However, when he beats a player having a rating higher than his, the ratings change from a little (Constant Weighting) to moderately (Linear Weighting) to a lot (Quadratic Weighting).
I'd like to hear your opinions on all the above. And sorry for the longish post. :(