Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sprt calculator: allow for specifying alpha and beta parameters #2078

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

sprt calculator: allow for specifying alpha and beta parameters #2078

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
14a23ed
Update calc.js
gahtan-syarif Jun 17, 2024
54777b4
Update sprt_calc.mak
gahtan-syarif Jun 17, 2024
e2634d5
Merge branch 'official-stockfish:master' into alphabeta
gahtan-syarif Jul 3, 2024
95b9407
Merge branch 'official-stockfish:master' into alphabeta
gahtan-syarif Aug 16, 2024
9ef6231
Update calc.js
gahtan-syarif Aug 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions server/fishtest/static/js/calc.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ const defaultParameters = {
"elo-1": "2.0",
"draw-ratio": "0.49",
"rms-bias": "191",
"alpha": "0.05",
"beta": "0.05",
};

let validSprt = null;
Expand Down Expand Up @@ -64,10 +66,12 @@ function drawCharts(resize) {
let elo1 = parseFloat(document.getElementById("elo-1").value);
const drawRatio = parseFloat(document.getElementById("draw-ratio").value);
const rmsBias = parseFloat(document.getElementById("rms-bias").value);
let alpha = parseFloat(document.getElementById("alpha").value);
let beta = parseFloat(document.getElementById("beta").value);
let error = "";
let sprt = null;

if (isNaN(elo0) || isNaN(elo1) || isNaN(drawRatio) || isNaN(rmsBias)) {
if (isNaN(elo0) || isNaN(elo1) || isNaN(drawRatio) || isNaN(rmsBias) || isNaN(alpha) || isNaN(beta)) {
error = "Unreadable input.";
} else if (elo1 < elo0 + 0.5) {
error = "The difference between Elo1 and Elo0 must be at least 0.5.";
Expand All @@ -77,8 +81,12 @@ function drawCharts(resize) {
error = "The draw ratio must be strictly between 0.0 and 1.0.";
} else if (rmsBias < 0) {
error = "The RMS bias must be positive.";
} else if (alpha <= 0.0 || alpha >= 1.0 || beta <= 0.0 || beta >= 1.0) {
error = "The value of alpha and beta must be strictly between 0.0 and 1.0.";
} else if (alpha + beta >= 1.0) {
error = "The sum of alpha and beta must be strictly below 1.0";
} else {
sprt = new Sprt(0.05, 0.05, elo0, elo1, drawRatio, rmsBias, eloModel);
sprt = new Sprt(alpha, beta, elo0, elo1, drawRatio, rmsBias, eloModel);
if (sprt.variance <= 0) {
error = "The draw ratio and the RMS bias are not compatible.";
}
Expand All @@ -101,7 +109,7 @@ function drawCharts(resize) {
history.replaceState(
null,
"",
`/sprt_calc?elo-model=${eloModel}&elo-0=${elo0}&elo-1=${elo1}&draw-ratio=${drawRatio}&rms-bias=${rmsBias}`,
`/sprt_calc?elo-model=${eloModel}&elo-0=${elo0}&elo-1=${elo1}&draw-ratio=${drawRatio}&rms-bias=${rmsBias}&alpha=${alpha}&beta=${beta}`,
);
elo0 = sprt.elo0;
elo1 = sprt.elo1;
Expand Down
28 changes: 25 additions & 3 deletions server/fishtest/templates/sprt_calc.mak
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,28 @@
min="0"
>
</div>
<div class="col-6 col-md-auto mb-3">
<label for="alpha" class="form-label">Alpha</label>
<input
id="alpha"
class="form-control number no-arrows"
type="number"
step="0.01"
min="0"
max="1"
>
</div>
<div class="col-6 col-md-auto mb-3">
<label for="beta" class="form-label">Beta</label>
<input
id="beta"
class="form-control number no-arrows"
type="number"
step="0.01"
min="0"
max="1"
>
</div>
<div class="col-12 col-md-auto mb-3 d-flex align-items-end">
<input
class="btn btn-success w-100"
Expand All @@ -79,10 +101,10 @@
<a href="https://en.wikipedia.org/wiki/Sequential_probability_ratio_test"
>SPRT test</a
>
with &alpha;=&beta;=0.05. In other words, if the true Elo is less than
Elo0 then the probability of the test passing is less than 5%. On the
with error probabilities &alpha; and &beta;. If the true Elo is less than
Elo0 then the probability of the test passing is less than &alpha;. On the
other hand if the true Elo is more than Elo1 then the pass probability is
more than 95%.
more than (1-&beta;).
</li>
<li>
If the Elo model is <b>Logistic</b> then the pass/fail probabilities are
Expand Down