vdbergh / spsa_simul Goto Github PK

The basis will be the document which was posted in the corresponding Fishtest issue. However it needs to be cleaned up.

• Remove the words Theorem/Proof. There is neither a theorem, nor a proof. Use instead "Main results" and "Derivation".

• Write the formulas in terms of sigma^2. Add a comment that sigma^2 can be approximated by 1-d in the trinomial case where d is the draw ratio, or more accurately by 1-d-4*b in the pentanomial case where b is the RMS bias of the book expressed in standard score units. The current footnote which says that it is 1-d-b is ambiguous, as is does not specify the units of b. For running a tune this is not really relevant as sigma^2 can be measured while the tune is running. It is however relevant if we want to compute an estimate for the number of required games before a tune starts.

• Box the important formulas.

• Move the more precise results for a diagonal Hessian to a separate section.

This will make it possible to judge the accuracy of the formulas.

Only the the optima of the true loss function have to be provided for the simulation.

They are (with some examples)

• General: (threads, seed, truncate,quiet/verbose)

• Context: (draw_ratio, confidence).

• Requirements: (precision).

• Design: (num_params,est_elos,minima,optima,maxima,c_ratio,lambda_ratio)

• Simulation: (true_elos,start_elo).

Now the program generates too much output. Ideally it should do the following.

• Print the actual loss function parameters (elos,minima,optima,maxima) in tabular form (and add a column with the diagonal Hessian entries).

• Print the data for unambiguously simulating the tune (besides the loss function) (the draw ratio, the starting point, c and r).

• When specifying --verbose print more stuff.

Use --est_start_elo instead to calculate an optimal value for r. This work has already started in the branch optimal.