nielsma

@nielsma

Joined on Oct 3, 2023

  • Definitions There are miners $m \in M$ and questions $q \in Q$, each with a respective cutoff time $t_q$. For any pair $(m,q)$, a miner $m$ submits a time series of forecasts $$ (p_{m,q,t}){t \le t_q}, $$ with $t \in T_q$, a list of time points (depending on the question) preceding the cutoff $t_q$. If a miner does not submit a forecast, we denote their submission by $p{\emptyset}$. Let $S(p_{m,q,t}, o_q)$ be the score of a given prediction when the question resolves to $o_q \in {0,1}$ (with $1$ indicating that the underlying event occurred and $0$ otherwise). Peer Score
     Like  Bookmark
  • key points we incentivize early prediction accuracy by giving more weight to such submissions. we introduce a differentiation mechanism to reward more top miners along the obvious fact that (in terms of Brier score) an accuracy of 0.80 is much better than 0.76 - this mechanism is currently linear. It will be made exponential soon after to also reflect the also obvious fact that improving from 0.89 to 0.90 is exponentially harder than improving from 0.75 to 0.76. definitions Let K be the amount of miners, E an event, $I$ the first time $E$ is listed on our network, D the cutoff, and $T$ a threshold. Let $S(p,o_E)$ be the Brier score of a prediction $p$ sent for the event $E$ which resolution was $o_E$ (1 if it occured and 0 otherwise). Let $(p_{t,k})_{t \leq D}$ be the time series of all the predictions submitted by miner $k$ for $E$ before cutoff. The scoring rule
     Like  Bookmark
  • exponential component (current update) Recall that given the time series of the miner's predictions $(p_t)$ we are scoring the miner at settlement through exponentially decreasing weights $(w_t)$: $$m_k = \frac{\sum_{j} w_{j,D} \times S(p_{j,k}, o_E)}{\sum_j w_{j,D}}$$ where $S$ is currently the Brier scoring rule. Right now we are applying the following transformation to $m_k$: $$B_{k} = \frac{m_{k} - \min(m_{i})}{\max(m_{i}) - \min(m_{i})}$$
     Like  Bookmark