🔢Preconfirmation Pricing & Backtest

Preconfirmation of Inclusion as Blockspace Futures

• POI is a future of block space inclusion at block $B_T$.

• Intrinsic value of POI is $S_T=\text{MEV}+V_{block}$ where

  • $\text{MEV}$ is the MEV value of the POI.

  • $V_{block}$ is the value of inclusion the users transaction in the future block $B_t$.

POI Pricing

$C=S_t*e^{-r(T-t_0)}+\text{Premium}$ where

• $T$ is the block time that the transaction will be included

• $t_0$ is the time of selling POI.

• $r$ is the riskless interest.

• $\text{Premium}$ is the risk premium term to cover preconfer’s risk.

To apply the formula, the pricing needs to be based on our prediction of congestion gas + contention gas, namely baseFee + priorityFee. Our current pricing formula is:

C_{Taiyi}=\text{total\_gas\_prediction} + c_1*\text{std}+c_2*\text{avg\_top\_k\_gas}$$C_{Taiyi}=\text{total\_gas\_prediction} + c_1*\text{std}+c_2*\text{avg\_top\_k\_gas}$, where

• $\text{total\_gas\_prediction}$ is our total gas(base+priority) prediction.

• $c_1$and $c_2$ are our tuned coefficients to ensure our pricing is

  • a proper approximation real total gas and

  • user would've used preconf had it been offered to them back then in the context of a backtest

• $\text{avg\_top\_k\_gas}$ is the average of the top k values of block average total gas from previous blocks.

The choice of $C_{Taiyi}$ is based on our backtest for base fee forecast.

Model Selection of Total Gas Prediction

In our quest to accurately predict total gas prices, we evaluated three families of models:

1. Simple Statistical Models

2. Auto-Regressive Models

3. Recurrent Neural Network (RNN) Based Models

First we exclude model 1 & 2 for because

• Simple Statistical Models:

  • While some used simple statistical approaches (e.g., quantile of tips from the mempool) with decent result, our backtest revealed these methods lack the accuracy required for predicting total gas in future blocks. These models generally fail to capture the complex dynamics of gas prices in the Ethereum network.

• Auto-Regressive Models:

  • These models assume a linear relationship between variables, which doesn't align with the non-linear nature of total gas fees.

  • For non-linear variants, determining an explicit form proved challenging and potentially limiting.

After evaluating various options, we narrowed our focus to two advanced machine learning models: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). Both are types of Recurrent Neural Networks (RNNs) well-suited for time series prediction tasks like gas fee forecasting.

We approached the model selection without preconceptions, testing both LSTM and GRU thoroughly. Our findings revealed that:

1. Performance Similarity: Both models performed similarly in predicting total gas prices.

2. Complexity: LSTM is slightly more complex than GRU, but this added complexity didn't translate to significantly better results in our case.

3. Industry Precedent: LSTM has been widely used in previous gas fee prediction studies, giving it a slight edge in terms of proven track record.

4. Prediction Accuracy: While both models showed promise, they both exhibited some deviation from actual gas fees. This aligns with our expectations given the inherent unpredictability in block-building.

After careful consideration, we chose to implement the LSTM model for our current system. This decision was based on its widespread use in similar applications and its ability to handle long-term dependencies in data.

Backtest Results

Our backtest provides forecasts for 15 Ethereum blocks, equivalent to approximately 3 minutes, into the future.

In the backtest conducted in Apr/2024, the average preconfirmation (preconf) price was about 9.3% higher than what a user paid for a single transaction(baseFee+priorityFee). This aligns with our belief that user would pay 10%~15% more for preconfirmation. The increased revenue would represent the upper bound of what a proposer would earn on average.

Glossary

1. Mean Squared Error (MSE): MSE measures the average of the squares of the errors, representing the average squared difference between the actual and predicted values. A lower MSE indicates a better model fit.

2. Root Mean Squared Error (RMSE): RMSE is the square root of the Mean Squared Error, making it easier to interpret as it shares the same units as the original data.

3. Mean Absolute Error (MAE): MAE measures the average magnitude of errors in a set of predictions, without considering their direction, providing a straightforward measure of average error.

4. Mean Absolute Percentage Error (MAPE): MAPE expresses model accuracy as a percentage, making it scale-independent and useful for comparing models on different scales.

5. R-squared (R²): R² is a statistical measure that indicates the proportion of the variance in the dependent variable explained by the independent variable(s) in a regression model. R² ranges from 0 to 1, with higher values indicating a better model fit.

Window Size: 1 block (first 1000 blocks)

The graph below shows the comparison of the actual gas fee and preconf pricing, with the x-axis representing the block index.

Evaluation Metrics:

MSE: 261.6231850366638

RMSE: 16.174770014954273

MAE: 4.2415752204634405

MAPE: 0.19671249553472564

R2: 0.571902358744013

Window Size: 50 blocks (10 mins)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 50 (unit: 10 mins), and the average values for each segment.

Evaluation Metrics:

MSE: 16.50100675201061

RMSE: 4.062143123033778

MAE: 1.6900811664237785

MAPE: 0.08577219000631359

R2: 0.9550149737122783

Window Size: 300 blocks (1 hour)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 300 (unit: 1 hour), and the average values for each segment.

Untitled

Evaluation Metrics:

MSE: 3.234182361255306

RMSE: 1.7983832631714813

MAE: 0.9206226636959047

MAPE: 0.04987972275958826

R2: 0.9902850275659132

Window Size: 900 blocks (3 hours)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 900 (unit: 3 hours), and the average values for each segment.

Untitled

Evaluation Metrics:

MSE: 1.3372782519807416

RMSE: 1.156407476619181

MAE: 0.7062495042012438

MAPE: 0.04008814800955924

R2: 0.9949308780537647

Window Size: 7200 blocks (1 day)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 7200 (unit: 1 day), and the average values for each segment.

Untitled

Evaluation Metrics:

MSE: 0.4000870460167505

RMSE: 0.6325243442087826

MAE: 0.5403508491073973

MAPE: 0.03123625335442407

R2: 0.9958112012796164