# Preconfirmation Pricing & Backtest

## Preconfirmation of Inclusion as Blockspace Futures

* POI is a future of block space inclusion at block $$B_T$$.
* Intrinsic value of POI is $$S_T=\text{MEV}+V_{block}$$ where
  * $$\text{MEV}$$ is the MEV value of the POI.
  * $$V_{block}$$ is the value of inclusion the users transaction in the future block $$B_t$$.

#### **POI Pricing**

$$C=S_t*e^{-r(T-t_0)}+\text{Premium}$$ where

* $$T$$ is the block time that the transaction will be included
* $$t_0$$ is the time of selling POI.
* $$r$$ is the riskless interest.
* $$\text{Premium}$$ is the risk premium term to cover preconfer’s risk.

To apply the formula, the pricing needs to be based on our prediction of congestion gas + contention gas, namely `baseFee + priorityFee`. Our current pricing formula is:

$$ C_{Taiyi}=\text{total\_gas\_prediction} + c_1*\text{std}+c_2*\text{avg\_top\_k\_gas}$ $$$$C_{Taiyi}=\text{total\_gas\_prediction} + c_1*\text{std}+c_2*\text{avg\_top\_k\_gas}$$, where

* $$\text{total\_gas\_prediction}$$ is our total gas(base+priority) prediction.
* $$c_1$$and $$c_2$$ are our tuned coefficients to ensure our pricing is
  * a proper approximation real total gas and
  * user would've used preconf had it been offered to them back then in the context of a backtest
* $$\text{avg\_top\_k\_gas}$$ is the average of the top k values of block average total gas from previous blocks.

The choice of $$C_{Taiyi}$$ is based on our backtest for base fee forecast.&#x20;

### Model Selection of Total Gas Prediction

In our quest to accurately predict total gas prices, we evaluated three families of models:

1. Simple Statistical Models
2. Auto-Regressive Models
3. Recurrent Neural Network (RNN) Based Models

First we exclude model 1 & 2 for because

* Simple Statistical Models:
  * While some used simple statistical approaches (e.g., quantile of tips from the mempool) with decent result, our backtest revealed these methods lack the accuracy required for predicting total gas in future blocks. These models generally fail to capture the complex dynamics of gas prices in the Ethereum network.
* Auto-Regressive Models:
  * These models assume a linear relationship between variables, which doesn't align with the non-linear nature of total gas fees.
  * For non-linear variants, determining an explicit form proved challenging and potentially limiting.

After evaluating various options, we narrowed our focus to two advanced machine learning models: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). Both are types of Recurrent Neural Networks (RNNs) well-suited for time series prediction tasks like gas fee forecasting.

We approached the model selection without preconceptions, testing both LSTM and GRU thoroughly. Our findings revealed that:

1. Performance Similarity: Both models performed similarly in predicting total gas prices.
2. Complexity: LSTM is slightly more complex than GRU, but this added complexity didn't translate to significantly better results in our case.
3. Industry Precedent: LSTM has been widely used in previous gas fee prediction studies, giving it a slight edge in terms of proven track record.
4. Prediction Accuracy: While both models showed promise, they both exhibited some deviation from actual gas fees. This aligns with our expectations given the inherent unpredictability in block-building.

After careful consideration, we chose to implement the LSTM model for our current system. This decision was based on its widespread use in similar applications and its ability to handle long-term dependencies in data.

## Backtest Results

Our backtest provides forecasts for 15 Ethereum blocks, equivalent to approximately 3 minutes, into the future.

In the backtest conducted in Apr/2024, the average preconfirmation (preconf) price was about 9.3% higher than what a user paid for a single transaction(`baseFee`+`priorityFee`). This aligns with our belief that user would pay 10%\~15% more for preconfirmation. The increased revenue would represent the upper bound of what a proposer would earn on average.&#x20;

### Glossary

1. **Mean Squared Error (MSE)**: MSE measures the average of the squares of the errors, representing the average squared difference between the actual and predicted values. A lower MSE indicates a better model fit.
2. **Root Mean Squared Error (RMSE)**: RMSE is the square root of the Mean Squared Error, making it easier to interpret as it shares the same units as the original data.
3. **Mean Absolute Error (MAE)**: MAE measures the average magnitude of errors in a set of predictions, without considering their direction, providing a straightforward measure of average error.
4. **Mean Absolute Percentage Error (MAPE)**: MAPE expresses model accuracy as a percentage, making it scale-independent and useful for comparing models on different scales.
5. **R-squared (R²)**: R² is a statistical measure that indicates the proportion of the variance in the dependent variable explained by the independent variable(s) in a regression model. R² ranges from 0 to 1, with higher values indicating a better model fit.

### Window Size: 1 block (first 1000 blocks)

The graph below shows the comparison of the actual gas fee and preconf pricing, with the x-axis representing the block index.

<figure><img src="https://1315766109-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fh88lV66yWTqsYqtm6Wjh%2Fuploads%2FZwWlsnj6A5govui2caCg%2Fimage1.png?alt=media&#x26;token=d07bee2f-4985-4547-87b6-8a372fcec135" alt=""><figcaption></figcaption></figure>

![](https://prod-files-secure.s3.us-west-2.amazonaws.com/bb2bebf3-c510-4d62-8622-fa13f61d708a/2280fbcd-371f-48c6-951f-e842ea8cc63f/Untitled.png)

Evaluation Metrics:

MSE: 261.6231850366638&#x20;

RMSE: 16.174770014954273&#x20;

MAE: 4.2415752204634405&#x20;

MAPE: 0.19671249553472564&#x20;

R2: 0.571902358744013

### Window Size: 50 blocks (10 mins)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 50 (unit: 10 mins), and the average values for each segment.

<figure><img src="https://1315766109-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fh88lV66yWTqsYqtm6Wjh%2Fuploads%2FEYB5qCrNDhHYiXbXNpkX%2Fimage2.png?alt=media&#x26;token=951d7851-7536-41c8-bcca-44b25a853113" alt=""><figcaption></figcaption></figure>

Evaluation Metrics:

MSE: 16.50100675201061&#x20;

RMSE: 4.062143123033778&#x20;

MAE: 1.6900811664237785&#x20;

MAPE: 0.08577219000631359&#x20;

R2: 0.9550149737122783

### Window Size: 300 blocks (1 hour)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 300 (unit: 1 hour), and the average values for each segment.

<figure><img src="https://1315766109-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fh88lV66yWTqsYqtm6Wjh%2Fuploads%2F9ANTqjfBkeTWsAKOe8b4%2Fimage3.png?alt=media&#x26;token=83423864-9b7b-4335-afec-df012e1f76e7" alt=""><figcaption></figcaption></figure>

![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/bb2bebf3-c510-4d62-8622-fa13f61d708a/17030230-649e-41af-adf8-afa15ef4fa25/Untitled.png)

Evaluation Metrics:

MSE: 3.234182361255306&#x20;

RMSE: 1.7983832631714813&#x20;

MAE: 0.9206226636959047&#x20;

MAPE: 0.04987972275958826&#x20;

R2: 0.9902850275659132

### Window Size: 900 blocks (3 hours)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 900 (unit: 3 hours), and the average values for each segment.

<figure><img src="https://1315766109-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fh88lV66yWTqsYqtm6Wjh%2Fuploads%2F11QZ5PROJ8vO6Rsw4CSW%2FUntitled.png?alt=media&#x26;token=c30956d6-e8a0-4daf-90a9-d94f53a6bad2" alt=""><figcaption></figcaption></figure>

![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/bb2bebf3-c510-4d62-8622-fa13f61d708a/2b960e4b-549f-4d1b-a07c-8be1d369babf/Untitled.png)

Evaluation Metrics:

MSE: 1.3372782519807416&#x20;

RMSE: 1.156407476619181&#x20;

MAE: 0.7062495042012438&#x20;

MAPE: 0.04008814800955924&#x20;

R2: 0.9949308780537647

### Window Size: 7200 blocks (1 day)

The graph below shows the comparison of the total gas and preconf pricing, with the x-axis representing the block index divided by 7200 (unit: 1 day), and the average values for each segment.

<figure><img src="https://1315766109-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fh88lV66yWTqsYqtm6Wjh%2Fuploads%2FNOGeXiXxCRNjzKv2GN2I%2Fimage5.png?alt=media&#x26;token=a51afe39-a744-4702-92b3-96ca8eb66de9" alt=""><figcaption></figcaption></figure>

![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/bb2bebf3-c510-4d62-8622-fa13f61d708a/8ea447c2-2247-49e5-900b-e7647539b7ad/Untitled.png)

Evaluation Metrics:

MSE: 0.4000870460167505

RMSE: 0.6325243442087826&#x20;

MAE: 0.5403508491073973&#x20;

MAPE: 0.03123625335442407&#x20;

R2: 0.9958112012796164
