Predict the PV Power Generation
Published:
- Who: the solo- & side-project
- When: July 5th 2021 - July 26th 2021
- Where: DACON Time Series Prediction Competition for AI/ML, a section in “Predict the PV Power Generation”.
- Dataset: Dacon dataset
- What: Prediction the PV power generated in the next 2 days based on the weather information of 7 days.
- How: TensorFlow 2.x. Feature engineering to obtain the reasonable feature based on the given features. Stacking(Ensemble) of MLP, LSTM, CNN(Conv1D), CNN-LSTM, LGBM, RF models. Using the pinball loss. Quantile regression. Rectified Adam + LookAhead optimizer.
- Learn: SOTA optimizer(RAdam + Lookahead). Quantile Regression. Pinball loss. Ensemble/Stacking of ML/DL models. Feature engineering to get the meaningful information from the given features in the PV power generation. A combined model of CNN & LSTM in the sequential.
- I got the 15th position in the public LB, and finally ranked at the 2nd of private LB, but it’s not officially counted because I submitted this after the end of competition period. The motivation for joinging this terminated-competition was that I have done the very similar ML task from N*** hiring process, but there was no ranking chart/verification for ML task at the hiring process. I needed to check where I was and how I was doing well.
- The used models (stacking and ensemble), data preprocessing method, feature engineering method, prediction method/rule, and requested loss function(pinball loss) were very similar in DACON and N***.
- Models: Sequential CNN(two for Conv1D layers with LayerNormalized input), Sequential FC(two for Dense layers with l2 regularizers and LayerNormalized input), LSTM (one for LSTM layer with LayerNormalized input), Sequentially connected CNN-LSTM layer(two for Conv1D layers and one for LSTM layer with LayerNormalized input), LGBM(bagging_fraction and feature_fraction were used), and RF(feature_fraction and bagging were used). Totally, 4 (DL models) * 9 (# of quantile) * 2 (separately targeted days) models from deep learning and 2 (separately targeted days) * 9 (# of quantile) models from LGBM, and 2 RF models were ensembled with the linear meta-model, which is the one FC layer with LayerNormalized input. That is to say, 54 models in total, each of 54 models are expected to extract similar but different features from input data, therefore the ensembling will improve accuracy.
- Score: 1.79348 (Public) / 1.98949 (Private)
- Note: I cannot open the dataset I received from N*** and submitted code, but I just uploaded the DACON dataset and code I submitted.