Handling satellite rainfall estimates Data Using Hyperparameter Tuning with GridSearchCV

Project info

• Field of studyMeteorology and Climate
• LevelDoctoral dissertation (S3)
• Method typeData Quality
• Estimated build time± 7 working days
• Toolsscikit-learn GridSearchCV, Pipeline, cv_results_
• PriceUS$28 - US$175

Research goal

Inspect and improve the quality of rainfall estimate data with cloud-top temperature and cloud cover using Hyperparameter Tuning with GridSearchCV, then assess the impact on analysis.

Variables

Example result

• 0.889Best CV accuracy
• 0.944Test accuracy
• 158.489Best C
• 0.04Best gamma
• 0.085CV std (best)
• 180Model fits

GridSearchCV evaluates 36 SVM combinations of C and gamma; the best is C = 158, gamma = 0.04 with a cross-validated accuracy of 0.889 and a test accuracy of 0.944.

Computed from illustrative synthetic data, not from a real study.

About Hyperparameter Tuning with GridSearchCV

GridSearchCV tries every hyperparameter combination in a defined grid using cross-validation to select the best configuration. It is used so model parameter selection is objective rather than trial and error.

Steps

1. 1Define the model and evaluation metric
2. 2Split into training and test sets, keeping the test set until the end
3. 3Build a sensible hyperparameter grid
4. 4Run GridSearchCV with cross-validation on the training data
5. 5Inspect the best parameters and results for each combination
6. 6Evaluate the best model once on the test set

Assumptions to check

• The test set is not used during tuning
• The grid covers a sufficient range, not only values near the defaults
• Computational cost grows quickly with the number of combinations

Reading and reporting the results

Report the grid searched, the cross-validation scheme, best parameters, validation score, and test score. If time is limited, explain RandomizedSearchCV as an alternative.

Example Python code

A short starting script with this project's variable names. The delivered version is adapted to your data and fully commented.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV, train_test_split, StratifiedKFold
from sklearn.metrics import accuracy_score

X, rainfall_estimate = make_classification(n_samples=500, n_features=10, n_informative=5, random_state=39)
cloud_top_temperature = X[:, 0]
cloud_cover = X[:, 1]

X_tr, X_te, y_tr, y_te = train_test_split(X, rainfall_estimate, test_size=0.25, stratify=rainfall_estimate, random_state=39)
grid = {"n_estimators": [50, 150], "max_depth": [3, 6, None], "min_samples_leaf": [1, 5]}
gs = GridSearchCV(RandomForestClassifier(random_state=39), grid,
                  cv=StratifiedKFold(5, shuffle=True, random_state=39),
                  scoring="accuracy", n_jobs=1)
gs.fit(X_tr, y_tr)
print("Best params:", gs.best_params_)
print("Best CV accuracy:", round(gs.best_score_, 3))
print("Test accuracy:", round(accuracy_score(y_te, gs.predict(X_te)), 3))

res = pd.DataFrame(gs.cv_results_).sort_values("mean_test_score", ascending=False)
print(res[["params", "mean_test_score", "std_test_score"]].head(5).to_string())

plt.errorbar(range(len(res)), res["mean_test_score"], yerr=res["std_test_score"], fmt="o")
plt.xlabel("Combination (sorted)"); plt.ylabel("CV accuracy")
plt.show()

Packages and prices

Choose how much help you need. Each higher package includes everything in the lower ones. Prices exclude 11% VAT.

• Payment: 50% down payment before work starts, balance after the deliverables are handed over
• Tax: Prices exclude 11% VAT
• Data and academic integrity: Results are computed from your own research data; you write and stand behind your manuscript, we help with the analysis and your understanding of it
• Revisions: Up to 2 revisions are included in the price (Package 3: 3 rounds)

Frequently asked questions