Python: How to retrive the best model from Optuna LightGBM study?

To retrieve the best model from an Optuna LightGBM study, you can use the study.best_trial method to get the best trial in the study, and then use the trial.user_attrs attribute to get the trained LightGBM model. Here's an example of how to do this:

import lightgbm as lgb
import optuna

# Define a function to optimize with Optuna
def optimize(trial):
    # Get the current value for the hyperparameter being optimized
    param = trial.suggest_uniform('param', 0, 1)

    # Train a LightGBM model with the current value of the hyperparameter
    model = lgb.LGBMClassifier(param=param)
    model.fit(X_train, y_train)

    # Return the cross-validated accuracy of the model
    return model.score(X_val, y_val)

# Create an Optuna study and optimize the function
study = optuna.create_study()
study.optimize(optimize, n_trials=100)

# Get the best trial in the study
best_trial = study.best_trial

# Get the trained LightGBM model from the best trial's user attributes
best_model = best_trial.user_attrs['model']

You can then use the best_model variable to make predictions or save the model for later use.