Make VQC & QNN Scikit-learn compliant

[Sudeep09]

What is the expected enhancement?

VQC & QNN does not allow to implement Sklearn BaseEstimator to be inherited. Hence, we are unable to use Sklearn Gridsearch on VQC & QNN estimators for hyperparameter tuning.

[adekusar-drl]

There is a guide how to develop scikit learn compatible estimators: https://scikit-learn.org/stable/developers/develop.html
In general, inheritance is not required, but method compatibility is.

[Sudeep09]

class VQC_Sklearn(VQC, NeuralNetworkClassifier, BaseEstimator, ClassifierMixin):       
    def get_params(self, deep=True):
        out = dict()
        for key in self._get_param_names():
        ## The if statement is to bypass "callback" and "quantum_instance" attribute not defined
            if (key != "callback" and key != "quantum_instance"):
                value = getattr(self, key)
                if deep and hasattr(value, "get_params") and not isinstance(value, type):
                    deep_items = value.get_params().items()
                    out.update((key + "__" + k, val) for k, val in deep_items)
                out[key] = value
        return out

However, I still get the AttributeError: can't set attribute

[adekusar-drl]

However, I still get the AttributeError: can't set attribute

I guess, set_params to be implemented as well. The implementation may be tricky though.

[Sudeep09]

Finally, thanks a lot to blquanz@us.ibm.com, the solution below worked for me:

class vqc_wrapper(BaseEstimator):    
    def __init__(self, feature_map,ansatz,optimizer,initial_point,quantum_instance,callback):
        self.feature_map = feature_map
        self.ansatz = ansatz
        self.optimizer = optimizer
        self.initial_point = initial_point
        self.quantum_instance = quantum_instance
        self.callback = callback
        self.vqc = None
        
    def fit(self, X, y):
        self.vqc = VQC(
            feature_map=self.feature_map,
            ansatz=self.ansatz,
            optimizer=self.optimizer,
            initial_point=self.initial_point,
            quantum_instance=self.quantum_instance,
            callback=self.callback)
        return self.vqc.fit(X,y)
    
    def predict(self, X):
        return self.vqc.predict(X)
    
    def score(self, X, y):
        return self.vqc.score(X,y)

[adekusar-drl]

If you don't mind, I'd keep this issue open as it is not resolved in the QML code.

[gayatri-valp]

I have an opinion on this, we can try some approaches to fix this issue.
Instead of using Sklearn Gridsearch, we can implement a custom hyperparameter search function for VQC and QNN. Taking the reference from this: https://scikit-learn.org/stable/modules/grid_search.html . I think this is possible by defining a parameter space for the hyperparameters and then performing a grid search over that space as like we do there.
On seraching for other alternatives, I found another approach.
It is to create wrapper functions that allow VQC and QNN to be used as Sklearn estimators. This can be done by creating a wrapper class that inherits from BaseEstimator and implements the necessary methods such as fit(), predict(), and score(). The wrapper class can then be used with Sklearn Gridsearch for hyperparameter tuning.
I'm just checking to implement any one of them.

from sklearn.base import BaseEstimator, ClassifierMixin
from qiskit import Aer
from qiskit.aqua import QuantumInstance
from qiskit.aqua.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes

class VQCWrapper(BaseEstimator, ClassifierMixin):
    def __init__(self, feature_map=None, ansatz=None, optimizer=None, quantum_instance=None):
        self.feature_map = feature_map or ZZFeatureMap()
        self.ansatz = ansatz or RealAmplitudes()
        self.optimizer = optimizer or SPSA(max_trials=100)
        self.quantum_instance = quantum_instance or QuantumInstance(Aer.get_backend('qasm_simulator'))

    def fit(self, X, y):
        self.vqc = VQC(feature_map=self.feature_map, ansatz=self.ansatz, optimizer=self.optimizer, quantum_instance=self.quantum_instance)
        self.vqc.train(X, y)
        return self

    def predict(self, X):
        return self.vqc.predict(X)

    def score(self, X, y):
        return self.vqc.test(X, y)['testing_accuracy']

[AbdullahKazi500]

@GayatriVadaparty Let me know if you want to work on this Issue as a collaboration
One approach is to use a custom hyperparameter search function, as you mentioned earlier. Another approach is to create a custom estimator class that inherits from Sklearn's BaseEstimator and wraps the VQC or QNN model. This custom estimator can then be used with Sklearn's GridSearchCV or RandomizedSearchCV functions for hyperparameter tuning.
Bayesian optimization This is an alternative to grid search that uses a probabilistic model of the objective function to find promising hyperparameters. It iteratively selects the next set of hyperparameters to evaluate based on the model's predictions of where the optimal point might be.
Another option is to use other libraries or frameworks that support hyperparameter tuning for VQC and QNN models. For example, PennyLane is a popular open-source library for quantum machine learning that supports hyperparameter tuning for quantum models, including VQC and QNN. PennyLane integrates with Sklearn and other popular machine learning libraries and provides a range of optimization and gradient-based methods for hyperparameter tuning.

[gayatri-valp]

@AbdullahKazi500 Yes, I wanted to work on this issue. I have recently got to understand all the approaches you mentioned above.

[AbdullahKazi500]

@adekusar-drl
this is my code

!pip install qiskit-machine-learning
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from qiskit import Aer
from qiskit_machine_learning.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
from qiskit.algorithms.optimizers import SPSA
from sklearn.metrics import accuracy_score

# Generate synthetic dataset for demonstration
X, y = make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create a parameter grid for hyperparameter tuning
param_grid = {
    'ansatz': [RealAmplitudes(), None],
    'optimizer': [SPSA(maxiter=100)],
    'quantum_instance': [Aer.get_backend('qasm_simulator'), Aer.get_backend('statevector_simulator')]
}

best_score = 0
best_params = None

# Loop over the parameter grid
for ansatz in param_grid['ansatz']:
    for optimizer in param_grid['optimizer']:
        for quantum_instance in param_grid['quantum_instance']:
            try:
                # Create the feature map and VQC
                num_qubits = X.shape[1]
                feature_map = ZZFeatureMap(num_qubits, reps=1)
                vqc = VQC(feature_map, ansatz, optimizer, quantum_instance=quantum_instance)

                # Fit the model
                vqc.fit(X_train, y_train)

                # Predict on the test set
                y_pred = vqc.predict(X_test)

                # Calculate accuracy score
                score = accuracy_score(y_test, y_pred)

                # Check if the current model has the best score
                if score > best_score:
                    best_score = score
                    best_params = {'ansatz': ansatz, 'optimizer': optimizer, 'quantum_instance': quantum_instance}

            except Exception as e:
                print(f"Error: {e}")
                continue

# Print the best hyperparameters and score
print("Best hyperparameters:", best_params)
print("Best score:", best_score)```

 It seems that the ZZFeatureMap class in Qiskit is not compatible with scikit-learn's GridSearchCV due to the dynamic nature of the circuit construction.

To work around this issue, I have  manually loop over the parameter grid and perform the grid search without relying on scikit-learn's GridSearchCV
In this code, I manually loop over the parameter grid and instantiate the ZZFeatureMap, VQC, and other components. We Can fit the model, predict on the test set, and calculate the accuracy score. If the current model achieves a better score than the previous best model, we update the best score and best parameters.

[AbdullahKazi500]

@adekusar-drl

@adekusar-drl

It seems that the ZZFeatureMap class in Qiskit is not compatible with scikit-learn's GridSearchCV due to the dynamic nature of the circuit construction.

To work around this issue, I have manually loop over the parameter grid and perform the grid search without relying on scikit-learn's GridSearchCV
In this code, I manually loop over the parameter grid and instantiate the ZZFeatureMap, VQC, and other components. We Can fit the model, predict on the test set, and calculate the accuracy score. If the current model achieves a better score than the previous best model, we update the best score and best parameters.

[AbdullahKazi500]

@GayatriVadaparty Great contact me on discord

[AbdullahKazi500]

from sklearn.model_selection import train_test_split
from qiskit import Aer
from qiskit_machine_learning.algorithms import VQC
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
from qiskit.algorithms.optimizers import SPSA
from sklearn.metrics import accuracy_score
from sklearn.model_selection import cross_val_score
from hyperopt import hp, tpe, Trials, fmin

# Generate synthetic dataset for demonstration
X, y = make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define the feature map
num_qubits = X.shape[1]
feature_map = ZZFeatureMap(num_qubits, reps=1)

# Define the objective function
def objective(params):
    optimizer = SPSA(maxiter=100)
    quantum_instance = Aer.get_backend('qasm_simulator')

    model = VQC(feature_map, RealAmplitudes(), optimizer, quantum_instance=quantum_instance)
    model.set_params(**params)

    scores = cross_val_score(model, X_train, y_train, cv=5)
    avg_score = scores.mean()

    return -avg_score

# Define the search space for hyperparameters
param_space = {
    'n': hp.choice('n', [None, 1, 2]),  # Number of qubits for RealAmplitudes ansatz
    'quantum_instance': hp.choice('quantum_instance', [Aer.get_backend('qasm_simulator'), Aer.get_backend('statevector_simulator')])
}

# Perform hyperparameter tuning with Bayesian optimization
trials = Trials()
best = fmin(fn=objective, space=param_space, algo=tpe.suggest, max_evals=20, trials=trials)

# Get the best hyperparameters
best_params = {key: param_space[key][value] for key, value in best.items()}

# Train the VQC with the best hyperparameters
optimizer = SPSA(maxiter=100)
quantum_instance = Aer.get_backend('qasm_simulator')

model = VQC(feature_map, RealAmplitudes(), optimizer, quantum_instance=quantum_instance)
model.set_params(**best_params)
model.fit(X_train, y_train)

# Predict on the test set
y_pred = model.predict(X_test)

# Calculate accuracy score
score = accuracy_score(y_test, y_pred)

# Print the best hyperparameters and final score
print("Best hyperparameters:", best_params)
print("Final score:", score)

To perform hyperparameter tuning using Bayesian optimization with the Qiskit Machine Learning package, we can use the BayesianOptimization class provided by the skopt library, which is compatible with scikit-learn's GridSearchCV and RandomizedSearchCV In this code, I have first define the parameter search space using a dictionary called param_space, which includes the possible values for the hyperparameters. Then, define the feature map, create the VQC estimator, and instantiate the BayesSearchCV object, providing the VQC estimator and the parameter search space. We set the number of iterations (n_iter) and the number of cross-validation folds (cv).

Next, we call the fit method of the search object, passing the training data. The search object performs Bayesian optimization to find the best hyperparameters based on the specified search space.

After the hyperparameter tuning is complete, we can access the best hyperparameters using the best_params_ attribute of the search object and the corresponding best score using the best_score_ attribute.

Finally, we set the best hyperparameters to the VQC estimator, fit the model on the training data, predict on the test set, and calculate the accuracy score. I had some problems working on the TQDM part here