Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory Error while using VQC #837

Open
Ilyaant opened this issue Oct 3, 2024 · 4 comments
Open

Memory Error while using VQC #837

Ilyaant opened this issue Oct 3, 2024 · 4 comments
Labels
type: question 🙋 Question about the code or its uses

Comments

@Ilyaant
Copy link

Ilyaant commented Oct 3, 2024

Hello! While fitting VQC on a (3599, 20) training dataset I faced the following exception:

MemoryError: Unable to allocate 1.64 GiB for an array with shape (1048576,) and data type <U420

Could you please explain the error to me? I'm working on a machine with 120 GB of RAM and don't understand how the program cannot allocate 1.64 GB. Is there a way solve the problem?

The idea I have is to divide the dataset into smaller parts and fit the VQC on them. But here's another question: can I call the fit() method multiple times on different data chunks, or will the model be training from scratch each time?

Still, I don't like this idea, I'd want to do something with the memory usage limitation. Thank you!
@edoaltamura edoaltamura added the type: question 🙋 Question about the code or its uses label Oct 8, 2024
@edoaltamura
Copy link
Collaborator

Hi @Ilyaant, thanks for your question. Could you please share the part of your code that triggers the allocation?
This sometimes happens when trying to accidentally allocate eg 2**1048576 rather than 1048576 bits, but somehow NumPy recognises it as of reasonable size.

@Ilyaant
Copy link
Author

Ilyaant commented Oct 8, 2024

Hello @edoaltamura, thank you very much for your answer. The code snippet that I run is:

num_features = X_train.shape[1]

feature_map = ZZFeatureMap(feature_dimension=num_features, reps=1)
ansatz = RealAmplitudes(num_qubits=num_features, reps=3)
optimizer = COBYLA(maxiter=100)

vqc = VQC(
    feature_map=feature_map,
    ansatz=ansatz,
    optimizer=optimizer
)

vqc.fit(X_train, y_train.to_numpy())

The vqc.fit(X_train, y_train.to_numpy()) is the exact row that causes the exception.

The full description of the exception:

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\neural_networks\sampler_qnn.py:393, in SamplerQNN._forward(self, input_data, weights)
    [392](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:392) try:
--> [393](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:393)     results = job.result()
    [394](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:394) except Exception as exc:

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit\primitives\primitive_job.py:51, in PrimitiveJob.result(self)
     [50](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/primitive_job.py:50) self._check_submitted()
---> [51](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/primitive_job.py:51) return self._future.result()

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\_base.py:456, in Future.result(self, timeout)
    [455](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:455) elif self._state == FINISHED:
--> [456](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:456)     return self.__get_result()
    [457](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:457) else:

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\_base.py:401, in Future.__get_result(self)
    [400](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:400) try:
--> [401](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:401)     raise self._exception
    [402](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:402) finally:
    [403](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/_base.py:403)     # Break a reference cycle with the exception in self._exception

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py:58, in _WorkItem.run(self)
     [57](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/thread.py:57) try:
---> [58](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/thread.py:58)     result = self.fn(*self.args, **self.kwargs)
     [59](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/concurrent/futures/thread.py:59) except BaseException as exc:

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit\primitives\sampler.py:109, in Sampler._call(self, circuits, parameter_values, **run_options)
    [107](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:107)     qargs_list.append(self._qargs_list[i])
    [108](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:108) probabilities = [
--> [109](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:109)     Statevector(bound_circuit_to_instruction(circ)).probabilities_dict(
    [110](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:110)         qargs=qargs, decimals=16
    [111](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:111)     )
    [112](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:112)     for circ, qargs in zip(bound_circuits, qargs_list)
    [113](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:113) ]
    [114](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/primitives/sampler.py:114) if shots is not None:

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit\quantum_info\states\quantum_state.py:240, in QuantumState.probabilities_dict(self, qargs, decimals)
    [221](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:221) """Return the subsystem measurement probability dictionary.
    [222](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:222) 
    [223](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:223) Measurement probabilities are with respect to measurement in the
   (...)
    [238](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:238)     dict: The measurement probabilities in dict (ket) form.
    [239](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:239) """
--> [240](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:240) return self._vector_to_dict(
    [241](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:241)     self.probabilities(qargs=qargs, decimals=decimals),
    [242](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:242)     self.dims(qargs),
    [243](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:243)     string_labels=True,
    [244](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:244) )

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit\quantum_info\states\quantum_state.py:399, in QuantumState._vector_to_dict(vec, dims, decimals, string_labels)
    [398](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:398) # Convert to ket tuple based on subsystem dimensions
--> [399](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:399) kets = QuantumState._index_to_ket_array(inds, dims, string_labels=string_labels)
    [401](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:401) # Make dict of tuples

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit\quantum_info\states\quantum_state.py:371, in QuantumState._index_to_ket_array(inds, dims, string_labels)
    [370](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:370)         str_kets = np.char.add(",", str_kets)
--> [371](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:371)     str_kets = np.char.add(row, str_kets)
    [372](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit/quantum_info/states/quantum_state.py:372) return str_kets.T

MemoryError: Unable to allocate 1.64 GiB for an array with shape (1048576,) and data type <U420

The above exception was the direct cause of the following exception:

QiskitMachineLearningError                Traceback (most recent call last)
Cell In[12], [line 14](vscode-notebook-cell:?execution_count=12&line=14)
      [7](vscode-notebook-cell:?execution_count=12&line=7) vqc = VQC(
      [8](vscode-notebook-cell:?execution_count=12&line=8)     feature_map=feature_map,
      [9](vscode-notebook-cell:?execution_count=12&line=9)     ansatz=ansatz,
     [10](vscode-notebook-cell:?execution_count=12&line=10)     optimizer=optimizer
     [11](vscode-notebook-cell:?execution_count=12&line=11) )
     [13](vscode-notebook-cell:?execution_count=12&line=13) # vqc.fit(X_train.to_numpy(), y_train.to_numpy())
---> [14](vscode-notebook-cell:?execution_count=12&line=14) vqc.fit(X_train, y_train.to_numpy())

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\algorithms\trainable_model.py:199, in TrainableModel.fit(self, X, y)
    [196](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:196) if not self._warm_start:
    [197](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:197)     self._fit_result = None
--> [199](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:199) self._fit_result = self._fit_internal(X, y)
    [200](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:200) return self

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\algorithms\classifiers\vqc.py:167, in VQC._fit_internal(self, X, y)
    [164](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/classifiers/vqc.py:164)     self._neural_network.set_interpret(self._get_interpret(num_classes), num_classes)
    [166](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/classifiers/vqc.py:166) function = self._create_objective(X, y)
--> [167](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/classifiers/vqc.py:167) return self._minimize(function)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\algorithms\trainable_model.py:295, in TrainableModel._minimize(self, function)
    [291](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:291)     optimizer_result = self._optimizer(
    [292](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:292)         fun=objective, x0=initial_point, jac=function.gradient
    [293](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:293)     )
    [294](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:294) else:
--> [295](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:295)     optimizer_result = self._optimizer.minimize(
    [296](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:296)         fun=objective,
    [297](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:297)         x0=initial_point,
    [298](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:298)         jac=function.gradient,
    [299](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:299)     )
    [300](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/trainable_model.py:300) return optimizer_result

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_algorithms\optimizers\scipy_optimizer.py:148, in SciPyOptimizer.minimize(self, fun, x0, jac, bounds)
    [145](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:145)     swapped_deprecated_args = True
    [146](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:146)     self._options["maxfun"] = self._options.pop("maxiter")
--> [148](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:148) raw_result = minimize(
    [149](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:149)     fun=fun,
    [150](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:150)     x0=x0,
    [151](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:151)     method=self._method,
    [152](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:152)     jac=jac,
    [153](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:153)     bounds=bounds,
    [154](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:154)     options=self._options,
    [155](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:155)     **self._kwargs,
    [156](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:156) )
    [157](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:157) if swapped_deprecated_args:
    [158](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_algorithms/optimizers/scipy_optimizer.py:158)     self._options["maxiter"] = self._options.pop("maxfun")

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_minimize.py:737, in minimize(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)
    [734](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:734)     res = _minimize_tnc(fun, x0, args, jac, bounds, callback=callback,
    [735](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:735)                         **options)
    [736](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:736) elif meth == 'cobyla':
--> [737](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:737)     res = _minimize_cobyla(fun, x0, args, constraints, callback=callback,
    [738](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:738)                            bounds=bounds, **options)
    [739](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:739) elif meth == 'cobyqa':
    [740](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:740)     res = _minimize_cobyqa(fun, x0, args, bounds, constraints, callback,
    [741](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_minimize.py:741)                            **options)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_cobyla_py.py:35, in synchronized.<locals>.wrapper(*args, **kwargs)
     [32](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:32) @functools.wraps(func)
     [33](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:33) def wrapper(*args, **kwargs):
     [34](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:34)     with _module_lock:
---> [35](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:35)         return func(*args, **kwargs)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_cobyla_py.py:278, in _minimize_cobyla(fun, x0, args, constraints, rhobeg, tol, maxiter, disp, catol, callback, bounds, **unknown_options)
    [275](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:275) def _jac(x, *args):
    [276](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:276)     return None
--> [278](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:278) sf = _prepare_scalar_function(fun, x0, args=args, jac=_jac)
    [280](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:280) def calcfc(x, con):
    [281](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_cobyla_py.py:281)     f = sf.fun(x)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_optimize.py:288, in _prepare_scalar_function(fun, x0, jac, args, bounds, epsilon, finite_diff_rel_step, hess)
    [284](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:284)     bounds = (-np.inf, np.inf)
    [286](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:286) # ScalarFunction caches. Reuse of fun(x) during grad
    [287](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:287) # calculation reduces overall function evaluations.
--> [288](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:288) sf = ScalarFunction(fun, x0, args, grad, hess,
    [289](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:289)                     finite_diff_rel_step, bounds, epsilon=epsilon)
    [291](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_optimize.py:291) return sf

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_differentiable_functions.py:222, in ScalarFunction.__init__(self, fun, x0, args, grad, hess, finite_diff_rel_step, finite_diff_bounds, epsilon)
    [219](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:219)     finite_diff_options["as_linear_operator"] = True
    [221](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:221) # Initial function evaluation
--> [222](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:222) self._update_fun()
    [224](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:224) # Initial gradient evaluation
    [225](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:225) self._wrapped_grad, self._ngev = _wrapper_grad(
    [226](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:226)     grad,
    [227](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:227)     fun=self._wrapped_fun,
    [228](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:228)     args=args,
    [229](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:229)     finite_diff_options=finite_diff_options
    [230](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:230) )

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_differentiable_functions.py:294, in ScalarFunction._update_fun(self)
    [292](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:292) def _update_fun(self):
    [293](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:293)     if not self.f_updated:
--> [294](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:294)         fx = self._wrapped_fun(self.x)
    [295](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:295)         if fx < self._lowest_f:
    [296](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:296)             self._lowest_x = self.x

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\scipy\optimize\_differentiable_functions.py:20, in _wrapper_fun.<locals>.wrapped(x)
     [16](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:16) ncalls[0] += 1
     [17](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:17) # Send a copy because the user may overwrite it.
     [18](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:18) # Overwriting results in undefined behaviour because
     [19](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:19) # fun(self.x) will change self.x, with the two no longer linked.
---> [20](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:20) fx = fun(np.copy(x), *args)
     [21](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:21) # Make sure the function returns a true scalar
     [22](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/scipy/optimize/_differentiable_functions.py:22) if not np.isscalar(fx):

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\algorithms\objective_functions.py:191, in OneHotObjectiveFunction.objective(self, weights)
    [189](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:189) def objective(self, weights: np.ndarray) -> float:
    [190](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:190)     # probabilities is of shape (N, num_outputs)
--> [191](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:191)     probs = self._neural_network_forward(weights)
    [192](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:192)     # float(...) is for mypy compliance
    [193](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:193)     value = float(np.sum(self._loss(probs, self._y)) / self._num_samples)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\algorithms\objective_functions.py:102, in ObjectiveFunction._neural_network_forward(self, weights)
     [97](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:97) # if we get the same weights, we don't compute the forward pass again.
     [98](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:98) if self._last_forward_weights is None or (
     [99](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:99)     not np.all(np.isclose(weights, self._last_forward_weights))
    [100](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:100) ):
    [101](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:101)     # compute forward and cache the results for re-use in backward
--> [102](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:102)     self._last_forward = self._neural_network.forward(self._X, weights)
    [103](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:103)     # a copy avoids keeping a reference to the same array, so we are sure we have
    [104](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:104)     # different arrays on the next iteration.
    [105](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/algorithms/objective_functions.py:105)     self._last_forward_weights = np.copy(weights)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\neural_networks\neural_network.py:228, in NeuralNetwork.forward(self, input_data, weights)
    [226](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/neural_network.py:226) input_, shape = self._validate_input(input_data)
    [227](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/neural_network.py:227) weights_ = self._validate_weights(weights)
--> [228](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/neural_network.py:228) output_data = self._forward(input_, weights_)
    [229](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/neural_network.py:229) return self._validate_forward_output(output_data, shape)

File c:\Users\iantonov\AppData\Local\Programs\Python\Python312\Lib\site-packages\qiskit_machine_learning\neural_networks\sampler_qnn.py:395, in SamplerQNN._forward(self, input_data, weights)
    [393](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:393)     results = job.result()
    [394](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:394) except Exception as exc:
--> [395](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:395)     raise QiskitMachineLearningError("Sampler job failed.") from exc
    [396](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:396) result = self._postprocess(num_samples, results)
    [398](file:///C:/Users/iantonov/AppData/Local/Programs/Python/Python312/Lib/site-packages/qiskit_machine_learning/neural_networks/sampler_qnn.py:398) return result

QiskitMachineLearningError: 'Sampler job failed.'

@Ilyaant
Copy link
Author

Ilyaant commented Oct 16, 2024

@edoaltamura For now, it seems to me that the issue is inside the fit() method because the command y_train.to_numpy() itself runs without problems. Please, take a look when you have time. Thank you

@oscar-wallis
Copy link
Collaborator

oscar-wallis commented Nov 13, 2024 •
edited
Loading

Hi @Ilyaant , so I am not 100% sure why this is going wrong but it is worth noting that log2(1048576) = 20 exactly, so i am assuming that something is going wrong with your array size of 20. Try checking the shape of the arrays heading into the network / what happens to them when you run .to_numpy(). Otherwise, it is challenging for us to investigate this bug without access to the full script. I'd also try running VQC.score() with some of your data before fitting. This is essentially checking if the problem is VQC or the optimiser COBYLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: question 🙋 Question about the code or its uses
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Ilyaant @edoaltamura @oscar-wallis