Can we build two different `PQC` or `ControlledPQC` layers but sharing same weights (or `trainable_weights`)?

[Shuhul24]

I have built a custom tf.keras.layers.Layer layer having a quantum circuit which is ControlledPQC. But now that I have to use two different models but at some point I want them to share the same trainable weights, i.e., say model H1 have a quantum circuit whose weights gets changed after training (say, for some number of epochs). After a certain number of epochs, I need the same weight values to be for, say, another model H2 which has the same circuit (using ControlledPQC) but needs to have same weight values as that of model H1 quantum circuit. I saw the documentation which consists of tf.keras.layers.Embedding but haven't got any idea how it can be used. Can you help me figure this out?

[lockwo]

Should just be a matter of h2.set_weights(h1.get_weights()) whenever you want to do this (in training or whatever epochs or whatnot). I just tested that with H1 being a PQC based model, and H2 with a ControlledPQC custom layer and it worked (although I had to do some reshaping, specifically h2.set_weights(tf.expand_dims(h1.get_weights()[0], axis=0)])). I'm pretty sure they don't have to even be the same circuits at all, but I only tested with same circuits.

[Shuhul24]

Thanks for the reply!
Actually in my case, I have got 2 circuits attached in one ControlledPQC inside tf.keras.layers.Layer and out of those 2 circuits, one circuit is as ControlledPQC, but the issue is that I can't return the parameters of the same circuit from the combined ControlledPQC as it is a tf.Variable value. Can you help me out on how this can be done?

So its like, circuit H1 -> state -> H2 as ControlledPQC in, say, layer1 (by creating a subclass with tf.keras.layers.Layer module) and circuit H1 -> observable as ControlledPQC in, say, layer2 (similarly built with tf.keras.layers.Layer). Now in this case, layer2 updates similarly as H1 updates itself in layer1. But I can't take the weights from layer1 of H1 and integrate it into layer2. I tried using get_weights(), but I can't slice out of it as weights are tf.Variable.

[lockwo]

I see, that this was all happening in the custom layer was unclear. Here is an example of what I think you want, which can be achieved by just indexing the weights. I made up circuits H1 and H2 (which are different here but could be the same), then made one controlled PQC for H1 -> H2 and one for H1 -> obs. Then when calling it, you can just take the H1 params you care about for the second PQC. I tested this and it runs, and seems gradable. I verified this, as the gradients for the H2 params are all zero when using the H1 obs.

import tensorflow as tf
import tensorflow_quantum as tfq 
import numpy as np
import cirq
import sympy

class Combined(tf.keras.layers.Layer):
    def __init__(self) -> None:
        super(Combined, self).__init__()
        self.num_params = 4 * 2
        self.qubits = [cirq.GridQubit(0, i) for i in range(4)]
        self.theta = tf.Variable(initial_value=np.random.uniform(0, 2 * np.pi, (1, self.num_params)), dtype="float32", trainable=True)
        self.params = sympy.symbols("params0:%d"%self.num_params)
        self.h1_to_h2_to_obs = tfq.layers.ControlledPQC(self.h1(self.params[:4]) + self.h2(self.params[4:]), [cirq.Z(self.qubits[0])])
        self.h1_to_obs = tfq.layers.ControlledPQC(self.h1(self.params[:4]), [cirq.Z(self.qubits[0])])

    def h1(self, params):
        c = cirq.Circuit()
        for i in range(len(self.qubits)):
          c += cirq.ry(params[i]).on(self.qubits[i])
        return c

    def h2(self, params):
        c = cirq.Circuit()
        for i in range(len(self.qubits)):
          c += cirq.rx(params[i]).on(self.qubits[i])
        return c

    def call(self, inputs):
        h1h2obs = self.h1_to_h2_to_obs([inputs, self.theta])
        h1obs = self.h1_to_obs([inputs, self.theta[:, :4]])
        return h1h2obs, h1obs


l = Combined()
inputs = tfq.convert_to_tensor([cirq.Circuit()])

with tf.GradientTape() as tape:
  value, energy = l(inputs)

grads = tape.gradient(energy, l.trainable_variables)
print(grads)

Output:

[<tf.Tensor: shape=(1, 8), dtype=float32, numpy=
array([[-3.6007202e-01,  4.1406602e-06,  2.5118254e-06,  4.3869954e-06,
         0.0000000e+00,  0.0000000e+00,  0.0000000e+00,  0.0000000e+00]],
      dtype=float32)>]

[Shuhul24]

Thanks for the reply!
One last thing. Will the parameters of H1 circuit change simultaneously if I apply gradients on value and energy both? Or will it cause some sort of error? I mean that the parameter values will be same for H1 circuit in both cases when I will use opt.apply_gradient()?

[lockwo]

If you apply gradients sequentially (i.e. take gradient for value, apply gradient, take gradient for energy, apply it), it will work and H1 params (which are just part of the overall theta Variable) will be changed both times (as both value depend on those parameters). If you mean, your "loss" is actually a combination of those values (not applying the gradients for each), then it will still work but H1 params (well all theta params) will be changes only once. TF autograd will deal with exactly how to compute it. See for example, these two situations in order (same code as above):

l = Combined()
inputs = tfq.convert_to_tensor([cirq.Circuit()])

opt = tf.keras.optimizers.SGD(learning_rate=1.0)

with tf.GradientTape(persistent=True) as tape:
  value, energy = l(inputs)

print("Init", l.trainable_variables)
grads = tape.gradient(energy, l.trainable_variables)
opt.apply_gradients(zip(grads, l.trainable_variables))
print("Params[0] (H1) should change", l.trainable_variables)
grads = tape.gradient(value, l.trainable_variables)
opt.apply_gradients(zip(grads, l.trainable_variables))
print("Params[0] (H1) and Params[4] (H2) should change", l.trainable_variables)

with tf.GradientTape() as tape:
  value, energy = l(inputs)
  loss = value + energy

grads = tape.gradient(loss, l.trainable_variables)
opt.apply_gradients(zip(grads, l.trainable_variables))
print("Both grads", l.trainable_variables)

Results in:

Init [<tf.Variable 'Variable:0' shape=(1, 8) dtype=float32, numpy=
array([[0.25698686, 5.79846   , 4.4762807 , 5.869993  , 2.9522095 ,
        3.7284074 , 4.629468  , 1.5974815 ]], dtype=float32)>]
Params[0] (H1) should change [<tf.Variable 'Variable:0' shape=(1, 8) dtype=float32, numpy=
array([[0.5111524, 5.7984576, 4.4762874, 5.869991 , 2.9522095, 3.7284074,
        4.629468 , 1.5974815]], dtype=float32)>]
Params[0] (H1) and Params[4] (H2) should change [<tf.Variable 'Variable:0' shape=(1, 8) dtype=float32, numpy=
array([[0.03072384, 5.79846   , 4.4762893 , 5.87      , 3.1163952 ,
        3.7284098 , 4.629465  , 1.5974805 ]], dtype=float32)>]
Both grads [<tf.Variable 'Variable:0' shape=(1, 8) dtype=float32, numpy=
array([[0.03073359, 5.79846   , 4.4762893 , 5.87      , 3.1415799 ,
        3.728407  , 4.629471  , 1.597479  ]], dtype=float32)>]

[Shuhul24]

Thanks!
But how can I extract the parameters of just H1 circuit, because by using get_weights() I am getting the parameters of H1 and H2 circuit combined. Also, the combined circuit parameters are inter-twined (I guess that's because ControlledPQC does it internally). Because if we can extract the trainable_variables of H1 and H2 separately, then I guess using tape.gradient and opt.apply_gradients separately for separate circuits would make sense, right?

[lockwo]

You can access the H1 params just by indexing the layer weights (as I did in the call function). However, if you are taking the gradient and the function only depends on them (I.e. energy) then the gradient will be 0 elsewhere so you wouldn't have to specifically select for them.

You could extract each by just indexing the trainable variables.

It makes sense that H1 could be separate (but you don't need to index as I mentioned above), but the other function depends on both H1 params and H2 params. You of course could just update the H2 params if that is what you want.

[Shuhul24]

How I can update the parameters of just H2?

[lockwo]

Probably easiest to just make the H1 and H2 params separate variables since you are using them a lot independently it seems. The following does that, allowing just params of H2 to be updates easily.

import tensorflow as tf
import tensorflow_quantum as tfq 
import numpy as np
import cirq
import sympy

class Combined(tf.keras.layers.Layer):
    def __init__(self) -> None:
        super(Combined, self).__init__()
        self.num_params = 4 * 2
        self.qubits = [cirq.GridQubit(0, i) for i in range(4)]
        self.h1_weights = tf.Variable(initial_value=np.random.uniform(0, 2 * np.pi, (1, self.num_params // 2)), dtype="float32", trainable=True)
        self.h2_weights = tf.Variable(initial_value=np.random.uniform(0, 2 * np.pi, (1, self.num_params // 2)), dtype="float32", trainable=True)
        self.params = sympy.symbols("params0:%d"%self.num_params)
        self.h1_to_h2_to_obs = tfq.layers.ControlledPQC(self.h1(self.params[:4]) + self.h2(self.params[4:]), [cirq.Z(self.qubits[0])])
        self.h1_to_obs = tfq.layers.ControlledPQC(self.h1(self.params[:4]), [cirq.Z(self.qubits[0])])

    def h1(self, params):
        c = cirq.Circuit()
        for i in range(len(self.qubits)):
          c += cirq.ry(params[i]).on(self.qubits[i])
        return c

    def h2(self, params):
        c = cirq.Circuit()
        for i in range(len(self.qubits)):
          c += cirq.rx(params[i]).on(self.qubits[i])
        return c

    def __call__(self, inputs):
        h1h2obs = self.h1_to_h2_to_obs([inputs, tf.concat([self.h1_weights, self.h2_weights], axis=1)])
        h1obs = self.h1_to_obs([inputs, self.h1_weights])
        return h1h2obs, h1obs


l = Combined()
inputs = tfq.convert_to_tensor([cirq.Circuit()])

opt = tf.keras.optimizers.SGD(learning_rate=1.0)

with tf.GradientTape(persistent=True) as tape:
  value, energy = l(inputs)

print("Init", l.trainable_variables)
grads = tape.gradient(value, [l.trainable_variables[1]])
print(grads)
opt.apply_gradients(zip(grads, [l.trainable_variables[1]]))
print("Params[:4] (H1) Same and Params[4:] (H2) should change", l.trainable_variables)

Result

Init [<tf.Variable 'Variable:0' shape=(1, 4) dtype=float32, numpy=array([[4.926237 , 0.6610151, 4.226024 , 3.861875 ]], dtype=float32)>, <tf.Variable 'Variable:0' shape=(1, 4) dtype=float32, numpy=array([[2.571854  , 0.9393705 , 0.01724625, 1.6942394 ]], dtype=float32)>]
[<tf.Tensor: shape=(1, 4), dtype=float32, numpy=
array([[-1.1447796e-01, -4.6193600e-07, -7.8300945e-07, -9.3504786e-07]],
      dtype=float32)>]
Params[:4] (H1) Same and Params[4:] (H2) should change [<tf.Variable 'Variable:0' shape=(1, 4) dtype=float32, numpy=array([[4.926237 , 0.6610151, 4.226024 , 3.861875 ]], dtype=float32)>, <tf.Variable 'Variable:0' shape=(1, 4) dtype=float32, numpy=array([[2.686332  , 0.939371  , 0.01724703, 1.6942403 ]], dtype=float32)>]

[Shuhul24]

Thanks for the reply!
So I want to clear out what you just wrote above. So when you doing the tape.gradient with repsect to l.trainable_variables[1], you are basically evaluating gradient of value with respect to just the weights of h2, right?Similarily, for the opt.apply_gradients?

[lockwo]

Yes

[Shuhul24]

I wanted to know that is there any difference in the circuit h1 in self.h1h2obs and self.h1obs? I mean, since both of these circuits are working in two different tfq.layers.ControlledPQC, is there any possibility that these two circuits operate differently, say, the output density matrix of self.h1obs (which we can not evaluate) is different from the output density matrix of h1 in self.h1h2obs (also, which we cannot observe)?

[lockwo]

The same weights and circuits are fed into the controlled PQC so they should be the same in the call function. Since the h1 circuit is pre defined and the same weights are used, I don't think there is a way for them to be different statevectors. They could operate differently, but I am feeding them the same circuit and parameters (for h1), so they shouldn't be different (of course the H2 at the end changes one of them). But if the flow is input -> h1 -> state() -> op and input -> h1 -> state() -> h2 -> op, and h1 is the same structure and params in both, then state is the same in both (assuming simulator, h1 might encounter different noise on a real hardware system on a second run).

[Shuhul24]

Ok, got it! Can you please suggest a way to evaluate the probability of each state from a tfq.layers.ControlledPQC? I mean like in above example, I am using 2 qubit circuit for h1 and h1h2obs. So I need the porbability of each state (in this case |00>, |01>, |10> and |11>) after h1 and h1h2obs. Is it possible in ControlledPQC or is there any other way you may suggest?

[lockwo]

Sure, you can just use a state layer (https://www.tensorflow.org/quantum/api_docs/python/tfq/layers/State). Just put in the circuit and parameters and it will give you the state. This isn't from a controlled PQC (since it is tied to observables) but it will generate your state.