Can we have inputs as vectors and not only scalars?

[fredericosantos]

Can we have forest.forward accept shape of [population_size, input_len, n_dims]?

I need to use vectors instead of scalars for my inputs, and if I do [population_size, input_len * n_dims] then, as an example with dim=2, my x0 and x1 will be independent.

Am I missing something?

[Kelvin-Doremi]

Thanks for your issues, please try to use the code below. Hope this helps! If you run into issues, just reply here.

# (input_len, n_dims) -> (pop_size, output_len, n_dims)
def vector_forward(self, x: Tensor) -> Tensor:
    assist_x = x.permute(1, 0) # (input_len, n_dims) -> (batch_size, input_len)
    assist_res = self.batch_forward(assist_x)
    res = assist_res.permute(0, 2, 1) # (pop_size, batch_size, output_len) -> (pop_size, output_len, n_dims)
    return res

[fredericosantos]

I appreciate your answer @Kelvin-Doremi . However I have some concerns. The code you provide is essentially just a permutation of the input, (input_len, n_dims) to (n_dims, input_len). For the purpose of this comment, let's assume n_dims = batch_size.

Take the simple example below:

positions = torch.randn([pop_size, batch_size])
velocities = torch.randn([pop_size, batch_size]) 

This will give me unique coordinates for each of my individual. However, batch_forward asks for (batch_size, input_len). So, for the batch_forward, we should set:

positions = torch.randn(batch_size)
velocities = torch.randn(batch_size) 
x = torch.stack(
    [
        positions,
        velocities,
    ],
    dim=1,
)

This will create a tensor x of shape (batch_size, input_len). If you agree with me so far, great!

Now, when we use batch_forward, we repeat this x tensor pop_size times, effectively having the same inputs for all individuals!

assist_x = x.repeat(self.pop_size, 1)

Shouldn't batch_forward accept instead (pop_size, batch_size, input_len)?

I can create a PR that changes the batch_forward to accept the shape (pop_size, batch_size, input_len) that will still create an input of pop_size * batch_size into tree_evaluate.

[Kelvin-Doremi]

You’re right. If each individual requires different inputs, then the batch_forward function should be adjusted.

Yes, you can try to create a PR to make batch_forward adaptive: when the input tensor is 3-dimensional, it can process it directly; when it is 2-dimensional, it can automatically expand it by repeating along the population size (pop_size).

That would make the function more flexible and better suited for both use cases.