Combining feature vectors for a neural network

[themagiciant95]

Let's consider this scenario. I have two conceptually different video datasets, for example a dataset A composed of videos about cats and a dataset B composed of videos about houses. Now, I'm able to extract a feature vectors from both the samples of the datasets A and B, and I know that, each sample in the dataset A is related to one and only one sample in the dataset B and they belong to a specific class (there are only 2 classes).

For example:

Sample x1 AND sample y1 ---> Class 1
Sample x2 AND sample y2 ---> Class 2
Sample x3 AND sample y3 ---> Class 1
and so on...

If I extract the feature vectors from samples in both datasets , which is the best way to combine them in order to give a correct input to the classifier (for example a neural network) ?

feature vector v1 extracted from x1 + feature vector v1' extracted from y1 ---> input for classifier

I ask this because I suspect that neural networks only take one vector as input, while I have to combine two vectors

 

[lomidrevo]

I ask this because I suspect that neural networks only take one vector as input

why do you think so?

 

[themagiciant95]

why do you think so?

Hi, do you know neural networks that take more than 1 single vector as input ?

 

[lomidrevo]

Generally, the inputs to ANN can be seen as array of numbers, or vector if you wish. But on the other hand, any (finite) multidimensional array can be reshaped to a vector. So you can pass components of the two vectors "serialized" as components of a single vector, without loosing any information. If the size of your inputs vectors is the same, say N, the size of the input layer of your net will be just 2*N. In general, if you had M vectors of size N, your input layer would be of size M*N. For example, that's the case of ANNs that process grayscale images.

 

[themagiciant95]

Are you talking about concatenating vectors in order to create an appropriate input vector to the deep neural network ?

 

[lomidrevo]

Yes. Or reshape the matrix into vector.. it depends on how do you store the data.
It is useful to realize that at the beginning of training, when the network hasn't seen any instanes yet, the input features can be considered as "independent" variables, so the network doesn't really care about our representation of the data. It doesn't matter whether those are components of one vector or of M vectors, or of any other structure. It is the goal of the training to find the hidden relationships between data.
We talk about supervised learning, right?

 

[lomidrevo]

do you work in some ML framework, or do you try to code it from scratch? I definitely recommend the first option

 

[themagiciant95]

I would like to try RBF-ANN as the classifier.

Let me explain better. For the moment ignore my original question.
I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has a high dimension (about 16000), and I would like to classify these vectors using an RBF-ANN (there are only 2 possible classes).

Is it a problem if the input vector to the RBF-ANN has a high dimension (16000) ? If yes, any way to deal with it ?

 

[lomidrevo]

Honestly I am not familiar with RBF-ANN. Just for my curiosity, do you have a specific reason to use it?

Is it a problem if the input vector to the RBF-ANN has a high dimension (16000) ?

If you haven't tried it to run yet, give it a chance. Maybe there won't be any problems.
Eventually, the training might become very slow, or you might have issues with insufficient memory. Also this phenomena could be a problem:
https://en.wikipedia.org/wiki/Curse_of_dimensionality

If yes, any way to deal with it ?

Maybe to try some of these techniques?:
https://en.wikipedia.org/wiki/Dimensionality_reduction
(no experience on my side...)

 

[themagiciant95]

Thank you infinitely. I'll try