How can I implement a multi-task deep learning neural network?

In summary, the conversation discusses the process of creating a neural network for a multi-task learning problem. The code provided includes 6 inputs for 6 features, 2 hidden layers, and 5 outputs for 5-character string patterns. The dilemma mentioned is that the code is not working due to the shape of the data. It is suggested to refer to examples and resources such as Geron's book, "Hands on ML with ScikitLearn, Keras and Tensorflow 2nd edition," Neilson's online book, "Neural Networks and Deep Learning," and Burkiv's "100 page Machine Learning book." 3brown1blue's YouTube videos are also recommended for a better understanding of neural networks.
  • #1
user366312
Gold Member
89
3
TL;DR Summary
I have 3 classes (A, B, and C). I have 6 features.
These features represent a 5-character string pattern comprising of 3-classes(e.g. AABBC, etc.). So, that means, features are not directly dependent on classes.
Hence, I am confused about the solution.
I have 3 classes (A, B, and C).

I have 6 features:

features:
train_x = [[ 6.442  6.338  7.027  8.789 10.009 12.566]
           [ 6.338  7.027  5.338 10.009  8.122 11.217]
           [ 7.027  5.338  5.335  8.122  5.537  6.408]
           [ 5.338  5.335  5.659  5.537  5.241  7.043]]

These features represent a 5-character string pattern comprising of 3-classes(e.g. AABBC, etc.).

Let, a 5-character string pattern is one-hot encoded as follows:

one_hot_encoding:
train_z = [[0. 0. 1. 0. 0. 1. 0. 0. 1. 0. 0. 1. 1. 0. 0.]   
           [0. 0. 1. 0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0. 0.]
           [0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0. 0. 1. 0. 0.]   
           [0. 0. 1. 1. 0. 0. 1. 0. 0. 1. 0. 0. 0. 0. 1.]]

So, I think this is a Multi-task learning problem.

source_code:
   # there would be 6 inputs for 6 features
    inputs_tensor = keras.Input(shape=(FEATURES_COUNT,))

    # there would be 2 hidden layers
    hidden_layer_1 = keras.layers.Dense(LAYER_1_NEURON_COUNT, activation="relu")
    hidden_layer_2 = keras.layers.Dense(LAYER_2_NEURON_COUNT, activation="relu")

    # there would be 5 outputs for 5-characters
    # each o/p layer will have 3 neurons for 3 classes
    output_layer_1 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # 3 neuraons for 3 classes
    output_layer_2 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # -do-
    output_layer_3 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # -do-
    output_layer_4 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # -do-
    output_layer_5 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # -do-
    output_layer_6 = keras.layers.Dense(CLASSES_COUNT, activation='softmax')  # -do-

    # assembling the layers.
    x = hidden_layer_1(inputs_tensor)
    x = hidden_layer_2(x)
    # configuring the output
    output1 = output_layer_1(x)
    output2 = output_layer_2(x)
    output3 = output_layer_3(x)
    output4 = output_layer_4(x)
    output5 = output_layer_5(x)

    model = keras.Model(inputs=[inputs_tensor],
                        outputs=[output1, output2, output3, output4, output5],
                        name="functional_model")

    # model.summary()

    opt_function = keras.optimizers.SGD(lr=0.01, decay=1e-1, momentum=0.9, nesterov=True)

    model.compile(loss='categorical_crossentropy',
                  optimizer=opt_function,
                  metrics=['accuracy'])

    model.fit(
        tx, tz,
        epochs=EPOCHS,
        batch_size=BATCH_SIZE
    )

In the above source code, I am passing a 5-character string pattern as a 15-digit one-hot-encoded to be used as classes.
And, this is the dilemma for me.

Naturally, this is not working:

error:
C:\ProgramData\Miniconda3\envs\by_nn\python.exe C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py

using 2 points for training and 3 for validation
Traceback (most recent call last):
  File "C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py", line 63, in <module>
    tz = tf.concat([tf.constant(d, shape=(1, CLASSES_COUNT), dtype=np.float32) for d in train_z], 0)
  File "C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py", line 63, in <listcomp>
    tz = tf.concat([tf.constant(d, shape=(1, CLASSES_COUNT), dtype=np.float32) for d in train_z], 0)
  File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 263, in constant
    return _constant_impl(value, dtype, shape, name, verify_shape=False,
  File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 275, in _constant_impl
    return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
  File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 322, in _constant_eager_impl
    raise TypeError("Eager execution of tf.constant with unsupported shape "
TypeError: Eager execution of tf.constant with unsupported shape (value has 15 elements, shape is (1, 3) with 3 elements).

Process finished with exit code 1

How can I properly design my neural network?
 
Technology news on Phys.org
  • #2
Your question is very broad and unlikely to get any meaningful response On PF. My suggestion is to look for examples of this class of ML problems and see how these folks solved it.

in the end, you are going to need a course / book on ML and it’s practices to solve your problem.

Take a look at Geron’s book: Hands on ML with ScikitLearn, Keras and Tensorflow 2nd edition.

https://www.amazon.com/dp/1492032646/?tag=pfamazon01-20
 
  • #3
Neural Networks and Deep Learning is an excellent free online book by Michael Neilson. You can also find free online copies of the source code for everything discussed in the book.

http://neuralnetworksanddeeplearning.com/
 
  • Like
Likes jedishrfu
  • #4
There is also the 100 page Machine Learning book by Burkiv that is a try and buy license if search a bit vs buy it on Amazon.

In addition 3brown1blue of YouTube fame has a sequence of videos on understanding Neural Nets.
 
  • Like
Likes anorlunda

Related to How can I implement a multi-task deep learning neural network?

1. How do I choose the right architecture for a multi-task deep learning neural network?

Choosing the right architecture for a multi-task deep learning neural network depends on various factors such as the complexity of the tasks, the amount of data available, and the computational resources. It is recommended to start with a simple architecture and gradually increase the complexity based on the performance on the tasks.

2. How can I handle imbalanced data in a multi-task deep learning neural network?

Handling imbalanced data in a multi-task deep learning neural network can be done by using techniques such as data augmentation, class weighting, and oversampling. These techniques can help balance the data and improve the performance of the network.

3. What is the best way to train a multi-task deep learning neural network?

The best way to train a multi-task deep learning neural network is to use a combination of techniques such as transfer learning, early stopping, and regularization. Transfer learning can help leverage pre-trained models for better performance, while early stopping and regularization can prevent overfitting.

4. How can I evaluate the performance of a multi-task deep learning neural network?

Evaluating the performance of a multi-task deep learning neural network can be done by using metrics such as accuracy, precision, recall, and F1-score for classification tasks, and mean squared error or mean absolute error for regression tasks. It is also important to validate the results on a separate test dataset to ensure the generalizability of the model.

5. Can I use a multi-task deep learning neural network for any type of data?

Multi-task deep learning neural networks can be used for a wide range of data types such as text, images, audio, and video. However, the architecture and training process may vary depending on the type of data. It is important to understand the nature of the data and tailor the network accordingly for optimal performance.

Similar threads

  • Programming and Computer Science
Replies
2
Views
1K
  • Programming and Computer Science
Replies
7
Views
6K
  • Programming and Computer Science
Replies
3
Views
4K
Back
Top