How can I implement a multi-task deep learning neural network?

[user366312]

TL;DR

I have 3 classes (A, B, and C). I have 6 features.
These features represent a 5-character string pattern comprising of 3-classes(e.g. AABBC, etc.). So, that means, features are not directly dependent on classes.
Hence, I am confused about the solution.

I have 3 classes (A, B, and C).

I have 6 features:

[CODE title="features"]train_x = [[ 6.442 6.338 7.027 8.789 10.009 12.566]
[ 6.338 7.027 5.338 10.009 8.122 11.217]
[ 7.027 5.338 5.335 8.122 5.537 6.408]
[ 5.338 5.335 5.659 5.537 5.241 7.043]][/CODE]

These features represent a 5-character string pattern comprising of 3-classes(e.g. AABBC, etc.).

Let, a 5-character string pattern is one-hot encoded as follows:

[CODE title="one_hot_encoding"]train_z = [[0. 0. 1. 0. 0. 1. 0. 0. 1. 0. 0. 1. 1. 0. 0.]
[0. 0. 1. 0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0. 0.]
[0. 0. 1. 0. 0. 1. 1. 0. 0. 1. 0. 0. 1. 0. 0.]
[0. 0. 1. 1. 0. 0. 1. 0. 0. 1. 0. 0. 0. 0. 1.]][/CODE]

So, I think this is a Multi-task learning problem.

[CODE lang="python" title="source_code"] # there would be 6 inputs for 6 features
inputs_tensor = keras.Input(shape=(FEATURES_COUNT,))

# there would be 2 hidden layers
hidden_layer_1 = keras.layers.Dense(LAYER_1_NEURON_COUNT, activation="relu")
hidden_layer_2 = keras.layers.Dense(LAYER_2_NEURON_COUNT, activation="relu")

# there would be 5 outputs for 5-characters
# each o/p layer will have 3 neurons for 3 classes
output_layer_1 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # 3 neuraons for 3 classes
output_layer_2 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # -do-
output_layer_3 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # -do-
output_layer_4 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # -do-
output_layer_5 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # -do-
output_layer_6 = keras.layers.Dense(CLASSES_COUNT, activation='softmax') # -do-

# assembling the layers.
x = hidden_layer_1(inputs_tensor)
x = hidden_layer_2(x)
# configuring the output
output1 = output_layer_1(x)
output2 = output_layer_2(x)
output3 = output_layer_3(x)
output4 = output_layer_4(x)
output5 = output_layer_5(x)

model = keras.Model(inputs=[inputs_tensor],
outputs=[output1, output2, output3, output4, output5],
name="functional_model")

# model.summary()

opt_function = keras.optimizers.SGD(lr=0.01, decay=1e-1, momentum=0.9, nesterov=True)

model.compile(loss='categorical_crossentropy',
optimizer=opt_function,
metrics=['accuracy'])

model.fit(
tx, tz,
epochs=EPOCHS,
batch_size=BATCH_SIZE
)[/CODE]

In the above source code, I am passing a 5-character string pattern as a 15-digit one-hot-encoded to be used as classes.
And, this is the dilemma for me.

Naturally, this is not working:

[CODE title="error"]C:\ProgramData\Miniconda3\envs\by_nn\python.exe C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py

using 2 points for training and 3 for validation
Traceback (most recent call last):
File "C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py", line 63, in <module>
tz = tf.concat([tf.constant(d, shape=(1, CLASSES_COUNT), dtype=np.float32) for d in train_z], 0)
File "C:/Users/pc/source/repos/OneHotEncodingLayer__test/by_nn_k____5_outputs.py", line 63, in <listcomp>
tz = tf.concat([tf.constant(d, shape=(1, CLASSES_COUNT), dtype=np.float32) for d in train_z], 0)
File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 263, in constant
return _constant_impl(value, dtype, shape, name, verify_shape=False,
File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 275, in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
File "C:\ProgramData\Miniconda3\envs\by_nn\lib\site-packages\tensorflow\python\framework\constant_op.py", line 322, in _constant_eager_impl
raise TypeError("Eager execution of tf.constant with unsupported shape "
TypeError: Eager execution of tf.constant with unsupported shape (value has 15 elements, shape is (1, 3) with 3 elements).

Process finished with exit code 1[/CODE]

How can I properly design my neural network?

 

[jedishrfu]

Your question is very broad and unlikely to get any meaningful response On PF. My suggestion is to look for examples of this class of ML problems and see how these folks solved it.

in the end, you are going to need a course / book on ML and it’s practices to solve your problem.

Take a look at Geron’s book: Hands on ML with ScikitLearn, Keras and Tensorflow 2nd edition.

https://www.amazon.com/dp/1492032646/?tag=pfamazon01-20

 

[anorlunda]

Neural Networks and Deep Learning is an excellent free online book by Michael Neilson. You can also find free online copies of the source code for everything discussed in the book.

http://neuralnetworksanddeeplearning.com/

 

[jedishrfu]

There is also the 100 page Machine Learning book by Burkiv that is a try and buy license if search a bit vs buy it on Amazon.

In addition 3brown1blue of YouTube fame has a sequence of videos on understanding Neural Nets.