How to get output from a layer during training in Keras?

In summary, the method model.fit() in Keras is used to train a neural network and the output from any hidden layer during training can be obtained by manipulating the training process using one element at a time. This can be implemented in the given code by modifying the model.fit() method and accessing the output of the desired layer during training.
  • #1
Atr cheema
69
0
In Keras, the method model.fit() is used to train the neural network. How can I get the output from any hidden layer during training? Consider following code where neural network is trained to add two time series
Python:
#multivariate data preparation
#multivariate multiple input cnn example
from numpy import array
from numpy import hstackfrom tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense
from tensorflow.python.keras.layers import Flatten
from tensorflow.python.keras.layers.convolutional import Conv1D
from tensorflow.python.keras.layers.convolutional import MaxPooling1D#split a multivariate sequence into samples
def split_sequences(sequences, n_steps):
    X, y = list(), list()
    for i in range(len(sequences)):
        # find the end of this pattern
        end_ix = i + n_steps
        # check if we are beyond the dataset
        if end_ix > len(sequences):
            break
        # gather input and output parts of the pattern
        seq_x, seq_y = sequences[i:end_ix, :-1], sequences[end_ix-1, -1]
        X.append(seq_x)
        y.append(seq_y)
    return array(X), array(y)

#define input sequence
dataset_len = 100
is1 = array([i*10+10 for i in range(dataset_len)])
is2 = array([i*10+15 for i in range(dataset_len)])

out_seq = array([is1[i]+is2[i]
                for i in range(len(is1))])
#convert to [rows, columns] structure
is1 = is1.reshape((len(is1), 1))
is2 = is2.reshape((len(is2), 1))

out_seq = out_seq.reshape((len(out_seq), 1))
#horizontally stack columns
dataset = hstack((is1, is2,
                  #is3,is4,is5,is6,is7,is8,is9,
                  out_seq))
#print('raw input data is : \n{}\n'.format(dataset[0:7]))

#choose a number of time steps
n_steps = 3
#convert into input/output
X, y = split_sequences(dataset, n_steps)

#print('given \n{} we want to predict {}\n'.format(X[0], y[0]))

#model input must have shape [samples, timesteps, features]

#the dataset knows the number of features, e.g. 2
n_features = X.shape[2]
#define model
model = Sequential()
conv1d = Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(n_steps, n_features))
model.add(conv1d)
mp1d = MaxPooling1D(pool_size=2)
model.add(mp1d)
fl = Flatten()
model.add(fl)
model.add(Dense(50, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(2, activation='relu'))
model.add(Dense(1))

#print('\nCompiling model\n')

model.compile(optimizer='adam', loss='mse')
#plot_model(model, to_file='multivariate_laos.png', show_shapes=True,)

#fit model
model.fit(X, y, epochs=1000, batch_size=None, verbose=0, shuffle=False, steps_per_epoch=None)##demonstrate prediction
x_input = array([[80, 85], [90, 95], [100, 105]])
x_input = x_input.reshape((1, n_steps, n_features))
yhat = model.predict(x_input, verbose=0)
print(yhat)`

I the above code, how can I get output of last hidden layer which is in this case dense layer, but I want the output during training of neural network. This means I want output every time a 'sample' is evaluated and its corresponding loss function is calculated.
 
Technology news on Phys.org
  • #3
Borg said:
All the answers in that post provide the way to get output once the training is finished but I want the output during the training process simultaneously. This might require some modification in model.fit method in training.py file but I am unable to find a solution so far. Can you tell me which line in Keras source code actually runs the tensorflow session during execution of model.fit() method. .
 
  • #4
I don't know - I'm pretty new at Keras as well. I know that it's mainly a wrapper around Tensorflow to make life easier but, if you want something else, you may have to build your model without it. Maybe @jedishrfu knows a way.
 
  • #5
The only thing i can think of is to manipulate your training using one element at a time.
 
  • #6
jedishrfu said:
The only thing i can think of is to manipulate your training using one element at a time.
Can you implement this for this example code? Thanks.
 
  • #7
Atr cheema said:
Can you implement this for this example code? Thanks.

Nope, the task is left to the student. :-)
 
  • #8
Atr cheema said:
In Keras, the method model.fit() is used to train the neural network. How can I get the output from any hidden layer during training? Consider following code where neural network is trained to add two time series
Python:
#multivariate data preparation
#multivariate multiple input cnn example
from numpy import array
from numpy import hstackfrom tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense
from tensorflow.python.keras.layers import Flatten
from tensorflow.python.keras.layers.convolutional import Conv1D
from tensorflow.python.keras.layers.convolutional import MaxPooling1D#split a multivariate sequence into samples
def split_sequences(sequences, n_steps):
    X, y = list(), list()
    for i in range(len(sequences)):
        # find the end of this pattern
        end_ix = i + n_steps
        # check if we are beyond the dataset
        if end_ix > len(sequences):
            break
        # gather input and output parts of the pattern
        seq_x, seq_y = sequences[i:end_ix, :-1], sequences[end_ix-1, -1]
        X.append(seq_x)
        y.append(seq_y)
    return array(X), array(y)

#define input sequence
dataset_len = 100
is1 = array([i*10+10 for i in range(dataset_len)])
is2 = array([i*10+15 for i in range(dataset_len)])

out_seq = array([is1[i]+is2[i]
                for i in range(len(is1))])
#convert to [rows, columns] structure
is1 = is1.reshape((len(is1), 1))
is2 = is2.reshape((len(is2), 1))

out_seq = out_seq.reshape((len(out_seq), 1))
#horizontally stack columns
dataset = hstack((is1, is2,
                  #is3,is4,is5,is6,is7,is8,is9,
                  out_seq))
#print('raw input data is : \n{}\n'.format(dataset[0:7]))

#choose a number of time steps
n_steps = 3
#convert into input/output
X, y = split_sequences(dataset, n_steps)

#print('given \n{} we want to predict {}\n'.format(X[0], y[0]))

#model input must have shape [samples, timesteps, features]

#the dataset knows the number of features, e.g. 2
n_features = X.shape[2]
#define model
model = Sequential()
conv1d = Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(n_steps, n_features))
model.add(conv1d)
mp1d = MaxPooling1D(pool_size=2)
model.add(mp1d)
fl = Flatten()
model.add(fl)
model.add(Dense(50, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(2, activation='relu'))
model.add(Dense(1))

#print('\nCompiling model\n')

model.compile(optimizer='adam', loss='mse')
#plot_model(model, to_file='multivariate_laos.png', show_shapes=True,)

#fit model
model.fit(X, y, epochs=1000, batch_size=None, verbose=0, shuffle=False, steps_per_epoch=None)##demonstrate prediction
x_input = array([[80, 85], [90, 95], [100, 105]])
x_input = x_input.reshape((1, n_steps, n_features))
yhat = model.predict(x_input, verbose=0)
print(yhat)`

I the above code, how can I get output of last hidden layer which is in this case dense layer, but I want the output during training of neural network. This means I want output every time a 'sample' is evaluated and its corresponding loss function is calculated.
Hi Atr cheema,
I have the same problem, did you find a solution for your question?
 

Related to How to get output from a layer during training in Keras?

1. How do I access the output from a specific layer during training in Keras?

In order to access the output from a specific layer during training in Keras, you can use the model.layers[index].output method. This will return the output from the layer at the specified index.

2. Can I get the output from multiple layers during training in Keras?

Yes, you can get the output from multiple layers during training in Keras by using the model.layers[index].output method for each layer you want to access. You can then store the outputs in a list or array for further use.

3. How can I visualize the output from a layer during training in Keras?

To visualize the output from a layer during training in Keras, you can use the model.predict() method. This will return the output from the specific layer and you can then plot it using a visualization library like Matplotlib or Seaborn.

4. Is it possible to modify the output from a layer during training in Keras?

Yes, you can modify the output from a layer during training in Keras by using the model.layers[index].output method to access the output and then applying any desired modifications to it. You can then use this modified output for further processing or visualization.

5. How can I get the output from a layer in a specific data batch during training in Keras?

In order to get the output from a layer in a specific data batch during training in Keras, you can use the model.predict() method with the batch of data as input. This will return the output from the layer for that specific batch of data.

Similar threads

  • Programming and Computer Science
Replies
2
Views
1K
  • Programming and Computer Science
Replies
3
Views
970
  • Programming and Computer Science
Replies
1
Views
1K
Back
Top