Can CNNs Be Tuned to Enhance Specific Texture Parameters in Image Simulation?

[emmasaunders12]

Hi I am using a convolution neural network (with inversion) to simulate images with the same "texture" as the input image, using a random image to start with. The activations of the CNN are first learned with an example or source image. A cost function then minimizes the difference between the simulated features and the source features. The new simulated image has the same texture profile as the source image. The method is described here.

http://bethgelab.org/deeptextures/

My question is, is there a way to somehow tune the network to output a simulated image that has a greater texture parameter, such as skew or energy, or perhaps a way to incorporate these parametrized textures into the cost function.

I'm new to CNN's so sorry if this is trivial

Thanks

Emma

 

[lofgran]

,

Thank you for sharing your work and question with us. Your approach of using a convolutional neural network (CNN) with inversion to simulate images with the same texture as the input image is very interesting. It is a unique and creative way to generate new images with desired texture profiles.

To answer your question, yes, there are ways to incorporate texture parameters into the cost function and tune the network to output images with specific texture parameters. One approach is to use a multi-objective optimization technique, where the cost function includes both the similarity between the simulated and source features, as well as the desired texture parameters. This way, the network will not only learn to generate images with similar features, but also with the desired texture parameters.

Another approach is to add additional layers to the CNN that specifically learn to capture and manipulate texture parameters. These layers can be trained separately or simultaneously with the rest of the network, depending on the complexity of the desired texture parameters.

I recommend looking into existing research on texture synthesis and texture parameterization, as they may provide valuable insights and techniques for incorporating texture parameters into your CNN. Also, don't hesitate to reach out to other researchers in the field for collaborations and discussions.

Best of luck with your work!