Have you heard of the VDSR upscaling algorithm?

[bhobba]

Hi All

Here is an interesting algorithm for up-scaling images:
https://arxiv.org/pdf/1511.04587.pdf

It has 20 3x3x64 (some I have read use 3x3x16) convolutions. Now my understanding of convolutions means, the first produces 64 images from the 64 filers, then in the second convolution each of the 64 images has 64 filters applied etc so the last convolution would have 20^64 images. That's simply impossible memory wise. Either my understanding of convolutions is wrong or something else is going on. Maybe because its producing a difference output between the up-scaled image and bi-cubic up-scaling nearly all elements are zero and you can use sparse matrix techniques? I have read pruning generally is not used, which to me also suggests sparse array techniques

Thanks
Bill

 

[pbuk]

I've only skimmed it but I think you are missing the 'residual' step between each convolution. So each step is a convolution of 64 filters over the LR image which are then reduced down to 1 residual HR image. The details of this residual step are I think the essence of the novelty of the method described in the paper; see the sub-section titled 'Residual-Learning' in section 3.2.

But this is not my area of expertise (although it looks fascinating and the results appear remarkable) so I may be completely wrong