💡 [REQUEST] - Neural Style Transfer: change layers to compute style/content

[tutubalin]

🚀 Describe the improvement or the new tutorial

I tried using code from Neural Style Transfer tutorial to apply a van Gogh style to my photo, and results were way poorer copared to what I've seen on DeepArt.io.

Luckily, I found what's the issue, and how to improve quality.

Original paper states:

Thus, the visually
most appealing images are usually created by matching the
style representation up to high layers in the network, which
is why for all images shown we match the style features
in layers ‘conv1_1’, ‘conv2_1’, ‘conv3_1’, ‘conv4_1’ and
‘conv5_1’ of the network.

and also:

The images shown in Fig 3
were synthesised by matching the content representation
on layer ‘conv4_2’ and the style representation on layers
‘conv1_1’, ‘conv2_1’, ‘conv3_1’, ‘conv4_1’ and ‘conv5_1’

In the tutorial, layer conv_4 is used for content and layers from conv_1 to conv_5 for style.:

# desired depth layers to compute style/content losses :
content_layers_default = ['conv_4']
style_layers_default = ['conv_1', 'conv_2', 'conv_3', 'conv_4', 'conv_5']

At first glance everything looks fine, but indexing patterns are different: in the article (and original VGG model) layers were split into groups between poolings, and in the tutorial a continuous numbering is used.

Article	Tutorial
conv1_1	conv_1
conv1_2	conv_2
---	---
conv2_1	conv_3
conv2_2	conv_4
---	---
conv3_1	conv_5
conv3_2	conv_6
conv3_3	conv_7
conv3_4	conv_8
---	---
conv4_1	conv_9
conv4_2	conv_10
conv4_3	conv_11
conv4_4	conv_12
---	---
conv5_1	conv_13
conv5_2	conv_14
conv5_3	conv_15
conv5_4	conv_16

So layers conv_1, conv_3, conv_5, conv_9 and conv_13 should be used for style and layer conv_10 for content.

That produces way better results.

Suggested change

# desired depth layers to compute style/content losses :
content_layers_default = ['conv_10']
style_layers_default = ['conv_1', 'conv_3', 'conv_5', 'conv_9', 'conv_13']

Existing tutorials on this topic

https://docs.pytorch.org/tutorials/advanced/neural_style_tutorial.html

Additional context

Another suggestion to improve quality: #3731