What does 1x1 convolution mean in a neural network? 1x1 conv creates channel-wise dependencies with a negligible cost This is especially exploited in depthwise-separable convolutions Nobody said anything about this but I'm writing this as a comment since I don't have enough reputation here
What is the difference between Conv1D and Conv2D? I was going through the keras convolution docs and I have found two types of convultuion Conv1D and Conv2D I did some web search and this is what I understands about Conv1D and Conv2D; Conv1D is u
How do bottleneck architectures work in neural networks? We define a bottleneck architecture as the type found in the ResNet paper where [two 3x3 conv layers] are replaced by [one 1x1 conv, one 3x3 conv, and another 1x1 conv layer] I understand that t
In CNN, are upsampling and transpose convolution the same? Both the terms "upsampling" and "transpose convolution" are used when you are doing "deconvolution" (<-- not a good term, but let me use it here) Originally, I thought that they mean the same t
Pooling vs. stride for downsampling - Cross Validated Pooling and stride both can be used to downsample the image Let's say we have an image of 4x4, like below and a filter of 2x2 Then how do we decide whether to use (2x2 pooling) vs (stride of 2)?
Batch normalization and the need for bias in neural networks Here's a quote from the original BN paper that should answer your question: i e each activation is shifted by its own shift parameter (beta) So yes, the batch normalization eliminates the need for a bias vector Just a side note: in Pytorch the BN's betas are all initialized to zero by default, whereas the biases in linear and convolutional layers are initialized to random values
Where should I place dropout layers in a neural network? I've updated the answer to clarify that in the work by Park et al , the dropout was applied after the RELU on each CONV layer I do not believe they investigated the effect of adding dropout following max pooling layers
deep learning - What is the definition of a feature map (aka . . . Typical-looking activations on the first CONV layer (left), and the 5th CONV layer (right) of a trained AlexNet looking at a picture of a cat Every box shows an activation map corresponding to some filter Notice that the activations are sparse (most values are zero, in this visualization shown in black) and mostly local