Conv2d Intro
Conv2d Intro
Conv2d Intro
Cross-correlation
Dr. Thanh-Sach LE
LTSACH@hcmut.edu.vn
FC
FC
FC
POOL
CONV
CONV
CONV
POOL
NORM
CONV
POOL
NORM
CONV
FC
FC
FC
POOL
CONV
They used convolution to extract features by
CONV transforming from raw pixels to meaningful
representation
CONV
POOL
NORM
CONV
POOL
NORM
CONV
}
POOL
CONV
CONV
CONV
They used convolution to extract features by
}
POOL
transforming from raw pixels to meaningful
CONV
representation
CONV
CONV
}
POOL
CONV
CONV
CONV
POOL
CONV
CONV }
POOL
CONV
CONV } VGG-16 (2014)
AKaren Simonyan, Andrew Zisserman, “Very Deep Convolutional Networks for Large-Scale
Image Recognition,” arXiv:1409.1556v6
UNet (2015)
Olaf Ronneberger, Philipp Fischer, Thomas Brox, “U-Net: Convolutional Networks for Biomedical Image
Segmentation,” arXiv:1505.04597v1 [cs.CV]
They used convolution to extract features by transforming from raw pixels to meaningful
representation
UNet (2015)
Olaf Ronneberger, Philipp Fischer, Thomas Brox, “U-Net: Convolutional Networks for Biomedical Image
Segmentation,” arXiv:1505.04597v1 [cs.CV]
Convolution and Cross-correlation
Mathematical Definition
Dr. Thanh-Sach LE
LTSACH@hcmut.edu.vn
❖ Computation node:
W
Convolution’s parameters
(Filter’s kernel)
❖ Notation:
Y = X*W
12 Mathematical Definition
❖ Computation node:
W
Convolution’s parameters
(Filter’s kernel)
❖ Notation:
Y = X*W
* : NOT a matrix multiplication
13 Mathematical Definition
❖ Definition:
u j
0 1 2 3 4 5 6 3
1 2
2 1
3 -3 -2 -1 0 1 2 3
i
∂b
4 -1
5 -2
6 -3
v W
X
Radius of kernel:
7
W : 7x7 ⇒ r = ⌊ ⌋ = 3
2
14 Mathematical Definition
❖ Definition:
{
Y(u, v) = X * W
r r
∑∑
∂b
Convolution = X(u − i, v − j)W(i, j)
i=−r j=−r
{
Y(u, v) = X ⋆ W
r r
∑∑
Cross-Correlation = X(u + i, v + j)W(i, j)
i=−r j=−r
15 Mathematical Definition
❖ Convolution vs cross-correlation:
W W
w11 w12 w11 w12
∂b
Rot1800
w22 w21
w12 w11
x11 x12 x13
=
x31 x32 x33
Y Y
16 Mathematical Definition
❖ Convolution vs cross-correlation:
W W
w11 w12 w11 w12
∂b
Rot1800
w22 w21
w12 w11
x11 x12 x13
=
x31 x32 x33
Y Y
Convolution and Cross-correlation
Computation of convolution
Dr. Thanh-Sach LE
LTSACH@hcmut.edu.vn
∂b
(b) Place the kernel aligned with the left-top of the
image
(c) Take the dot product between the kernel and
the sub-image occupied by the kernel and
assign the result to the output at the
corresponding location
(d) Slide the kernel to left and down; do task (b)
after each sliding
Yes 5
End
21 Computation of convolution
3 1 0 1
1
1
2
2
2
0
1
CONV Output ?
0 1 0 2
1 0 2
Input image
1 2 0
0 1 1
Filter’s kernel
22 Computation of convolution
1 Rotate the kernel
1 0 2 1 1 0
Rotation 180o
1 2 0 0 2 1
0 1 1 2 0 1
0
W Rot180 (W)
Flattening
1 1 0 0 2 1 2 0 1
24 Computation of convolution
2 Padding the input
3 1 0 1
1 1 2 0
1 2 2 1
0 1 0 2
X 2
0 1 0 2 0 1 0 2
Input image
1 1 0
0 2 1
2 0 1
Rot1800(W)
26 Computation of convolution
4 Compute dot-product 3 1 0
X and W: aligned at left-top
1 1 2
·
Get sub-image
1x3 1x1 0x0 1 1 2 2
1 1 0 0 2 1 2 0 1
1 1 0
0 2 1
3x1 + 1x1 + 0x0 +
2 0 1 1x0 + 1x2 + 2x1 + 12
Rot1800(W) 1x2 + 2x0 + 2x1
= 12
27 Computation of convolution
4 Compute dot-product 1 0 1
X and W: aligned at left-top
1 2 0
·
Get sub-image
3 1 0 1 2 2 1
1 1 2 0 flattening
1 2 2 1
1 0 1 1 2 0 2 2 1
0 1 0 2
dot-product
Input image
1 1 0 0 2 1 2 0 1
1 1 0
0 2 1
1x1 + 0x1 + 1x0 +
2 0 1 1x0 + 2x2 + 0x1 + 12 10
Rot1800(W) 2x2 + 2x0 + 1x1
= 10
28 Computation of convolution
4 Compute dot-product 1 1 2
X and W: aligned at left-top
1 2 2
·
Get sub-image
3 1 0 1 0 1 0
1 1 2 0 flattening
1 2 2 1
1 1 2 1 2 2 0 1 0
0 1 0 2
dot-product
Input image
1 1 0 0 2 1 2 0 1
1 1 0
0 2 1
1x1 + 1x1 + 2x0 +
2 0 1 1x0 + 2x2 + 2x1 + 12 10
Rot1800(W) 0x2 + 1x0 + 0x1
8
=8
29 Computation of convolution
4 Compute dot-product 1 2 0
X and W: aligned at left-top
2 2 1
·
Get sub-image
3 1 0 1 1 0 2
1 1 2 0 flattening
1 2 2 1
1 2 0 2 2 1 1 0 2
0 1 0 2
dot-product
Input image
1 1 0 0 2 1 2 0 1
1 1 0
0 2 1
1x1 + 2x1 + 0x0 +
2 0 1 12 10
2x0 + 2x2 + 1x1 +
Rot1800(W) 1x2 + 0x0 + 2x1
8 12
= 12
30 Computation of convolution
Final result
3 1 0 1
1 1 2 0 12 10
CONV
1 2 2 1 8 12
0 1 0 2 Output
1 0 2
Input image
1 2 0
0 1 1
Filter’s kernel
31 Computation of convolution
Final result
3 1 0 1
1 1 2 0 12 10
CONV
1 2 2 1 8 12
0 1 0 2 Output
1 0 2 (4 − 3 + 1) × (4 − 3 + 1)
Input image
1 2 0 2×2
4×4
0 1 1
Filter’s kernel
3×3
32 Computation of convolution
Final result
3 1 0 1
1 1 2 0 12 10
CONV
1 2 2 1 8 12
0 1 0 2 Output
1 0 2 (4 − 3 + 1) × (4 − 3 + 1)
Input image
1 2 0 2×2
4×4
0 1 1
Filter’s kernel
3×3
i1 × i2
* (i1 − k1 + 1) × (i2 − k2 + 1)
k1 × k2