Neural Network : CNN, RNN & DNN

Neural Network 神經網絡

Some basic knowledge are required to better understand what is Neural Network.

• Linear Regression 
• Sigmoid Squashing Function
• Rectified linear unit
• Derivation
• Error

Two important types of artificial neuron

• perceptron neuron
• sigmoid neuron

Perceptron Neuron

• Detemine output with Weight and Bias
• Each perceptron has only one output
• input x usually zero or one
• output = wx + b
• w = weight
• b = bias

Sigmoid Neuron

• Detemine output with Weight and Bias, plus sigmoid σσ
• Each perceptron has only one output
• input x between zero or one
• output = σ (wx + b)
• w = weight
• b = bias
• σ =sigmoid
• sigmoid function: σz = 1+ (1 / 1 + e^(-z))
• Or ,  output = 1 /  1 +  exp (−∑jwjxj−b)

Reference: https://en.wikipedia.org/wiki/Derivative#Rules_for_basic_functions

Reference:

https://brohrer.github.io/how_neural_networks_work.html

http://www.neuralnetworksanddeeplearning.com

Architecture of Neural Network

Feedforward Neural Network : neural networks where the output from one layer is used as input to the next layer. There are no loops in the network - information is always fed forward, never fed back. 

Recurrent Neural Network: Neural networks where the output from one layer is used as input to the next layer with loops in a limited duration of time in the network.

CNN - Convolutional Neural Network 卷積神經網絡

CNN is about space?

RNN - Recurrent Neural Network 

RNN is about time?

DNN - Deep Neural Network