Abstract: Batch normalization (BN) enhances the training of deep ReLU neural network with a composition of mean centering (centralization) and variance scaling (unitization). Despite the success of BN ...