Curriculum Learning & Multiple Stage Learning
Curriculum learning is borrowed from the human education system. In our education system, we present students easier concepts first and gradually increase the difficulty. In FlowNet 2.0, simpler training data are fed into the network first and then the more difficult dataset. This method can guide the training process to converge faster and reach better minima.
Multiple stage learning is widely used in object detection, such as MTCNN, Faster-RCNN, R-Mask. In MTCNN, simpler networks (PNet and RNet) are trained first. In Faster-RCNN and its following algorithms, the networks can be trained at the same time. However, the region proposal network tends to learn simpler concepts before the final regression and classification network, which is not quite different from the behaviour in MTCNN.
Evolution reveals the similar concepts. We, humans, also have a humble and simple start. From the evolution tree of life, we can see life never rebuilds its DNA from scratch ever since the start. The evolution of eyes is a typical example. Brains seem to follow a similar route. Can we use the same idea to improve evolution algorithm?
References:
- Curriculum Learning. https://ronan.collobert.com/pub/matos/2009_curriculum_icml.pdf
- Papers of MTCNN, FlowNet 2.0, Faster-RCN, etc.