Recommended System Architecture-Caterpie Generated By Blastoise

Shanzai CL

Published on: 2021-08-12

Abstract

The motivation for writing the paper is to verify that each transformer has a different architecture and hyper-parameter settings without considering the constraints of resource consumption. In actual application scenarios, how much performance will be improved. First, we use the components in the evolutionary version of DynaBert to construct the search space, through NAS, we got a new Transformer at the beginning, including 6 new transformers in the encoder, and 6 new transformers in the decoder. We integrate a series of fresh technologies In this new Transformer, and developed it as a new model BLASTOISE, Secondly, in our experiments, BLASTOISE continuously improves Transformer on four data sets: WMT 2014 English-German, WMT 2014 English-French, WMT 2014 English- Czech and LM1B. Our model is always better than the latest model. Finally, we use the encoder in BLASTOISE as bert (CATERPIE). Extensive experiments conducted on four benchmark data sets show that our model is always better than the latest sequential model. Finally, we tested and verified the model on the recommended data set, showing that our model is effective.