BASE／BC_H13 v20220210

Model

Encoder

• Type: Transformer encoder layers (the same network structure as the one used for BERT_BASE)
  • Dimension: 768
  • # of heads: 12
  • Dimension of feedforward networks: 3072
  • # of layers: 12
  • Activation function: GELU
  • Dropout rate in training: 0.1
  • Initialization: Random

Decoder

• Type: Single-layer position-wise feedforward network
  • Dimension: 3072
  • Activation function: GELU
  • Dropout rate in training: 0.1
  • Initialization: Random

Objective

• Type: Behavioral cloning (BC)
• Loss function: Cross entropy

Data

Crawled Game Records

Crawled Game Records v202007_202107

Training Examples

43002752 samples consisting of action selections only by Saint 2, Saint 3, and Celestial players, which are randomly sampled from the crawled data and shuffled.

Optimization

• Optimizer: LAMB
• Learning rate: 0.001
• ε: 1.0e-6
• Batch size: 4096
• # of training epochs: 1