Tutorials

Tutorials

• Installation
  • Build LiBai from Source
• Getting Started
  • Train Bert-large Model Parallelly
  • Train VisionTransformer on ImageNet Dataset
• Training & Evaluation in Command Line
  • Training
  • Evaluation
  • Quickly check in the respective loop
• Features in LiBai
  • Automatic Mixed Precision Training
  • Gradient Clipping
  • Gradient Accumulation
  • Activation Checkpointing
  • ZeRO
• Config System
  • Configs in LiBai
  • Get the Default Config
  • Best Practice with LazyConfig
• Training
  • Trainer Abstraction
  • Customize a DefaultTrainer
  • Logging of Metrics
• Build New Project on LiBai
  • Introduce
  • Main Function Entry
  • Build Config
  • Start Training
• Distributed Configuration
  • Distributed Setting Example
  • Update Distributed Config with Command Line
• How to customize your own parallel
  • Define your own parallel model with LiBai.layers
  • Write your own pipeline parallel model
• LiBai Model Zoo
  • Parallelism Mode in LiBai
  • Supported Model in LiBai
  • Benchmarks (coming soon)