This design document contains notes that are relevant to the MXNet system design and deep learning libraries in general. We believe that open sourcing this system design note can help general audiences understand the motivations, benefits and drawbacks of our design choices. This will help deep learning practitioners as well as builders of other deep learning systems.

Deep Learning System Design Concepts

In this section, we will describe general design concepts of Deep Learning systems. We will mainly focus on following 3 areas influencing the design and architecture of a Deep Learning System.

  1. Abstraction
  2. Optimization
  3. Trade offs

Refer below sections for more details

MXNet System Design and Architecture

In this section, we will describe MXNet System architecture and various components.