Generic formulas

The two previous sections have highlighted the need for efficient Map-Reduce GPU routines in data sciences. To complete our guided tour of the inner workings of the KeOps library, we now explain how generic reductions and formulas are encoded within our codebase.

Note that this section is extracted from the Chapter 2.2 of Jean Feydy’s PhD thesis.

Generic formulas

• Design choices
  • The KeOps symbolic engine
  • Working with variadic templates
• Math operations
  • In practice
  • Contributing with a new operation
• Reductions
  • Summation
  • The online Log-Sum-Exp trick
• Backpropagation
  • Backprop through a Sum reduction
  • Backprop through a Log-Sum-Exp reduction
  • Bootstrapping derivatives of arbitrary order