Python install

PyKeOps is a Python 3 wrapper around the low-level KeOps library, which is written in C++/CUDA. It provides functions that can be used in any NumPy or PyTorch script.

Requirements

  • Python 3 with packages numpy and GPUtil.

  • A C++ compiler compatible with std=c++14: g++ version >=7 or clang++ version >=8.

  • The Cmake build system, version >= 3.10.

  • The Cuda toolkit, including the nvcc compiler (optional): version >=10.0 is recommended. Make sure that your C++ compiler is compatible with the installed nvcc.

  • PyTorch (optional): version >= 1.1.0.

On Google Colab

Google provides free virtual machines which are able to run KeOps out-of-the-box. In a new Colab notebook, typing:

!pip install pykeops[full] > install.log

should allow you to get a working version of KeOps in less than twenty seconds.

From source using git

  1. Clone the KeOps repo at a location of your choice (denoted here as /path/to)

git clone --recursive https://github.com/getkeops/keops.git /path/to/libkeops

Note that your compiled .so routines will be stored in the folder /path/to/libkeops/pykeops/build: this directory must have write permission.

  1. Manually add the directory /path/to/libkeops (and not /path/to/libkeops/pykeops/) to your python path.

  • This can be done once and for all, by adding the path to to your ~/.bashrc. In a terminal,

    echo "export PYTHONPATH=$PYTHONPATH:/path/to/libkeops/" >> ~/.bashrc
    
  • Otherwise, you may add the following line to the beginning of your python scripts:

    import os.path
    import sys
    sys.path.append('/path/to/libkeops')
    
  1. Test your installation, as described in the next section.

Testing your installation

  1. In a python terminal,

import numpy as np
import pykeops.numpy as pknp

x = np.arange(1, 10).reshape(-1, 3).astype('float32')
y = np.arange(3, 9).reshape(-1, 3).astype('float32')

my_conv = pknp.Genred('SqNorm2(x - y)', ['x = Vi(3)', 'y = Vj(3)'])
print(my_conv(x, y))

should return:

Compiling libKeOpsnumpy5ac3d464a2 in /path/to/build_dir/:
   formula: Sum_Reduction(SqNorm2(x - y),1)
   aliases: x = Vi(0,3); y = Vj(1,3);
   dtype  : float64
[[63.]
 [90.]]
  1. If you use PyTorch, the following code:

import torch
import pykeops.torch as pktorch

x = torch.arange(1, 10, dtype=torch.float32).view(-1, 3)
y = torch.arange(3, 9, dtype=torch.float32).view(-1, 3)

my_conv = pktorch.Genred('SqNorm2(x-y)', ['x = Vi(3)', 'y = Vj(3)'])
print(my_conv(x, y))

should return:

Compiling libKeOpstorch91c92bd508 in /path/to/build_dir/:
   formula: Sum_Reduction(SqNorm2(x-y),1)
   aliases: x = Vi(0,3); y = Vj(1,3);
   dtype  : float32
... Done.
tensor([[63.],
        [90.]])

Troubleshooting

Compilation issues

First of all, make sure that you are using a C++ compiler which is compatible with the C++11 revision and/or your nvcc (CUDA) compiler. Otherwise, compilation of formulas may fail in unexpected ways. Depending on your system, you can:

  1. Install a compiler system-wide: for instance, on Debian based Linux distros, this can be done by installing g++ with apt and then using update-alternatives to choose the right compiler.

  2. Install a compiler locally: if you are using a conda environment, you can install a new instance of gcc and g++ by following the documentation of conda.

Cache directory

If you experience problems with compilation (or numerical inaccuracies after a KeOps update), it may be a good idea to flush the build folder (i.e. the cache of already-compiled formulas). To get the directory name, just type:

import pykeops
print(pykeops.bin_folder)

Verbosity level

To help debugging, you can activate a verbose compilation mode. It may be done by defining the environment variable PYKEOPS_VERBOSE to 1. In a terminal

export PYKEOPS_VERBOSE=1
python my_script_calling_pykeops.py

Or directly in your python script by setting after your KeOps imports the flag pykeops.verbose to true. It gives in a python shell something like:

import pykeops
pykeops.verbose = True

Build type

You can force the (re)compilation of the KeOps shared objects by changing the build type from Release (default) to Debug. This may be done by defining the environment variable PYKEOPS_BUILD_TYPE , either in a terminal:

export PYKEOPS_BUILD_TYPE="Debug"
python my_script_calling_pykeops.py

Or directly in your python script, altering the value of the (string) variable pykeops.build_type right after your KeOps imports. In a python shell, simply type:

import pykeops
pykeops.build_type = 'Debug'

Warning

Beware! The shared objects generated in debug mode are not optimized, and should thus be deleted at the end of your debugging session. In order to do so, please flush your cache directory as described in the previous section.