CuPy
Numerical programming library for the Python programming language
- 5 min read
- 3 chapters
Related on Lably
Through Open-source software, Python (programming language) and NumPy
CuPy has been initially developed as a backend of Chainer deep learning framework, and later established as an independent project in 2017.
CuPy is a part of the NumPy ecosystem array libraries and is widely adopted to utilize GPU with Python, especially in high-performance computing environments such as Summit, Perlmutter, EULER, and ABCI.
CuPy is a NumFOCUS sponsored project.
Features
CuPy implements NumPy/SciPy-compatible APIs, as well as features to write user-defined GPU kernels or access low-level APIs.
NumPy-compatible APIs
The same set of APIs defined in the NumPy package (numpy.*) are available under cupy.* package.
- Multi-dimensional array (
cupy.ndarray) for boolean, integer, float, and complex data types - Module-level functions
- Linear algebra functions
- Fast Fourier transform
- Random number generator
SciPy-compatible APIs
The same set of APIs defined in the SciPy package (scipy.*) are available under cupyx.scipy.* package.
- Sparse matrices (
cupyx.scipy.sparse.*_matrix) of CSR, COO, CSC, and DIA format - Discrete Fourier transform
- Advanced linear algebra
- Multidimensional image processing
- Sparse linear algebra
- Special functions
- Signal processing
- Statistical functions
User-defined GPU kernels
- Kernel templates for element-wise and reduction operations
- Raw kernel (CUDA C/C++)
- Just-in-time transpiler (JIT)
- Kernel fusion
Distributed computing
- Distributed communication package (
cupyx.distributed), providing collective and peer-to-peer primitives
Low-level CUDA features
- Stream and event
- Memory pool
- Profiler
- Host API binding
- CUDA Python support
Interoperability
- DLPack
- CUDA Array Interface
- NEP 13 (
__array_ufunc__) - NEP 18 (
__array_function__) - Array API Standard
Sign in to keep reading
2 more chapters are free with a Lably account.
Examples
Array creation
<span id="mwjA"></span><span class="o" id="mwjQ">>>></span> <span class="kn" id="mwjg">import</span> <span class="nn" id="mwjw">cupy</span> <span class="k" id="mwkA">as</span> <span class="nn" id="mwkQ">cp</span> <span class="o" id="mwkg">>>></span> <span class="n" id="mwkw">x</span> <span class="o" id="mwlA">=</span> <span class="n" id="mwlQ">cp</span><span class="o" id="mwlg">.</span><span class="kp" id="mwlw">array</span><span class="p" id="mwmA">([</span><span class="mi" id="mwmQ">1</span><span class="p" id="mwmg">,</span> <span class="mi" id="mwmw">2</span><span class="p" id="mwnA">,</span> <span class="mi" id="mwnQ">3</span><span class="p" id="mwng">])</span> <span class="o" id="mwnw">>>></span> <span class="n" id="mwoA">x</span> <span class="kp" id="mwoQ">array</span><span class="p" id="mwog">([</span><span class="mi" id="mwow">1</span><span class="p" id="mwpA">,</span> <span class="mi" id="mwpQ">2</span><span class="p" id="mwpg">,</span> <span class="mi" id="mwpw">3</span><span class="p" id="mwqA">])</span> <span class="o" id="mwqQ">>>></span> <span class="n" id="mwqg">y</span> <span class="o" id="mwqw">=</span> <span class="n" id="mwrA">cp</span><span class="o" id="mwrQ">.</span><span class="kp" id="mwrg">arange</span><span class="p" id="mwrw">(</span><span class="mi" id="mwsA">10</span><span class="p" id="mwsQ">)</span> <span class="o" id="mwsg">>>></span> <span class="n" id="mwsw">y</span> <span class="kp" id="mwtA">array</span><span class="p" id="mwtQ">([</span><span class="mi" id="mwtg">0</span><span class="p" id="mwtw">,</span> <span class="mi" id="mwuA">1</span><span class="p" id="mwuQ">,</span> <span class="mi" id="mwug">2</span><span class="p" id="mwuw">,</span> <span class="mi" id="mwvA">3</span><span class="p" id="mwvQ">,</span> <span class="mi" id="mwvg">4</span><span class="p" id="mwvw">,</span> <span class="mi" id="mwwA">5</span><span class="p" id="mwwQ">,</span> <span class="mi" id="mwwg">6</span><span class="p" id="mwww">,</span> <span class="mi" id="mwxA">7</span><span class="p" id="mwxQ">,</span> <span class="mi" id="mwxg">8</span><span class="p" id="mwxw">,</span> <span class="mi" id="mwyA">9</span><span class="p" id="mwyQ">])</span>
Basic operations
<span id="mwzQ"></span><span class="o" id="mwzg">>>></span> <span class="kn" id="mwzw">import</span> <span class="nn" id="mw0A">cupy</span> <span class="k" id="mw0Q">as</span> <span class="nn" id="mw0g">cp</span> <span class="o" id="mw0w">>>></span> <span class="n" id="mw1A">x</span> <span class="o" id="mw1Q">=</span> <span class="n" id="mw1g">cp</span><span class="o" id="mw1w">.</span><span class="kp" id="mw2A">arange</span><span class="p" id="mw2Q">(</span><span class="mi" id="mw2g">12</span><span class="p" id="mw2w">)</span><span class="o" id="mw3A">.</span><span class="kp" id="mw3Q">reshape</span><span class="p" id="mw3g">(</span><span class="mi" id="mw3w">3</span><span class="p" id="mw4A">,</span> <span class="mi" id="mw4Q">4</span><span class="p" id="mw4g">)</span><span class="o" id="mw4w">.</span><span class="kp" id="mw5A">astype</span><span class="p" id="mw5Q">(</span><span class="n" id="mw5g">cp</span><span class="o" id="mw5w">.</span><span class="n" id="mw6A">float32</span><span class="p" id="mw6Q">)</span> <span class="o" id="mw6g">>>></span> <span class="n" id="mw6w">x</span> <span class="kp" id="mw7A">array</span><span class="p" id="mw7Q">([[</span> <span class="mf" id="mw7g">0.</span><span class="p" id="mw7w">,</span> <span class="mf" id="mw8A">1.</span><span class="p" id="mw8Q">,</span> <span class="mf" id="mw8g">2.</span><span class="p" id="mw8w">,</span> <span class="mf" id="mw9A">3.</span><span class="p" id="mw9Q">],</span> <span class="p" id="mw9g">[</span> <span class="mf" id="mw9w">4.</span><span class="p" id="mw-A">,</span> <span class="mf" id="mw-Q">5.</span><span class="p" id="mw-g">,</span> <span class="mf" id="mw-w">6.</span><span class="p" id="mw_A">,</span> <span class="mf" id="mw_Q">7.</span><span class="p" id="mw_g">],</span> <span class="p" id="mw_w">[</span> <span class="mf" id="mwAQA">8.</span><span class="p" id="mwAQE">,</span> <span class="mf" id="mwAQI">9.</span><span class="p" id="mwAQM">,</span> <span class="mf" id="mwAQQ">10.</span><span class="p" id="mwAQU">,</span> <span class="mf" id="mwAQY">11.</span><span class="p" id="mwAQc">]],</span> <span class="kp" id="mwAQg">dtype</span><span class="o" id="mwAQk">=</span><span class="n" id="mwAQo">float32</span><span class="p" id="mwAQs">)</span> <span class="o" id="mwAQw">>>></span> <span class="n" id="mwAQ0">x</span><span class="o" id="mwAQ4">.</span><span class="kp" id="mwAQ8">sum</span><span class="p" id="mwARA">(</span><span class="n" id="mwARE">axis</span><span class="o" id="mwARI">=</span><span class="mi" id="mwARM">1</span><span class="p" id="mwARQ">)</span> <span class="kp" id="mwARU">array</span><span class="p" id="mwARY">([</span> <span class="mf" id="mwARc">6.</span><span class="p" id="mwARg">,</span> <span class="mf" id="mwARk">22.</span><span class="p" id="mwARo">,</span> <span class="mf" id="mwARs">38.</span><span class="p" id="mwARw">],</span> <span class="kp" id="mwAR0">dtype</span><span class="o" id="mwAR4">=</span><span class="n" id="mwAR8">float32</span><span class="p" id="mwASA">)</span>
Raw CUDA C/C++ kernel
<span id="mwASQ"></span><span class="o" id="mwASU">>>></span> <span class="kn" id="mwASY">import</span> <span class="nn" id="mwASc">cupy</span> <span class="k" id="mwASg">as</span> <span class="nn" id="mwASk">cp</span> <span class="o" id="mwASo">>>></span> <span class="n" id="mwASs">kern</span> <span class="o" id="mwASw">=</span> <span class="n" id="mwAS0">cp</span><span class="o" id="mwAS4">.</span><span class="n" id="mwAS8">RawKernel</span><span class="p" id="mwATA">(</span><span class="sa" id="mwATE">r</span><span class="s1" id="mwATI">'''</span> <span class="s1" id="mwATM">... extern "C" __global__</span> <span class="s1" id="mwATQ">... void multiply_elemwise(const float* in1, const float* in2, float* out) {</span> <span class="s1" id="mwATU">... int tid = blockDim.x * blockIdx.x + threadIdx.x;</span> <span class="s1" id="mwATY">... out[tid] = in1[tid] * in2[tid];</span> <span class="s1" id="mwATc">... }</span> <span class="s1" id="mwATg">... '''</span><span class="p" id="mwATk">,</span> <span class="s1" id="mwATo">'multiply_elemwise'</span><span class="p" id="mwATs">)</span> <span class="o" id="mwATw">>>></span> <span class="n" id="mwAT0">in1</span> <span class="o" id="mwAT4">=</span> <span class="n" id="mwAT8">cp</span><span class="o" id="mwAUA">.</span><span class="kp" id="mwAUE">arange</span><span class="p" id="mwAUI">(</span><span class="mi" id="mwAUM">16</span><span class="p" id="mwAUQ">,</span> <span class="kp" id="mwAUU">dtype</span><span class="o" id="mwAUY">=</span><span class="n" id="mwAUc">cp</span><span class="o" id="mwAUg">.</span><span class="n" id="mwAUk">float32</span><span class="p" id="mwAUo">)</span><span class="o" id="mwAUs">.</span><span class="kp" id="mwAUw">reshape</span><span class="p" id="mwAU0">(</span><span class="mi" id="mwAU4">4</span><span class="p" id="mwAU8">,</span> <span class="mi" id="mwAVA">4</span><span class="p" id="mwAVE">)</span> <span class="o" id="mwAVI">>>></span> <span class="n" id="mwAVM">in2</span> <span class="o" id="mwAVQ">=</span> <span class="n" id="mwAVU">cp</span><span class="o" id="mwAVY">.</span><span class="kp" id="mwAVc">arange</span><span class="p" id="mwAVg">(</span><span class="mi" id="mwAVk">16</span><span class="p" id="mwAVo">,</span> <span class="kp" id="mwAVs">dtype</span><span class="o" id="mwAVw">=</span><span class="n" id="mwAV0">cp</span><span class="o" id="mwAV4">.</span><span class="n" id="mwAV8">float32</span><span class="p" id="mwAWA">)</span><span class="o" id="mwAWE">.</span><span class="kp" id="mwAWI">reshape</span><span class="p" id="mwAWM">(</span><span class="mi" id="mwAWQ">4</span><span class="p" id="mwAWU">,</span> <span class="mi" id="mwAWY">4</span><span class="p" id="mwAWc">)</span> <span class="o" id="mwAWg">>>></span> <span class="n" id="mwAWk">out</span> <span class="o" id="mwAWo">=</span> <span class="n" id="mwAWs">cp</span><span class="o" id="mwAWw">.</span><span class="kp" id="mwAW0">zeros</span><span class="p" id="mwAW4">((</span><span class="mi" id="mwAW8">4</span><span class="p" id="mwAXA">,</span> <span class="mi" id="mwAXE">4</span><span class="p" id="mwAXI">),</span> <span class="kp" id="mwAXM">dtype</span><span class="o" id="mwAXQ">=</span><span class="n" id="mwAXU">cp</span><span class="o" id="mwAXY">.</span><span class="n" id="mwAXc">float32</span><span class="p" id="mwAXg">)</span> <span class="o" id="mwAXk">>>></span> <span class="n" id="mwAXo">kern</span><span class="p" id="mwAXs">((</span><span class="mi" id="mwAXw">4</span><span class="p" id="mwAX0">,),</span> <span class="p" id="mwAX4">(</span><span class="mi" id="mwAX8">4</span><span class="p" id="mwAYA">,),</span> <span class="p" id="mwAYE">(</span><span class="n" id="mwAYI">in1</span><span class="p" id="mwAYM">,</span> <span class="n" id="mwAYQ">in2</span><span class="p" id="mwAYU">,</span> <span class="n" id="mwAYY">out</span><span class="p" id="mwAYc">))</span> <span class="c1" id="mwAYg"># grid, block and arguments</span> <span class="o" id="mwAYk">>>></span> <span class="n" id="mwAYo">out</span> <span class="kp" id="mwAYs">array</span><span class="p" id="mwAYw">([[</span> <span class="mf" id="mwAY0">0.</span><span class="p" id="mwAY4">,</span> <span class="mf" id="mwAY8">1.</span><span class="p" id="mwAZA">,</span> <span class="mf" id="mwAZE">4.</span><span class="p" id="mwAZI">,</span> <span class="mf" id="mwAZM">9.</span><span class="p" id="mwAZQ">],</span> <span class="p" id="mwAZU">[</span> <span class="mf" id="mwAZY">16.</span><span class="p" id="mwAZc">,</span> <span class="mf" id="mwAZg">25.</span><span class="p" id="mwAZk">,</span> <span class="mf" id="mwAZo">36.</span><span class="p" id="mwAZs">,</span> <span class="mf" id="mwAZw">49.</span><span class="p" id="mwAZ0">],</span> <span class="p" id="mwAZ4">[</span> <span class="mf" id="mwAZ8">64.</span><span class="p" id="mwAaA">,</span> <span class="mf" id="mwAaE">81.</span><span class="p" id="mwAaI">,</span> <span class="mf" id="mwAaM">100.</span><span class="p" id="mwAaQ">,</span> <span class="mf" id="mwAaU">121.</span><span class="p" id="mwAaY">],</span> <span class="p" id="mwAac">[</span><span class="mf" id="mwAag">144.</span><span class="p" id="mwAak">,</span> <span class="mf" id="mwAao">169.</span><span class="p" id="mwAas">,</span> <span class="mf" id="mwAaw">196.</span><span class="p" id="mwAa0">,</span> <span class="mf" id="mwAa4">225.</span><span class="p" id="mwAa8">]],</span> <span class="kp" id="mwAbA">dtype</span><span class="o" id="mwAbE">=</span><span class="n" id="mwAbI">float32</span><span class="p" id="mwAbM">)</span>Applications
- spaCy
- XGBoost
- turboSETI (Berkeley SETI)
- NVIDIA RAPIDS
- einops
- scikit-learn
- MONAI
- Chainer

