Theano (software)
Numerical computation library for Python
- 10 min read
- 3 chapters
Related on Lably
Through Python (programming language), NumPy and Compiler
History
Theano is an open source project primarily developed by the Montreal Institute for Learning Algorithms (MILA) at the Université de Montréal.
The name of the software references the ancient philosopher Theano, long associated with the development of the golden mean.
On 28 September 2017, Pascal Lamblin posted a message from Yoshua Bengio, Head of MILA: major development would cease after the 1.0 release due to competing offerings by strong industrial players. Theano 1.0.0 was then released on 15 November 2017.
On 17 May 2018, Chris Fonnesbeck wrote on behalf of the PyMC development team that the PyMC developers will officially assume control of Theano maintenance once the MILA development team steps down. On 29 January 2021, they started using the name Aesara for their fork of Theano.
On 29 Nov 2022, the PyMC development team announced that the PyMC developers will fork the Aesara project under the name PyTensor.
Sign in to keep reading
2 more chapters are free with a Lably account.
Sample code
The following code is the original Theano's example. It defines a computational graph with 2 scalars a and b of type double and an operation between them (addition) and then creates a Python function f that does the actual computation.
<span id="mwTA"></span><span class="kn" id="mwTQ">import</span><span class="w" id="mwTg"> </span><span class="nn" id="mwTw">theano</span> <span class="kn" id="mwUA">from</span><span class="w" id="mwUQ"> </span><span class="nn" id="mwUg">theano</span><span class="w" id="mwUw"> </span><span class="kn" id="mwVA">import</span> <span class="n" id="mwVQ">tensor</span> <span class="c1" id="mwVg"># Declare two symbolic floating-point scalars</span> <span class="n" id="mwVw">a</span> <span class="o" id="mwWA">=</span> <span class="n" id="mwWQ">tensor</span><span class="o" id="mwWg">.</span><span class="n" id="mwWw">dscalar</span><span class="p" id="mwXA">()</span> <span class="n" id="mwXQ">b</span> <span class="o" id="mwXg">=</span> <span class="n" id="mwXw">tensor</span><span class="o" id="mwYA">.</span><span class="n" id="mwYQ">dscalar</span><span class="p" id="mwYg">()</span> <span class="c1" id="mwYw"># Create a simple expression</span> <span class="n" id="mwZA">c</span> <span class="o" id="mwZQ">=</span> <span class="n" id="mwZg">a</span> <span class="o" id="mwZw">+</span> <span class="n" id="mwaA">b</span> <span class="c1" id="mwaQ"># Convert the expression into a callable object that takes (a, b)</span> <span class="c1" id="mwag"># values as input and computes a value for c</span> <span class="n" id="mwaw">f</span> <span class="o" id="mwbA">=</span> <span class="n" id="mwbQ">theano</span><span class="o" id="mwbg">.</span><span class="n" id="mwbw">function</span><span class="p" id="mwcA">([</span><span class="n" id="mwcQ">a</span><span class="p" id="mwcg">,</span> <span class="n" id="mwcw">b</span><span class="p" id="mwdA">],</span> <span class="n" id="mwdQ">c</span><span class="p" id="mwdg">)</span> <span class="c1" id="mwdw"># Bind 1.5 to 'a', 2.5 to 'b', and evaluate 'c'</span> <span class="k" id="mweA">assert</span> <span class="mf" id="mweQ">4.0</span> <span class="o" id="mweg">==</span> <span class="n" id="mwew">f</span><span class="p" id="mwfA">(</span><span class="mf" id="mwfQ">1.5</span><span class="p" id="mwfg">,</span> <span class="mf" id="mwfw">2.5</span><span class="p" id="mwgA">)</span>
Examples
Matrix Multiplication (Dot Product)
The following code demonstrates how to perform matrix multiplication using Theano, which is essential for linear algebra operations in many machine learning tasks.
<span id="mwhg"></span><span class="kn" id="mwhw">import</span><span class="w" id="mwiA"> </span><span class="nn" id="mwiQ">theano</span> <span class="kn" id="mwig">from</span><span class="w" id="mwiw"> </span><span class="nn" id="mwjA">theano</span><span class="w" id="mwjQ"> </span><span class="kn" id="mwjg">import</span> <span class="n" id="mwjw">tensor</span> <span class="c1" id="mwkA"># Declare two symbolic 2D arrays (matrices)</span> <span class="n" id="mwkQ">A</span> <span class="o" id="mwkg">=</span> <span class="n" id="mwkw">tensor</span><span class="o" id="mwlA">.</span><span class="n" id="mwlQ">dmatrix</span><span class="p" id="mwlg">(</span><span class="s2" id="mwlw">"A"</span><span class="p" id="mwmA">)</span> <span class="n" id="mwmQ">B</span> <span class="o" id="mwmg">=</span> <span class="n" id="mwmw">tensor</span><span class="o" id="mwnA">.</span><span class="n" id="mwnQ">dmatrix</span><span class="p" id="mwng">(</span><span class="s2" id="mwnw">"B"</span><span class="p" id="mwoA">)</span> <span class="c1" id="mwoQ"># Define a matrix multiplication (dot product) operation</span> <span class="n" id="mwog">C</span> <span class="o" id="mwow">=</span> <span class="n" id="mwpA">tensor</span><span class="o" id="mwpQ">.</span><span class="n" id="mwpg">dot</span><span class="p" id="mwpw">(</span><span class="n" id="mwqA">A</span><span class="p" id="mwqQ">,</span> <span class="n" id="mwqg">B</span><span class="p" id="mwqw">)</span> <span class="c1" id="mwrA"># Create a function that computes the result of the matrix multiplication</span> <span class="n" id="mwrQ">f</span> <span class="o" id="mwrg">=</span> <span class="n" id="mwrw">theano</span><span class="o" id="mwsA">.</span><span class="n" id="mwsQ">function</span><span class="p" id="mwsg">([</span><span class="n" id="mwsw">A</span><span class="p" id="mwtA">,</span> <span class="n" id="mwtQ">B</span><span class="p" id="mwtg">],</span> <span class="n" id="mwtw">C</span><span class="p" id="mwuA">)</span> <span class="c1" id="mwuQ"># Sample matrices</span> <span class="n" id="mwug">A_val</span> <span class="o" id="mwuw">=</span> <span class="p" id="mwvA">[[</span><span class="mi" id="mwvQ">1</span><span class="p" id="mwvg">,</span> <span class="mi" id="mwvw">2</span><span class="p" id="mwwA">],</span> <span class="p" id="mwwQ">[</span><span class="mi" id="mwwg">3</span><span class="p" id="mwww">,</span> <span class="mi" id="mwxA">4</span><span class="p" id="mwxQ">]]</span> <span class="n" id="mwxg">B_val</span> <span class="o" id="mwxw">=</span> <span class="p" id="mwyA">[[</span><span class="mi" id="mwyQ">5</span><span class="p" id="mwyg">,</span> <span class="mi" id="mwyw">6</span><span class="p" id="mwzA">],</span> <span class="p" id="mwzQ">[</span><span class="mi" id="mwzg">7</span><span class="p" id="mwzw">,</span> <span class="mi" id="mw0A">8</span><span class="p" id="mw0Q">]]</span> <span class="c1" id="mw0g"># Evaluate the matrix multiplication</span> <span class="n" id="mw0w">result</span> <span class="o" id="mw1A">=</span> <span class="n" id="mw1Q">f</span><span class="p" id="mw1g">(</span><span class="n" id="mw1w">A_val</span><span class="p" id="mw2A">,</span> <span class="n" id="mw2Q">B_val</span><span class="p" id="mw2g">)</span> <span class="nb" id="mw2w">print</span><span class="p" id="mw3A">(</span><span class="n" id="mw3Q">result</span><span class="p" id="mw3g">)</span>
Gradient Calculation
The following code uses Theano to compute the gradient of a simple operation (like a neuron) with respect to its input. This is useful in training machine learning models (backpropagation).
<span id="mw4w"></span><span class="kn" id="mw5A">import</span><span class="w" id="mw5Q"> </span><span class="nn" id="mw5g">theano</span> <span class="kn" id="mw5w">from</span><span class="w" id="mw6A"> </span><span class="nn" id="mw6Q">theano</span><span class="w" id="mw6g"> </span><span class="kn" id="mw6w">import</span> <span class="n" id="mw7A">tensor</span> <span class="c1" id="mw7Q"># Define symbolic variables</span> <span class="n" id="mw7g">x</span> <span class="o" id="mw7w">=</span> <span class="n" id="mw8A">tensor</span><span class="o" id="mw8Q">.</span><span class="n" id="mw8g">dscalar</span><span class="p" id="mw8w">(</span><span class="s2" id="mw9A">"x"</span><span class="p" id="mw9Q">)</span> <span class="c1" id="mw9g"># Input scalar</span> <span class="n" id="mw9w">y</span> <span class="o" id="mw-A">=</span> <span class="n" id="mw-Q">tensor</span><span class="o" id="mw-g">.</span><span class="n" id="mw-w">dscalar</span><span class="p" id="mw_A">(</span><span class="s2" id="mw_Q">"y"</span><span class="p" id="mw_g">)</span> <span class="c1" id="mw_w"># Weight scalar</span> <span class="c1" id="mwAQA"># Define a simple function (y * x, a simple linear function)</span> <span class="n" id="mwAQE">z</span> <span class="o" id="mwAQI">=</span> <span class="n" id="mwAQM">y</span> <span class="o" id="mwAQQ">*</span> <span class="n" id="mwAQU">x</span> <span class="c1" id="mwAQY"># Compute the gradient of z with respect to x (partial derivative of z with respect to x)</span> <span class="n" id="mwAQc">dz_dx</span> <span class="o" id="mwAQg">=</span> <span class="n" id="mwAQk">tensor</span><span class="o" id="mwAQo">.</span><span class="n" id="mwAQs">grad</span><span class="p" id="mwAQw">(</span><span class="n" id="mwAQ0">z</span><span class="p" id="mwAQ4">,</span> <span class="n" id="mwAQ8">x</span><span class="p" id="mwARA">)</span> <span class="c1" id="mwARE"># Create a function to compute the value of z and dz/dx</span> <span class="n" id="mwARI">f</span> <span class="o" id="mwARM">=</span> <span class="n" id="mwARQ">theano</span><span class="o" id="mwARU">.</span><span class="n" id="mwARY">function</span><span class="p" id="mwARc">([</span><span class="n" id="mwARg">x</span><span class="p" id="mwARk">,</span> <span class="n" id="mwARo">y</span><span class="p" id="mwARs">],</span> <span class="p" id="mwARw">[</span><span class="n" id="mwAR0">z</span><span class="p" id="mwAR4">,</span> <span class="n" id="mwAR8">dz_dx</span><span class="p" id="mwASA">])</span> <span class="c1" id="mwASE"># Sample values</span> <span class="n" id="mwASI">x_val</span> <span class="o" id="mwASM">=</span> <span class="mf" id="mwASQ">2.0</span> <span class="n" id="mwASU">y_val</span> <span class="o" id="mwASY">=</span> <span class="mf" id="mwASc">3.0</span> <span class="c1" id="mwASg"># Compute z and its gradient</span> <span class="n" id="mwASk">result</span> <span class="o" id="mwASo">=</span> <span class="n" id="mwASs">f</span><span class="p" id="mwASw">(</span><span class="n" id="mwAS0">x_val</span><span class="p" id="mwAS4">,</span> <span class="n" id="mwAS8">y_val</span><span class="p" id="mwATA">)</span> <span class="nb" id="mwATE">print</span><span class="p" id="mwATI">(</span><span class="s2" id="mwATM">"z:"</span><span class="p" id="mwATQ">,</span> <span class="n" id="mwATU">result</span><span class="p" id="mwATY">[</span><span class="mi" id="mwATc">0</span><span class="p" id="mwATg">])</span> <span class="c1" id="mwATk"># z = y * x = 3 * 2 = 6</span> <span class="nb" id="mwATo">print</span><span class="p" id="mwATs">(</span><span class="s2" id="mwATw">"dz/dx:"</span><span class="p" id="mwAT0">,</span> <span class="n" id="mwAT4">result</span><span class="p" id="mwAT8">[</span><span class="mi" id="mwAUA">1</span><span class="p" id="mwAUE">])</span> <span class="c1" id="mwAUI"># dz/dx = y = 3</span>
Building a Simple Neural Network
The following code shows how to start building a simple neural network. This is a very basic neural network with one hidden layer.
<span id="mwAUc"></span><span class="kn" id="mwAUg">import</span><span class="w" id="mwAUk"> </span><span class="nn" id="mwAUo">theano</span> <span class="kn" id="mwAUs">from</span><span class="w" id="mwAUw"> </span><span class="nn" id="mwAU0">theano</span><span class="w" id="mwAU4"> </span><span class="kn" id="mwAU8">import</span> <span class="n" id="mwAVA">tensor</span> <span class="k" id="mwAVE">as</span> <span class="n" id="mwAVI">T</span> <span class="kn" id="mwAVM">import</span><span class="w" id="mwAVQ"> </span><span class="nn" id="mwAVU">numpy</span><span class="w" id="mwAVY"> </span><span class="k" id="mwAVc">as</span><span class="w" id="mwAVg"> </span><span class="nn" id="mwAVk">np</span> <span class="c1" id="mwAVo"># Define symbolic variables for input and output</span> <span class="n" id="mwAVs">X</span> <span class="o" id="mwAVw">=</span> <span class="n" id="mwAV0">T</span><span class="o" id="mwAV4">.</span><span class="n" id="mwAV8">matrix</span><span class="p" id="mwAWA">(</span><span class="s2" id="mwAWE">"X"</span><span class="p" id="mwAWI">)</span> <span class="c1" id="mwAWM"># Input features</span> <span class="n" id="mwAWQ">y</span> <span class="o" id="mwAWU">=</span> <span class="n" id="mwAWY">T</span><span class="o" id="mwAWc">.</span><span class="n" id="mwAWg">ivector</span><span class="p" id="mwAWk">(</span><span class="s2" id="mwAWo">"y"</span><span class="p" id="mwAWs">)</span> <span class="c1" id="mwAWw"># Target labels (integer vector)</span> <span class="c1" id="mwAW0"># Define the size of the layers</span> <span class="n" id="mwAW4">input_size</span> <span class="o" id="mwAW8">=</span> <span class="mi" id="mwAXA">2</span> <span class="c1" id="mwAXE"># Number of input features</span> <span class="n" id="mwAXI">hidden_size</span> <span class="o" id="mwAXM">=</span> <span class="mi" id="mwAXQ">3</span> <span class="c1" id="mwAXU"># Number of neurons in the hidden layer</span> <span class="n" id="mwAXY">output_size</span> <span class="o" id="mwAXc">=</span> <span class="mi" id="mwAXg">2</span> <span class="c1" id="mwAXk"># Number of output classes</span> <span class="c1" id="mwAXo"># Initialize weights for input to hidden layer (2x3 matrix) and hidden to output (3x2 matrix)</span> <span class="n" id="mwAXs">W1</span> <span class="o" id="mwAXw">=</span> <span class="n" id="mwAX0">theano</span><span class="o" id="mwAX4">.</span><span class="n" id="mwAX8">shared</span><span class="p" id="mwAYA">(</span><span class="n" id="mwAYE">np</span><span class="o" id="mwAYI">.</span><span class="n" id="mwAYM">random</span><span class="o" id="mwAYQ">.</span><span class="n" id="mwAYU">randn</span><span class="p" id="mwAYY">(</span><span class="n" id="mwAYc">input_size</span><span class="p" id="mwAYg">,</span> <span class="n" id="mwAYk">hidden_size</span><span class="p" id="mwAYo">),</span> <span class="n" id="mwAYs">name</span><span class="o" id="mwAYw">=</span><span class="s2" id="mwAY0">"W1"</span><span class="p" id="mwAY4">)</span> <span class="n" id="mwAY8">b1</span> <span class="o" id="mwAZA">=</span> <span class="n" id="mwAZE">theano</span><span class="o" id="mwAZI">.</span><span class="n" id="mwAZM">shared</span><span class="p" id="mwAZQ">(</span><span class="n" id="mwAZU">np</span><span class="o" id="mwAZY">.</span><span class="n" id="mwAZc">zeros</span><span class="p" id="mwAZg">(</span><span class="n" id="mwAZk">hidden_size</span><span class="p" id="mwAZo">),</span> <span class="n" id="mwAZs">name</span><span class="o" id="mwAZw">=</span><span class="s2" id="mwAZ0">"b1"</span><span class="p" id="mwAZ4">)</span> <span class="n" id="mwAZ8">W2</span> <span class="o" id="mwAaA">=</span> <span class="n" id="mwAaE">theano</span><span class="o" id="mwAaI">.</span><span class="n" id="mwAaM">shared</span><span class="p" id="mwAaQ">(</span><span class="n" id="mwAaU">np</span><span class="o" id="mwAaY">.</span><span class="n" id="mwAac">random</span><span class="o" id="mwAag">.</span><span class="n" id="mwAak">randn</span><span class="p" id="mwAao">(</span><span class="n" id="mwAas">hidden_size</span><span class="p" id="mwAaw">,</span> <span class="n" id="mwAa0">output_size</span><span class="p" id="mwAa4">),</span> <span class="n" id="mwAa8">name</span><span class="o" id="mwAbA">=</span><span class="s2" id="mwAbE">"W2"</span><span class="p" id="mwAbI">)</span> <span class="n" id="mwAbM">b2</span> <span class="o" id="mwAbQ">=</span> <span class="n" id="mwAbU">theano</span><span class="o" id="mwAbY">.</span><span class="n" id="mwAbc">shared</span><span class="p" id="mwAbg">(</span><span class="n" id="mwAbk">np</span><span class="o" id="mwAbo">.</span><span class="n" id="mwAbs">zeros</span><span class="p" id="mwAbw">(</span><span class="n" id="mwAb0">output_size</span><span class="p" id="mwAb4">),</span> <span class="n" id="mwAb8">name</span><span class="o" id="mwAcA">=</span><span class="s2" id="mwAcE">"b2"</span><span class="p" id="mwAcI">)</span> <span class="c1" id="mwAcM"># Define the forward pass (hidden layer and output layer)</span> <span class="n" id="mwAcQ">hidden_output</span> <span class="o" id="mwAcU">=</span> <span class="n" id="mwAcY">T</span><span class="o" id="mwAcc">.</span><span class="n" id="mwAcg">nnet</span><span class="o" id="mwAck">.</span><span class="n" id="mwAco">sigmoid</span><span class="p" id="mwAcs">(</span><span class="n" id="mwAcw">T</span><span class="o" id="mwAc0">.</span><span class="n" id="mwAc4">dot</span><span class="p" id="mwAc8">(</span><span class="n" id="mwAdA">X</span><span class="p" id="mwAdE">,</span> <span class="n" id="mwAdI">W1</span><span class="p" id="mwAdM">)</span> <span class="o" id="mwAdQ">+</span> <span class="n" id="mwAdU">b1</span><span class="p" id="mwAdY">)</span> <span class="c1" id="mwAdc"># Sigmoid activation</span> <span class="n" id="mwAdg">output</span> <span class="o" id="mwAdk">=</span> <span class="n" id="mwAdo">T</span><span class="o" id="mwAds">.</span><span class="n" id="mwAdw">nnet</span><span class="o" id="mwAd0">.</span><span class="n" id="mwAd4">softmax</span><span class="p" id="mwAd8">(</span><span class="n" id="mwAeA">T</span><span class="o" id="mwAeE">.</span><span class="n" id="mwAeI">dot</span><span class="p" id="mwAeM">(</span><span class="n" id="mwAeQ">hidden_output</span><span class="p" id="mwAeU">,</span> <span class="n" id="mwAeY">W2</span><span class="p" id="mwAec">)</span> <span class="o" id="mwAeg">+</span> <span class="n" id="mwAek">b2</span><span class="p" id="mwAeo">)</span> <span class="c1" id="mwAes"># Softmax output</span> <span class="c1" id="mwAew"># Define the cost function (cross-entropy)</span> <span class="n" id="mwAe0">cost</span> <span class="o" id="mwAe4">=</span> <span class="n" id="mwAe8">T</span><span class="o" id="mwAfA">.</span><span class="n" id="mwAfE">nnet</span><span class="o" id="mwAfI">.</span><span class="n" id="mwAfM">categorical_crossentropy</span><span class="p" id="mwAfQ">(</span><span class="n" id="mwAfU">output</span><span class="p" id="mwAfY">,</span> <span class="n" id="mwAfc">y</span><span class="p" id="mwAfg">)</span><span class="o" id="mwAfk">.</span><span class="n" id="mwAfo">mean</span><span class="p" id="mwAfs">()</span> <span class="c1" id="mwAfw"># Compute gradients</span> <span class="n" id="mwAf0">grad_W1</span><span class="p" id="mwAf4">,</span> <span class="n" id="mwAf8">grad_b1</span><span class="p" id="mwAgA">,</span> <span class="n" id="mwAgE">grad_W2</span><span class="p" id="mwAgI">,</span> <span class="n" id="mwAgM">grad_b2</span> <span class="o" id="mwAgQ">=</span> <span class="n" id="mwAgU">T</span><span class="o" id="mwAgY">.</span><span class="n" id="mwAgc">grad</span><span class="p" id="mwAgg">(</span><span class="n" id="mwAgk">cost</span><span class="p" id="mwAgo">,</span> <span class="p" id="mwAgs">[</span><span class="n" id="mwAgw">W1</span><span class="p" id="mwAg0">,</span> <span class="n" id="mwAg4">b1</span><span class="p" id="mwAg8">,</span> <span class="n" id="mwAhA">W2</span><span class="p" id="mwAhE">,</span> <span class="n" id="mwAhI">b2</span><span class="p" id="mwAhM">])</span> <span class="c1" id="mwAhQ"># Create a function to compute the cost and gradients</span> <span class="n" id="mwAhU">train</span> <span class="o" id="mwAhY">=</span> <span class="n" id="mwAhc">theano</span><span class="o" id="mwAhg">.</span><span class="n" id="mwAhk">function</span><span class="p" id="mwAho">(</span><span class="n" id="mwAhs">inputs</span><span class="o" id="mwAhw">=</span><span class="p" id="mwAh0">[</span><span class="n" id="mwAh4">X</span><span class="p" id="mwAh8">,</span> <span class="n" id="mwAiA">y</span><span class="p" id="mwAiE">],</span> <span class="n" id="mwAiI">outputs</span><span class="o" id="mwAiM">=</span><span class="p" id="mwAiQ">[</span><span class="n" id="mwAiU">cost</span><span class="p" id="mwAiY">,</span> <span class="n" id="mwAic">grad_W1</span><span class="p" id="mwAig">,</span> <span class="n" id="mwAik">grad_b1</span><span class="p" id="mwAio">,</span> <span class="n" id="mwAis">grad_W2</span><span class="p" id="mwAiw">,</span> <span class="n" id="mwAi0">grad_b2</span><span class="p" id="mwAi4">])</span> <span class="c1" id="mwAi8"># Sample input data and labels (2 features, 2 samples)</span> <span class="n" id="mwAjA">X_val</span> <span class="o" id="mwAjE">=</span> <span class="n" id="mwAjI">np</span><span class="o" id="mwAjM">.</span><span class="n" id="mwAjQ">array</span><span class="p" id="mwAjU">([[</span><span class="mf" id="mwAjY">0.1</span><span class="p" id="mwAjc">,</span> <span class="mf" id="mwAjg">0.2</span><span class="p" id="mwAjk">],</span> <span class="p" id="mwAjo">[</span><span class="mf" id="mwAjs">0.3</span><span class="p" id="mwAjw">,</span> <span class="mf" id="mwAj0">0.4</span><span class="p" id="mwAj4">]])</span> <span class="n" id="mwAj8">y_val</span> <span class="o" id="mwAkA">=</span> <span class="n" id="mwAkE">np</span><span class="o" id="mwAkI">.</span><span class="n" id="mwAkM">array</span><span class="p" id="mwAkQ">([</span><span class="mi" id="mwAkU">0</span><span class="p" id="mwAkY">,</span> <span class="mi" id="mwAkc">1</span><span class="p" id="mwAkg">])</span> <span class="c1" id="mwAkk"># Train the network for a single step (you would iterate in practice)</span> <span class="n" id="mwAko">cost_val</span><span class="p" id="mwAks">,</span> <span class="n" id="mwAkw">grad_W1_val</span><span class="p" id="mwAk0">,</span> <span class="n" id="mwAk4">grad_b1_val</span><span class="p" id="mwAk8">,</span> <span class="n" id="mwAlA">grad_W2_val</span><span class="p" id="mwAlE">,</span> <span class="n" id="mwAlI">grad_b2_val</span> <span class="o" id="mwAlM">=</span> <span class="n" id="mwAlQ">train</span><span class="p" id="mwAlU">(</span><span class="n" id="mwAlY">X_val</span><span class="p" id="mwAlc">,</span> <span class="n" id="mwAlg">y_val</span><span class="p" id="mwAlk">)</span> <span class="nb" id="mwAlo">print</span><span class="p" id="mwAls">(</span><span class="s2" id="mwAlw">"Cost:"</span><span class="p" id="mwAl0">,</span> <span class="n" id="mwAl4">cost_val</span><span class="p" id="mwAl8">)</span> <span class="nb" id="mwAmA">print</span><span class="p" id="mwAmE">(</span><span class="s2" id="mwAmI">"Gradients for W1:"</span><span class="p" id="mwAmM">,</span> <span class="n" id="mwAmQ">grad_W1_val</span><span class="p" id="mwAmU">)</span>
Broadcasting in Theano
The following code demonstrates how broadcasting works in Theano. Broadcasting allows operations between arrays of different shapes without needing to explicitly reshape them.
<span id="mwAmo"></span><span class="kn" id="mwAms">import</span><span class="w" id="mwAmw"> </span><span class="nn" id="mwAm0">theano</span> <span class="kn" id="mwAm4">from</span><span class="w" id="mwAm8"> </span><span class="nn" id="mwAnA">theano</span><span class="w" id="mwAnE"> </span><span class="kn" id="mwAnI">import</span> <span class="n" id="mwAnM">tensor</span> <span class="k" id="mwAnQ">as</span> <span class="n" id="mwAnU">T</span> <span class="kn" id="mwAnY">import</span><span class="w" id="mwAnc"> </span><span class="nn" id="mwAng">numpy</span><span class="w" id="mwAnk"> </span><span class="k" id="mwAno">as</span><span class="w" id="mwAns"> </span><span class="nn" id="mwAnw">np</span> <span class="c1" id="mwAn0"># Declare symbolic arrays</span> <span class="n" id="mwAn4">A</span> <span class="o" id="mwAn8">=</span> <span class="n" id="mwAoA">T</span><span class="o" id="mwAoE">.</span><span class="n" id="mwAoI">dmatrix</span><span class="p" id="mwAoM">(</span><span class="s2" id="mwAoQ">"A"</span><span class="p" id="mwAoU">)</span> <span class="n" id="mwAoY">B</span> <span class="o" id="mwAoc">=</span> <span class="n" id="mwAog">T</span><span class="o" id="mwAok">.</span><span class="n" id="mwAoo">dvector</span><span class="p" id="mwAos">(</span><span class="s2" id="mwAow">"B"</span><span class="p" id="mwAo0">)</span> <span class="c1" id="mwAo4"># Broadcast B to the shape of A, then add them</span> <span class="n" id="mwAo8">C</span> <span class="o" id="mwApA">=</span> <span class="n" id="mwApE">A</span> <span class="o" id="mwApI">+</span> <span class="n" id="mwApM">B</span> <span class="c1" id="mwApQ"># Broadcasting B to match the shape of A</span> <span class="c1" id="mwApU"># Create a function to evaluate the operation</span> <span class="n" id="mwApY">f</span> <span class="o" id="mwApc">=</span> <span class="n" id="mwApg">theano</span><span class="o" id="mwApk">.</span><span class="n" id="mwApo">function</span><span class="p" id="mwAps">([</span><span class="n" id="mwApw">A</span><span class="p" id="mwAp0">,</span> <span class="n" id="mwAp4">B</span><span class="p" id="mwAp8">],</span> <span class="n" id="mwAqA">C</span><span class="p" id="mwAqE">)</span> <span class="c1" id="mwAqI"># Sample data (A is a 3x2 matrix, B is a 2-element vector)</span> <span class="n" id="mwAqM">A_val</span> <span class="o" id="mwAqQ">=</span> <span class="n" id="mwAqU">np</span><span class="o" id="mwAqY">.</span><span class="n" id="mwAqc">array</span><span class="p" id="mwAqg">([[</span><span class="mi" id="mwAqk">1</span><span class="p" id="mwAqo">,</span> <span class="mi" id="mwAqs">2</span><span class="p" id="mwAqw">],</span> <span class="p" id="mwAq0">[</span><span class="mi" id="mwAq4">3</span><span class="p" id="mwAq8">,</span> <span class="mi" id="mwArA">4</span><span class="p" id="mwArE">],</span> <span class="p" id="mwArI">[</span><span class="mi" id="mwArM">5</span><span class="p" id="mwArQ">,</span> <span class="mi" id="mwArU">6</span><span class="p" id="mwArY">]])</span> <span class="n" id="mwArc">B_val</span> <span class="o" id="mwArg">=</span> <span class="n" id="mwArk">np</span><span class="o" id="mwAro">.</span><span class="n" id="mwArs">array</span><span class="p" id="mwArw">([</span><span class="mi" id="mwAr0">10</span><span class="p" id="mwAr4">,</span> <span class="mi" id="mwAr8">20</span><span class="p" id="mwAsA">])</span> <span class="c1" id="mwAsE"># Evaluate the addition with broadcasting</span> <span class="n" id="mwAsI">result</span> <span class="o" id="mwAsM">=</span> <span class="n" id="mwAsQ">f</span><span class="p" id="mwAsU">(</span><span class="n" id="mwAsY">A_val</span><span class="p" id="mwAsc">,</span> <span class="n" id="mwAsg">B_val</span><span class="p" id="mwAsk">)</span> <span class="nb" id="mwAso">print</span><span class="p" id="mwAss">(</span><span class="n" id="mwAsw">result</span><span class="p" id="mwAs0">)</span>