$\newcommand{\B}[1]{{\bf #1}}$$\newcommand{\R}[1]{{\rm #1}}$
pycppad-20140710: A Python Algorithm Derivative Package

Syntax
Purpose
Simple Example
Features
Contents
 Manual split into sections    Manual as one web page Math displayed using Latex    pycppad.htm    _printable.htm Math displayed using MathML    pycppad.xml    _printable.xml

10: BSD

b: Syntax
from pycppad import *

c: Purpose
The command above imports a boost::python (http://www.boost.org) interface to the C++ Algorithmic Differentiation (AD) (http://en.wikipedia.org/wiki/Automatic_differentiation) package CppAD (http://www.coin-or.org/CppAD/CppAD/) .

d: Simple Example  from pycppad import * def pycppad_test_get_started() : def F(x) : # function to be differentiated return exp(-(x[0]**2. + x[1]**2.) / 2.) # is Gaussian density x = numpy.array( [ 1., 2.] ) a_x = independent(x) a_y = numpy.array( [ F(a_x) ] ) f = adfun(a_x, a_y) J = f.jacobian(x) # J = F'(x) assert abs( J[0, 0] + F(x) * x[0] ) < 1e-10 # J[0,0] ~= - F(x) * x[0] assert abs( J[0, 1] + F(x) * x[1] ) < 1e-10 # J[0,1] ~= - F(x) * x[1] 

e: Features
Operator overloading is used to store the operation sequence corresponding to a python algorithm. The operation sequence can be evaluated to obtain new function values or derivatives of arbitrary order. In addition, multiple levels of AD are supported. This means that AD derivatives can be used in the definition of a function which in turn can be differentiated using AD. See 9: whats_new for a list of recent extensions and bug fixes.

f: Contents
 _contents: 1 Table of Contents install: 2 Installing pycppad get_started.py: 3 get_started: Example and Test example: 4 List of All the pycppad Examples ad_variable: 5 AD Variable Methods ad_function: 6 AD Function Methods two_levels.py: 7 Using Two Levels of AD: Example and Test runge_kutta_4: 8 Fourth Order Runge Kutta whats_new: 9 Extensions, Bug Fixes, and Changes license: 10 License _reference: 11 Alphabetic Listing of Cross Reference Tags _index: 12 Keyword Index _external: 13 External Internet References

Input File: doc.omh

pycppad-20140710: A Python Algorithm Derivative Package: : pycppad
Installing pycppad: 2: install
get_started: Example and Test: 3: get_started.py
List of All the pycppad Examples: 4: example
Create an Object With One Higher Level of AD: 5.1: ad
Create an Object With One Lower Level of AD: 5.2: value
value: Example and Test: 5.2.1: value.py
Unary Plus and Minus Operators: 5.3: ad_unary
Unary Plus and Minus Operators: Example and Test: 5.3.1: ad_unary.py
Binary Numeric Operators With an AD Result: 5.4: ad_numeric
Binary Numeric Operators With an AD Result: Example and Test: 5.4.1: ad_numeric.py
Future Division Operator: Example and Test: 5.4.2: future_div_op.py
Computed Assignment Operators: 5.5: assign_op
Computed Assignment Operators: Example and Test: 5.5.1: assign_op.py
Binary Comparison Operators: 5.6: compare_op
a_float Comparison Operators: Example and Test: 5.6.1: compare_op.py
Standard Math Unary Functions: 5.7: std_math
Standard Math Unary Functions: Example and Test: 5.7.1: std_math.py
Absolute Value Functions: 5.8: abs
abs: Example and Test: 5.8.1: abs.py
Conditional Expressions: 5.9: condexp
condexp: Example and Test: 5.9.1: condexp.py
Create an Independent Variable Vector: 6.1: independent
independent: Example and Test: 6.1.1: independent.py
Create an AD Function Object: 6.2: adfun
Abort a Recording of AD Operations: 6.3: abort_recording
abort_recording: Example and Test: 6.3.1: abort_recording.py
Forward Mode: Derivative in One Domain Direction: 6.4: forward
Forward Order Zero: Example and Test: 6.4.1: forward_0.py
Forward Order One: Example and Test: 6.4.2: forward_1.py
Reverse Mode: Derivative in One Range Direction: 6.5: reverse
Reverse Order One: Example and Test: 6.5.1: reverse_1.py
Reverse Order Two: Example and Test: 6.5.2: reverse_2.py
Driver for Computing Entire Derivative: 6.6: jacobian
Entire Derivative: Example and Test: 6.6.1: jacobian.py
Driver for Computing Hessian in a Range Direction: 6.7: hessian
Hessian Driver: Example and Test: 6.7.1: hessian.py
Optimize an AD Function Object Tape: 6.8: optimize
Optimize Function Object: Example and Test: 6.8.1: optimize.py
Using Two Levels of AD: Example and Test: 7: two_levels.py
Fourth Order Runge Kutta: 8: runge_kutta_4
runge_kutta_4 A Correctness Example and Test: 8.1: runge_kutta_4_correct.py
runge_kutta_4 An AD Example and Test: 8.2: runge_kutta_4_ad.py
runge_kutta_4 With C++ Speed: Example and Test: 8.3: runge_kutta_4_cpp.py
Extensions, Bug Fixes, and Changes: 9: whats_new
Extensions, Bug Fixes, and Changes During 2012: 9.1: whats_new_12
Extensions, Bug Fixes, and Changes During 2011: 9.2: whats_new_11
Extensions, Bug Fixes, and Changes During 2010: 9.3: whats_new_10
Extensions, Bug Fixes, and Changes During 2009: 9.4: whats_new_09
Alphabetic Listing of Cross Reference Tags: 11: _reference
Keyword Index: 12: _index
External Internet References: 13: _external


$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$

Other Instructions
Requires
Unpacking
Required Setup Information
Building
With Debugging
Optimized
Bug in Python Distutils
Cygwin
Testing
Error
Installing
Uninstalling
Python Path

2.a: Other Instructions
If you are using WinPython (http://code.google.com/p/winpython/) , see building pycppad in Windows (http://list.coin-or.org/pipermail/cppad/2013q2/000309.html)

2.b: Requires
This package requires the following other packages:
1. The Boost Python (development version) must be installed.
2. The python numpy library must be installed.
3. The CppAD package.

2.d: Unpacking
On unix, you could use the command  tar -xvzf pycppad-20140710.tar.gz  which would create the directory pycppad-20140710.

2.e: Required Setup Information
The value of the following setup variables, in the file  pycppad-20140710/setup.py  must be set to agree with your system:  # Directory where CppAD include files are located # cppad_include_dir = [ '/usr/include' ] cppad_include_dir = [ os.environ['HOME'] + '/prefix/cppad/include' ] # # Directory where Boost Python include files are located boost_python_include_dir = [ '/usr/include' ] # # Director whre Boost Python library is located # boost_python_lib_dir = [ '/usr/lib' ] boost_python_lib_dir = [ '/usr/lib64' ] # # Name of the Boost Python library in boost_python_lib_dir. # boost_python_lib = [ 'boost_python-mt' ] boost_python_lib = [ 'boost_python' ]  Note that Boost Python and CppAD must be installed before you can properly set this information.

2.f: Building

2.f.a: With Debugging
Change into the directory pycppad-20140710 and execute the command  ./setup.py build_ext --inplace --debug --undef NDEBUG  to compile and link a version of the CppAD extension module with debugging (improved error messaging).

2.f.b: Optimized
You can compile and link an optimized version of the CppAD extension module with the command  ./setup.py build_ext --inplace  Note that in the optimized version, certain error checking is not done and improper use of pycppad.cpp may lead to a segmentation fault.

2.f.c: Bug in Python Distutils
When using the GNU C++ compiler gcc, the warning  cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++  will be printed. This is not requested by setup.py but is rather a bug in the Python distutils (http://docs.python.org/distutils/) package.

2.f.d: Cygwin
If python crashes during the build procedure under cygwin, you may need to run the rebaseall program. For example, execute the following steps:
1. Check which cygwin processes are running using the ps -e command, and shut them all down.
2. Open an DOS command window, for example, select Start | Run and then enter cmd in as the program to run.
3. Change into the cygwin bin directory. If you installed cygwin in the default location, the following command will do this:  cd c:\cygwin\bin 
4. Start the ash shell by executing the command  ash.exe 
5. The execute the command  /usr/bin/rebaseall  This will take a few minutes to execute. When it is done, you can close the command window by executing the command exit twice.

2.g: Testing
You can test of all the 4: examples in the pycppad documentation. Change into the directory pycppad-20140710 and execute the command  python test_example.py  You can run some more tests with the command       python test_more.py with_debugging  where with_debugging is True or False depending on if you built 2.f.a: with debugging .

2.g.a: Error
If you get a message about cppad_ being missing, it is probably because you did not use the --inplace flag when you 2.f: built pycppad.

2.h: Installing
Installing pycppad copies it to the standard location for you system. You may or may not preform this step:  Change into the directory pycppad-20140710 and execute the command       ./setup.py build_ext --debug --undef NDEBUG install --prefix=prefix  or       ./setup.py build_ext install --prefix=prefix  where prefix is the prefix for the location where you wish to install pycppad. Note that some common choices for prefix are
 $HOME if you are not the system administrator for this machine /usr/local you're administrator and the system has a package manager (like yum, apt-get). /usr you're the administrator but the system does not have a package manager. 2.h.a: Uninstalling Unfortunately, python's distutils package does not provide an uninstall command. You can uninstall the pycppad package by removing the entries  prefix/lib/pythonmajor.minor/site-packages/pycppad prefix/lib/pythonmajor.minor/site-packages/pycppad-20140710.egg-info prefix/share/doc/pycppad  where major and minor are the major and minor version numbers printed by the command  python --version  2.i: Python Path If you have not installed pycppad, you will not be able to use the python command  import pycppad  unless the distribution directory  pycppad-20140710  is in your python path. If you have installed pycppad, the installation directory  prefix/lib/pythonmajor.minor/site-packages  must be in you python path (to use the import pycppad command). You can check your python path with the following commands  python import sys sys.path  If the required directory is not yet there, you could add the directory above to your python path using the command  sys.path.append('prefix/lib/pythonmajor.minor/site-packages')  You can avoid having to do this every time you load python by adding the path to your environment variable PYTHONPATH. For example, if you are using the bash shell, you could add the command  export PYTHONPATH="prefix/lib/pythonmajor.minor/site-packages"  to your $HOME/.bashrc file.

The documentation for pycppad starts out in the directory  pycppad-20140710/doc  During the installation process, it is copied to the directory       prefix/share/doc/pycppad 
Input File: omh/install.omh
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
3: get_started: Example and Test

For our getting started example, we consider the Gaussian density for two independent random variables $F : \B{R}^2 \rightarrow \B{R}$ and its partial derivatives : $$\begin{array}{rcl} F(x) & = & \exp \left[ - ( x_0^2 + x_1^2 ) / 2. \right] \\ \partial_{x(0)} F(x) & = & - F(x) * x_0 \\ \partial_{x(1)} F(x) & = & - F(x) * x_1 \end{array}$$ The following Python code computes these derivatives using pycppad and then checks the results for correctness:  from pycppad import * def pycppad_test_get_started() : def F(x) : # function to be differentiated return exp(-(x[0]**2. + x[1]**2.) / 2.) # is Gaussian density x = numpy.array( [ 1., 2.] ) a_x = independent(x) a_y = numpy.array( [ F(a_x) ] ) f = adfun(a_x, a_y) J = f.jacobian(x) # J = F'(x) assert abs( J[0, 0] + F(x) * x[0] ) < 1e-10 # J[0,0] ~= - F(x) * x[0] assert abs( J[0, 1] + F(x) * x[1] ) < 1e-10 # J[0,1] ~= - F(x) * x[1] 
Input File: example/get_started.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
4: List of All the pycppad Examples

 6.3.1: abort_recording.py abort_recording: Example and Test 5.8.1: abs.py abs: Example and Test 5.1.1: ad.py ad: Example and Test 6.2.1: adfun.py adfun: Example and Test 5.4.1: ad_numeric.py Binary Numeric Operators With an AD Result: Example and Test 5.3.1: ad_unary.py Unary Plus and Minus Operators: Example and Test 5.5.1: assign_op.py Computed Assignment Operators: Example and Test 5.9.1: condexp.py condexp: Example and Test 5.6.1: compare_op.py a_float Comparison Operators: Example and Test 5.4.2: future_div_op.py Future Division Operator: Example and Test 6.4.1: forward_0.py Forward Order Zero: Example and Test 6.4.2: forward_1.py Forward Order One: Example and Test 3: get_started.py get_started: Example and Test 6.7.1: hessian.py Hessian Driver: Example and Test 6.1.1: independent.py independent: Example and Test 6.6.1: jacobian.py Entire Derivative: Example and Test 6.8.1: optimize.py Optimize Function Object: Example and Test 6.5.1: reverse_1.py Reverse Order One: Example and Test 6.5.2: reverse_2.py Reverse Order Two: Example and Test 8.2: runge_kutta_4_ad.py runge_kutta_4 An AD Example and Test 8.3: runge_kutta_4_cpp.py runge_kutta_4 With C++ Speed: Example and Test 8.1: runge_kutta_4_correct.py runge_kutta_4 A Correctness Example and Test 5.7.1: std_math.py Standard Math Unary Functions: Example and Test 7: two_levels.py Using Two Levels of AD: Example and Test 5.2.1: value.py value: Example and Test

Input File: omh/example.omh
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5: AD Variable Methods

Contents

5.a: Contents
 ad: 5.1 Create an Object With One Higher Level of AD value: 5.2 Create an Object With One Lower Level of AD ad_unary: 5.3 Unary Plus and Minus Operators ad_numeric: 5.4 Binary Numeric Operators With an AD Result assign_op: 5.5 Computed Assignment Operators compare_op: 5.6 Binary Comparison Operators std_math: 5.7 Standard Math Unary Functions abs: 5.8 Absolute Value Functions condexp: 5.9 Conditional Expressions

$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.1: Create an Object With One Higher Level of AD

Syntax
Purpose
x
a_x
Example

5.1.a: Syntax
a_x = ad(x)

5.1.b: Purpose
Creates an AD object a_x that records floating point operations. An 6.2: adfun object can later use this recording to evaluate function values and derivatives. These later evaluations are done using the same type as x (except when x is an instance of int, the later evaluations are done using float operations).

5.1.c: x
The argument x can be an instance of an int (AD level 0), or an instance of float (AD level 0), or an a_float (AD level 1). The argument x may also be a numpy.array with one of the element types listed in the previous sentence.

5.1.d: a_x
If x is an instance of int or float, a_x is an a_float (AD level 1). If x is an a_float, a_x is an a2float (AD level 2). If x is an numpy.array, a_x is also an numpy.array with the same shape as x .

5.1.e: Example
The file 5.1.1: ad.py contains an example and test of this function.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.1.1: ad: Example and Test

 from pycppad import * import numpy def pycppad_test_ad() : x = 1 a_x = ad(x) a2x = ad(a_x) # assert type(a_x) == a_float and a_x == x assert type(a2x) == a2float and a2x == x # x = numpy.array( [ 1 , 2 , 3 ] ) a_x = ad(x) a2x = ad(a_x) # for i in range( len(a_x) ) : assert type(a_x[i]) == a_float and a_x[i] == x[i] for i in range( len(a2x) ) : assert type(a2x[i]) == a2float and a2x[i] == x[i] 
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.2: Create an Object With One Lower Level of AD

Syntax
Purpose
a_x
x
Example

5.2.a: Syntax
x = value(a_x)

5.2.b: Purpose
Returns an object with one lower level of AD recording.

5.2.c: a_x
The argument a_x must be an a_float (AD level 1), or an a2float (AD level 2). The argument a_x may also be a numpy.array with one of the element types listed in the previous sentence.

5.2.d: x
If a_x is an a_float, x is a float (AD level 0). If a_x is an a2float, x is an a_float (AD level 1). If a_x is an numpy.array, x is also an numpy.array with the same shape as a_x .

5.2.e: Example
The file 5.2.1: value.py contains an example and test of this function.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.2.1: value: Example and Test

 from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_value() : x = 2 a_x = ad(x) # assert type(value(a_x)) == float and value(a_x) == x # x = numpy.array( [ 1 , 2 , 3 ] ) a_x = ad(x) # for i in range( len(a_x) ) : xi = value(a_x[i]) assert type(xi) == float and xi == x[i] # Example using a2float ------------------------------------------------------ def pycppad_test_value_a2() : x = 2 a2x = ad(ad(x)) # assert type(value(a2x)) == a_float and value(a2x) == x # x = numpy.array( [ 1 , 2 , 3 ] ) a2x = ad(ad(x)) # for i in range( len(a2x) ) : a_xi = value(a2x[i]) assert type(a_xi) == a_float and a_xi == x[i] 
Input File: example/value.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.3: Unary Plus and Minus Operators

Syntax
Purpose
Type
Arrays
Example

5.3.a: Syntax
y = + x  y = - x 
5.3.b: Purpose
The operator + ( - ) above results in z equal to x (minus x ).

5.3.c: Type
The argument x can be a_float or a2float and the result z will have the same type as x .

5.3.d: Arrays
The argument x may be a numpy.array with elements of type a_float or a2float. In this case, the result z is an array with the same shape and element type as x .

5.3.e: Example
The file 5.3.1: ad_unary.py contains an example and test of these functions.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.3.1: Unary Plus and Minus Operators: Example and Test

 from pycppad import * import numpy # Example using a_float ------------------------------------------------------ def pycppad_test_ad_unary() : x = ad(2.) plus_x = + x minus_x = - x # test using corresponding unary float operators assert value(plus_x) == + value(x) assert value(minus_x) == - value(x) # x = ad( numpy.array( [ 1. , 2. ] ) ) plus_x = + x minus_x = - x # test using corresponding unary float operators assert numpy.all( value(plus_x) == + value(x) ) assert numpy.all( value(minus_x) == - value(x) ) # Example using a2float ------------------------------------------------------ def pycppad_test_ad_unary_a2() : x = ad( ad(2.) ) plus_x = + x minus_x = - x # test using corresponding unary a_float operators assert value(plus_x) == + value(x) assert value(minus_x) == - value(x) # x = ad( ad( numpy.array( [ 1. , 2. ] ) ) ) plus_x = + x minus_x = - x # test using corresponding unary float operators assert numpy.all( value(plus_x) == + value(x) ) assert numpy.all( value(minus_x) == - value(x) ) 
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.4: Binary Numeric Operators With an AD Result

Syntax
Purpose
op
Types
Arrays
Example

5.4.a: Syntax
z = x op y

5.4.b: Purpose
Sets z to the result of the binary operation defined by op and with x as the left operand and y as the right operand.

5.4.c: op
The possible values for op are
 op    Meaning + addition - subtraction * multiplication / division ** exponentiation

5.4.d: Types
The following table lists the possible types for x and y and the corresponding result type for z .                        y  x           float    a_float   a2float          -------------------------------  float   -   float    a_float   a2float a_float  -  a_float   a_float a2float  -  a2float             a2float  The type float does not need to be matched exactly but rather as an instance of float.

5.4.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.4.f: Example
The file 5.4.1: ad_numeric.py contains an example and test of these operations. The file 5.4.2: future_div_op.py contains an example and test of the future division operator (http://legacy.python.org/dev/peps/pep-0238/)
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.4.1: Binary Numeric Operators With an AD Result: Example and Test

 from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_ad_numeric() : x = 2. y = 3. a_x = ad(x) a_y = ad(y) # assert a_x + a_y == x + y assert a_x + y == x + y assert x + a_y == x + y # assert a_x - a_y == x - y assert a_x - y == x - y assert x - a_y == x - y # assert a_x * a_y == x * y assert a_x * y == x * y assert x * a_y == x * y # assert a_x / a_y == x / y assert a_x / y == x / y assert x / a_y == x / y # assert a_x ** a_y == x ** y assert a_x ** y == x ** y assert x ** a_y == x ** y # # Example using a2float ----------------------------------------------------- def pycppad_test_ad_numeric_a2() : x = 2. y = 3. a2x = ad(ad(x)) a2y = ad(ad(y)) # assert a2x + a2y == x + y assert a2x + y == x + y assert x + a2y == x + y # assert a2x - a2y == x - y assert a2x - y == x - y assert x - a2y == x - y # assert a2x * a2y == x * y assert a2x * y == x * y assert x * a2y == x * y # assert a2x / a2y == x / y assert a2x / y == x / y assert x / a2y == x / y # assert a2x ** a2y == x ** y assert a2x ** y == x ** y assert x ** a2y == x ** y # 
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.4.2: Future Division Operator: Example and Test

 # Example using a_float ------------------------------------------------------ from __future__ import division from pycppad import * def pycppad_test_future_div_op() : x = 2. y = 3. # tmp1 = ad(x) tmp2 = ad(y) tmp3 = tmp1/tmp2 tmp4 = x/tmp2 tmp5 = tmp1/y assert tmp3 == x / y assert tmp4 == x / y assert tmp5 == x / y 
Input File: example/future_div_op.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.5: Computed Assignment Operators

Syntax
Purpose
op
Types
Arrays
Example

5.5.a: Syntax
u op= x

5.5.b: Purpose
We use y ( z ) to refer to the value of u before (after) the operation. This operation sets z equal to       y op x  .

5.5.c: op
The possible values for op are
 op    Meaning + addition - subtraction * multiplication / division

5.5.d: Types
The following table lists the possible types for x and y (the value of u before the operation) and the corresponding z (the value of u after the operation).                        y  x           float    a_float   a2float          -------------------------------  float   -   float    a_float   a2float a_float  -  a_float   a_float a2float  -  a2float             a2float  The type float does not need to be matched exactly but rather as an instance of float.

5.5.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.5.f: Example
The file 5.5.1: assign_op.py contains an example and test of these operations.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.5.1: Computed Assignment Operators: Example and Test

 # Example using a_float ------------------------------------------------------ from pycppad import * def pycppad_test_assign_op() : x = 2. y = 3. # tmp = ad(x) tmp += ad(y) assert tmp == x + y tmp = ad(x) tmp += y assert tmp == x + y # tmp = ad(x) tmp -= ad(y) assert tmp == x - y tmp = ad(x) tmp -= y assert tmp == x - y # tmp = ad(x) tmp *= ad(y) assert tmp == x * y tmp = ad(x) tmp *= y assert tmp == x * y # tmp = ad(x) tmp /= ad(y) assert tmp == x / y tmp = ad(x) tmp /= y assert tmp == x / y # Example using a2float ------------------------------------------------------ from pycppad import * def pycppad_test_assign_op_a2() : x = 2. y = 3. # tmp = ad(ad(x)) tmp += ad(ad(y)) assert tmp == x + y tmp = ad(ad(x)) tmp += y assert tmp == x + y # tmp = ad(ad(x)) tmp -= ad(ad(y)) assert tmp == x - y tmp = ad(ad(x)) tmp -= y assert tmp == x - y # tmp = ad(ad(x)) tmp *= ad(ad(y)) assert tmp == x * y tmp = ad(ad(x)) tmp *= y assert tmp == x * y # tmp = ad(ad(x)) tmp /= ad(ad(y)) assert tmp == x / y tmp = ad(ad(x)) tmp /= y assert tmp == x / y 
Input File: example/assign_op.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.6: Binary Comparison Operators

Syntax
Purpose
op
Types
Arrays
Example

5.6.a: Syntax
z = x op y

5.6.b: Purpose
Sets z to the result of the binary operation defined by op and with x as the left operand and y as the right operand.

5.6.c: op
The possible values for op are
 op    Meaning > greater than > less than >= greater than or equal <= less than or equal == equal != not equal

5.6.d: Types
The following table lists the possible (yes) and impossible (no) types for x and y . The corresponding result type for z is always bool.                        y  x           float    a_float   a2float          -------------------------------  float   -    yes      yes       yes    a_float  -    yes      yes       no a2float  -    yes      no        yes   The type float does not need to be matched exactly but rather as an instance of float.

5.6.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.6.f: Example
The file 5.6.1: compare_op.py contains an example and test of these operations.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.6.1: a_float Comparison Operators: Example and Test

 from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_compare_op(): x = ad(2.) y = ad(3.) z = ad(2.) # assert comparisons that should be true assert x == x assert x == z assert x != y assert x <= x assert x <= z assert x <= y assert x < y # assert comparisons that should be false assert not x == y assert not x != z assert not x != x assert not x >= y assert not x > y # Example using a2float ------------------------------------------------------ def pycppad_test_compare_op_a2(): x = ad(ad(2.)) y = ad(ad(3.)) z = ad(ad(2.)) # assert comparisons that should be true assert x == x assert x == z assert x != y assert x <= x assert x <= z assert x <= y assert x < y # assert comparisons that should be false assert not x == y assert not x != z assert not x != x assert not x >= y assert not x > y 
Input File: example/compare_op.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.7: Standard Math Unary Functions

Syntax
Purpose
x
y
fun
Example

5.7.a: Syntax
y = fun(x)

5.7.b: Purpose
Evaluate the standard math function fun where fun has one argument.

5.7.c: x
The argument x can be an instance of float, an a_float, an a2float, or a numpy.array of such objects.

5.7.d: y
If x is an instance of float, y will also be an instance of float. Otherwise y will have the same type as x .  In the case where x is an array, y will the same shape as x and the elements of y will have the same type as the elements of x .

5.7.e: fun
The function fun can be any of the following: arccos, arcsin, arctan, cos, cosh, exp, log, log10, sin, sinh, sqrt, tan, or tanh.

5.7.f: Example
The file 5.7.1: std_math.py contains an example and test of these functions.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.7.1: Standard Math Unary Functions: Example and Test

 from pycppad import * import numpy import math # Example using a_float ---------------------------------------------------- def pycppad_test_std_math(): delta = 10. * numpy.finfo(float).eps pi = numpy.pi x = pi / 6 a_x = ad(x) # all the a_float unary standard math functions assert abs( arccos(a_x) - math.acos(x) ) < delta assert abs( arcsin(a_x) - math.asin(x) ) < delta assert abs( arctan(a_x) - math.atan(x) ) < delta assert abs( cos(a_x) - math.cos(x) ) < delta assert abs( cosh(a_x) - math.cosh(x) ) < delta assert abs( exp(a_x) - math.exp(x) ) < delta assert abs( log(a_x) - math.log(x) ) < delta assert abs( log10(a_x) - math.log10(x) ) < delta assert abs( sin(a_x) - math.sin(x) ) < delta assert abs( sinh(a_x) - math.sinh(x) ) < delta assert abs( sqrt(a_x) - math.sqrt(x) ) < delta assert abs( tan(a_x) - math.tan(x) ) < delta assert abs( tanh(a_x) - math.tanh(x) ) < delta # example array and derivative calculation n = 5 x = numpy.array( [2 * pi * j / n for j in range(n) ] ) a_x = independent(x) a_y = sin(a_x) f = adfun(a_x, a_y) J = f.jacobian(x) for j in range(n) : for k in range(n) : if j == k : assert abs( J[j][k] - cos( x[j] ) ) < delta else : assert J[j][k] == 0. # Example using a2float ---------------------------------------------------- def pycppad_test_std_math_a2(): n = 10 delta = 10. * numpy.finfo(float).eps pi = numpy.pi x = pi / 6 a2x = ad(ad(x)) # all the a2float unary standard math functions assert abs( arccos(a2x) - math.acos(x) ) < delta assert abs( arcsin(a2x) - math.asin(x) ) < delta assert abs( arctan(a2x) - math.atan(x) ) < delta assert abs( cos(a2x) - math.cos(x) ) < delta assert abs( cosh(a2x) - math.cosh(x) ) < delta assert abs( exp(a2x) - math.exp(x) ) < delta assert abs( log(a2x) - math.log(x) ) < delta assert abs( log10(a2x) - math.log10(x) ) < delta assert abs( sin(a2x) - math.sin(x) ) < delta assert abs( sinh(a2x) - math.sinh(x) ) < delta assert abs( sqrt(a2x) - math.sqrt(x) ) < delta assert abs( tan(a2x) - math.tan(x) ) < delta assert abs( tanh(a2x) - math.tanh(x) ) < delta # example array and derivative calculation n = 5 x = numpy.array( [2 * pi * j / n for j in range(n) ] ) a_x = ad(x) a2x = independent(a_x) a2y = sin(a2x) a_f = adfun(a2x, a2y) a_J = a_f.jacobian(a_x) for j in range(n) : for k in range(n) : if j == k : assert abs( a_J[j][k] - cos( x[j] ) ) < delta else : assert a_J[j][k] == 0. 
Input File: example/std_math.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.8: Absolute Value Functions

Syntax
Purpose
x
y
Derivative
Directional Derivative
Example

5.8.a: Syntax
y = abs(x)

5.8.b: Purpose
Sets y equal to the absolute value of $x$.

5.8.c: x
The argument x can be an instance of float, an a_float, an a2float, or an numpy.array of such objects.

5.8.d: y
If x is an instance of float, y will also be an instance of float . Otherwise y will have the same type as x .  In the case where x is an array, y will the same shape as x and the elements of y will have the same type as the elements of x .

5.8.e: Derivative
pycppad defines the derivative of the absolute value function by $$\R{abs}^{(1)} (x) = \R{sign} (x) = \left\{ \begin{array}{ll} 1 & \R{if} \; x > 0 \\ 0 & \R{if} \; x = 0 \\ -1 & \R{if} \; x < 0 \end{array} \right.$$

5.8.f: Directional Derivative
Prior to 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) , 6.4: forward mode computed the directional derivative of the absolute value function which is defined by $$\R{abs}^\circ ( x , d ) = \lim_{\lambda \downarrow 0 } \frac{\R{abs}(x + \lambda d) - \R{abs}(x) }{ \lambda }$$ For $x \neq 0$, $$\R{abs}^\circ ( x , d ) = \R{abs}^{(1)} ( x ) * d$$ and $\R{abs}^\circ (0 , d) = |d|$.

5.8.g: Example
The file 5.8.1: abs.py contains an example and test of this function.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.8.1: abs: Example and Test

 # Example using a_float ---------------------------------------------------- from pycppad import * def pycppad_test_abs() : x = numpy.array( [ -1., 0., 1.] ) n = len(x) a_x = independent(x) a_y = abs( a_x ) f = adfun(a_x, a_y) f.forward(0, x) dx = numpy.zeros(n, dtype=float) for i in range( n ) : dx[i] = 1. df = f.forward(1, dx) if x[i] > 0. : assert df[i] == +1. elif x[i] < 0. : assert df[i] == -1. else : # There was a change in the CppAD specifictions for the abs function # see 12-30 on http://www.coin-or.org/CppAD/Doc/whats_new_11.htm assert df[i] == +1. or df[i] == 0. dx[i] = -1. df = f.forward(1, dx) if x[i] > 0. : assert df[i] == -1. elif x[i] < 0. : assert df[i] == +1. else : assert df[i] == +1 or df[i] == 0. dx[i] = 0. # Example using a2float ---------------------------------------------------- def pycppad_test_abs_a2() : x = ad( numpy.array( [-1, 0, 1] ) ) n = len(x) a_x = independent(x) a_y = abs( a_x ) f = adfun(a_x, a_y) f.forward(0, x) dx = numpy.array( list( ad(0) for i in range(n) ) ) for i in range( n ) : dx[i] = ad(0) for i in range( n ) : dx[i] = ad(1) df = f.forward(1, dx) if x[i] > 0. : assert df[i] == +1. elif x[i] < 0. : assert df[i] == -1. else : assert df[i] == +1. or df[i] == 0. dx[i] = ad(-1) df = f.forward(1, dx) if x[i] > 0. : assert df[i] == -1. elif x[i] < 0. : assert df[i] == +1. else: assert df[i] == +1. or df[i] == 0. dx[i] = ad(0) 
Input File: example/abs.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.9: Conditional Expressions

Syntax
Purpose
rel
left
right
if_true
if_false
result
Example

5.9.a: Syntax
result = condexp_rel(left, right, if_true, if_false)

5.9.b: Purpose
Record, as part of an operation sequence, the conditional result       if( left op right )           result = if_true      else result = if_false  The relation rel% , and operator op , have the following correspondence:       rel   lt   le   eq   ge   gt       op    <   <=   ==    >   >= 
5.9.c: rel
In the syntax above, the relation rel represents one of the following two characters: lt, le, eq, ge, gt. As in the table above, rel determines which comparison operator op is used when comparing left and right .

5.9.d: left
The argument left must have type a_float or a2float. It specifies the value for the left side of the comparison operator.

5.9.e: right
The argument right must have the same type as left . It specifies the value for the right side of the comparison operator.

5.9.f: if_true
The argument if_true must have the same type as left . It specifies the return value if the result of the comparison is true.

5.9.g: if_false
The argument if_false must have the same type as left . It specifies the return value if the result of the comparison is false.

5.9.h: result
This result has the same type as left .

5.9.i: Example
The file 5.9.1: condexp.py contains an example and test of these functions.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
5.9.1: condexp: Example and Test

 # Example using a_float ---------------------------------------------------- from pycppad import * def pycppad_test_condexp() : x = numpy.array( [1. , 1., 3., 4. ] ) a_x = independent(x) a_left = a_x[0]; a_right = a_x[1]; a_if_true = a_x[2]; a_if_false = a_x[3]; a_y_lt = condexp_lt(a_left, a_right, a_if_true, a_if_false); a_y_le = condexp_le(a_left, a_right, a_if_true, a_if_false); a_y_eq = condexp_eq(a_left, a_right, a_if_true, a_if_false); a_y_ge = condexp_ge(a_left, a_right, a_if_true, a_if_false); a_y_gt = condexp_gt(a_left, a_right, a_if_true, a_if_false); a_y = numpy.array( [ a_y_lt, a_y_le, a_y_eq, a_y_ge, a_y_gt ] ); f = adfun(a_x, a_y) y = f.forward(0, x) assert ( y[0] == 4. ) # 1 < 1 is false so result is 4 assert ( y[1] == 3. ) # 1 <= 1 is true so result is 3 assert ( y[2] == 3. ) # 1 == 1 is true so result is 3 assert ( y[3] == 3. ) # 1 >= 1 is true so result is 3 assert ( y[4] == 4. ) # 1 > 2 is false so result is 4 x = numpy.array( [4., 3., 2., 1.] ) y = f.forward(0, x) assert ( y[0] == 1. ) # 4 < 3 is false so result is 1 assert ( y[1] == 1. ) # 4 <= 3 is false so result is 1 assert ( y[2] == 1. ) # 4 == 3 is false so result is 1 assert ( y[3] == 2. ) # 4 >= 3 is true so result is 2 assert ( y[4] == 2. ) # 4 > 3 is true so result is 2 # Example using a2float ---------------------------------------------------- def pycppad_test_condexp_a2() : x = numpy.array( [1. , 1., 3., 4. ] ) a_x = ad(x) # begin level two recording of conditional expression a2x = independent(a_x) a2left = a2x[0]; a2right = a2x[1]; a2if_true = a2x[2]; a2if_false = a2x[3]; a2y_lt = condexp_lt(a2left, a2right, a2if_true, a2if_false); a2y_le = condexp_le(a2left, a2right, a2if_true, a2if_false); a2y_eq = condexp_eq(a2left, a2right, a2if_true, a2if_false); a2y_ge = condexp_ge(a2left, a2right, a2if_true, a2if_false); a2y_gt = condexp_gt(a2left, a2right, a2if_true, a2if_false); a2y = numpy.array( [ a2y_lt, a2y_le, a2y_eq, a2y_ge, a2y_gt ] ); a_f = adfun(a2x, a2y) # begin level one recording of conditional expression a_x = independent(x) a_y = a_f.forward(0, a_x) f = adfun(a_x, a_y) y = f.forward(0, x) assert ( y[0] == 4. ) # 1 < 1 is false so result is 4 assert ( y[1] == 3. ) # 1 <= 1 is true so result is 3 assert ( y[2] == 3. ) # 1 == 1 is true so result is 3 assert ( y[3] == 3. ) # 1 >= 1 is true so result is 3 assert ( y[4] == 4. ) # 1 > 2 is false so result is 4 x = numpy.array( [4., 3., 2., 1.] ) y = f.forward(0, x) assert ( y[0] == 1. ) # 4 < 3 is false so result is 1 assert ( y[1] == 1. ) # 4 <= 3 is false so result is 1 assert ( y[2] == 1. ) # 4 == 3 is false so result is 1 assert ( y[3] == 2. ) # 4 >= 3 is true so result is 2 assert ( y[4] == 2. ) # 4 > 3 is true so result is 2 
Input File: example/condexp.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6: AD Function Methods

Contents

6.a: Contents
 independent: 6.1 Create an Independent Variable Vector adfun: 6.2 Create an AD Function Object abort_recording: 6.3 Abort a Recording of AD Operations forward: 6.4 Forward Mode: Derivative in One Domain Direction reverse: 6.5 Reverse Mode: Derivative in One Range Direction jacobian: 6.6 Driver for Computing Entire Derivative hessian: 6.7 Driver for Computing Hessian in a Range Direction optimize: 6.8 Optimize an AD Function Object Tape

$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.1: Create an Independent Variable Vector

Syntax
Purpose
x
a_x
Example

6.1.a: Syntax
a_x = independent(x)

6.1.b: Purpose
Creates an independent variable vector and starts recording operations involving objects that are instances of type(a_x[0]) . You must create an 6.2: adfun object, or use 6.3: abort_recording , to stop the recording before making another call to independent,

6.1.c: x
The argument x must be a numpy.array with one dimension (i.e., a vector). All the elements of x must all be of the same type and instances of either int, float or  a_float.

6.1.d: a_x
The return value a_x is a numpy.array with the same shape as x . If the elements of x are instances of int or float the elements of a_x are instances of a_float. If the elements of x are instances of a_float the elements of a_x are instances of a2float. The 5.2: value of the elements of a_x are equal to the corresponding elements of x .

6.1.e: Example
The file 6.1.1: independent.py contains an example and test of this function.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.1.1: independent: Example and Test

 from pycppad import * # Example using a_float --------------------------------------------------- def pycppad_test_independent() : x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # level 1 independent variables and start recording assert type(a_x) == numpy.ndarray for j in range(len(x)) : assert isinstance(x[j], float) assert isinstance(a_x[j], a_float) assert a_x[j] == x[j] f = adfun(a_x, a_x) # stop level 1 recording # Example using a2float --------------------------------------------------- def pycppad_test_independent_a2() : x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # level 1 independent variables and start recording a2x = independent(a_x) # level 2 independent variables and start recording assert type(a_x) == numpy.ndarray assert type(a2x) == numpy.ndarray for j in range(len(x)) : assert isinstance(x[j], float) assert isinstance(a_x[j], a_float) assert isinstance(a2x[j], a2float) assert a_x[j] == x[j] assert a2x[j] == x[j] a_f = adfun(a2x, a2x) # stop level 2 recording f = adfun(a_x, a_x) # stop level 1 recording 
Input File: example/independent.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.2: Create an AD Function Object

Syntax
Purpose
a_x
a_y
f
m
n
level
Example

6.2.a: Syntax
f = adfun(a_x, a_y)

6.2.b: Purpose
The function object f will store the type( a_x[0] ) operation sequence that mapped the independent variable vector a_x to the dependent variable vector a_y .

6.2.c: a_x
The argument a_x is the numpy.array returned by the previous call to 6.1: independent . Neither the size of a_x , or the value it its elements, may change between calling       a_x = independent(x)  and       f = adfun(a_x, a_y)  The length of the vector a_x determines the domain size $n$ for the function $y = F(x)$ below.

6.2.d: a_y
The argument a_y specifies the dependent variables. It must be a numpy.array with one dimension (i.e., a vector) and with the same type of elements as a_x . The object f stores the type( a_x[0] ) operations that mapped the vector a_x to the vector a_y . The length of the vector a_y determines the range size $m$ for the function $y = F(x)$ below.

6.2.e: f
The return value f can be used to evaluate the function $$F : \B{R}^n \rightarrow \B{R}^m$$ and its derivatives, where $y = F(x)$ corresponds to the operation sequence mentioned above.

6.2.e.a: m
The range size $m$ is equal to the length of the vector a_y .

6.2.e.b: n
The domain size $n$ is equal to the length of the vector a_x .

6.2.e.c: level
The 5.1: ad level for the object f is one less than the AD level for the arguments a_x and a_y ; i.e., if type( a_x[0] ) is a_float (a2float) the corresponding AD level for f is zero (one).

6.2.f: Example
The file 6.2.1: adfun.py contains an example and test of this function.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.2.1: adfun: Example and Test

 from pycppad import * def pycppad_test_adfun() : # record operations at x = (0, 0, 0) x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # declare independent variables and start recording a_y0 = a_x[0]; a_y1 = a_x[0] * a_x[1]; a_y2 = a_x[0] * a_x[1] * a_x[2]; a_y = numpy.array( [ a_y0, a_y1, a_y2 ] ) f = adfun(a_x, a_y) # declare dependent variables and stop recording # evaluate function at x = (1, 2, 3) x = numpy.array( [ 1., 2., 3. ] ) y = f.forward(0, x) assert y[0] == x[0] assert y[1] == x[0] * x[1] assert y[2] == x[0] * x[1] * x[2] 
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.3: Abort a Recording of AD Operations

Syntax
Purpose
Example

6.3.a: Syntax
abort_recording()

6.3.b: Purpose
Sometimes it is necessary to abort the recording of AD operations that started with a call of the form       a_x = independent(x)  If such a recording is currently in progress, this will stop the recording and delete the corresponding information. Otherwise, abort_recording has no effect.

6.3.c: Example
The file 6.3.1: abort_recording.py contains an example and test of this operation. It returns true if it succeeds and false otherwise.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.3.1: abort_recording: Example and Test

 from pycppad import * # Example using a_float --------------------------------------------------- def pycppad_test_abort_recording() : from numpy import array try : x = numpy.array( [ 1., 2., 3. ] ) a_x = independent(x) # start first level recording a2_x = independent(a_x) # start second level recording a_y = array([sum(a_x)]) # record some operations if a_y[0] > 2 : raise ValueError except ValueError : # Pretend that we are not sure if there are any active recordings # and use this call to terminate any that may exist. abort_recording() a_x = independent(x) # test starting a level 1 recording a2_x = independent(a_x) # test starting a level 2 recording a_y = array([sum(a_x)]) # record some level 1 operations f = adfun(a_x, a_y) # terminate level 1 recording y = f.forward(0, x) # evaluate the function at original x value assert( y[0] == 6. ) # check the value abort_recording() # abort the level 2 recording 
Input File: example/abort_recording.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.4: Forward Mode: Derivative in One Domain Direction

Syntax
Purpose
x_k
y_k
f
p
x_p
y_p
Example

6.4.a: Syntax
y_p = f.forward(p, x_p)

6.4.b: Purpose
We use $F : \B{R}^n \rightarrow \B{R}^m$ to denote the function corresponding to the adfun object 6.2.e: f . Given the p-th order Taylor expansion for a function $X : \B{R} \rightarrow \B{R}^n$, this function can be used to compute the p-th order Taylor expansion for the function $Y : \B{R} \rightarrow \B{R}^m$ defined by $$Y(t) = F [ X(t) ]$$

6.4.c: x_k
For $k = 0 , \ldots , p$, we use $x^{(k)}$ to denote the value of x_k in the most recent call to       f.forward(k, x_k)  including $x^{(p)}$ as the value x_p in this call. We define the function $X(t)$ by $$X(t) = x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p$$

6.4.d: y_k
For $k = 0 , \ldots , p$, we use $y^{(k)}$ to denote the Taylor coefficients for $Y(t) = F[ X(t) ]$ expanded about zero; i.e., $$\begin{array}{rcl} y^{(k)} & = & Y^{(k)} (0) / k ! \\ Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p ) \end{array}$$ where $o( t^p ) / t^p \rightarrow 0$ as $t \rightarrow 0$. The coefficient $y^{(p)}$ is equal to the value y_p returned by this call.

6.4.e: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.4.f: p
The argument p is a non-negative int. It specifies the order of the Taylor coefficient for $Y(t)$ that is computed.

6.4.g: x_p
The argument x_p is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the p-th order Taylor coefficient for $X(t)$. If the AD 6.2.e.c: level for f is zero, all the elements of x_p must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x_p must be a_float objects.

6.4.h: y_p
The return value y_p is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It is set to the p-th order Taylor coefficient for $Y(t)$. If the AD 6.2.e.c: level for f is zero, all the elements of y_p will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of y_p will be a_float objects.

6.4.i: Example
 6.4.1: forward_0.py Forward Order Zero: Example and Test 6.4.2: forward_1.py Forward Order One: Example and Test

$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.4.1: Forward Order Zero: Example and Test

 from pycppad import * # Example using a_float ---------------------------------------------------- def pycppad_test_forward_0() : # start record a_float operations x = numpy.array( [ 2., 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1 # Example using a2float ---------------------------------------------------- def pycppad_test_forward_0_a2() : # start record a_float operations a_x = ad(numpy.array( [ 2., 3. ] )) # a_float value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad( numpy.array( [ 3. , 4. ] ) ) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp[0] == 2. * a_x[0] * a_x[1] # f(x0, x1) = 2 * x0 * x1 
Input File: example/forward_0.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.4.2: Forward Order One: Example and Test

 from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_forward_1() : # start record a_float operations x = numpy.array( [ 2., 3. ] ) # value of independent variables a_x = independent(x) # declare independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative of f(x0, x1) with respect to x0 p = 1 # order one for first derivatives xp = numpy.array( [ 1. , 0. ] ) # direction for differentiation fp = f.forward(p, xp) # value of directional derivative assert fp[0] == 2. * x[1] # f_x0 (x0, x1) = 2 * x1 # evalute partial derivative of f(x0, x1) with respect to x p = 1 xp = numpy.array( [ 0. , 1. ] ) # the x1 direction fp = f.forward(p, xp) assert fp[0] == 2. * x[0] # f_x1 (x0, x1) = 2 * x0 # Example using a2float ----------------------------------------------------- def pycppad_test_forward_1_a2() : # start record a_float operations a_x = ad(numpy.array( [ 2., 3. ] )) # a_float value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad(numpy.array( [ 3. , 4. ] )) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp[0] == 2. * a_x[0] * a_x[1] # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative of f(x0, x1) with respect to x0 p = 1 # order one for first derivatives a_xp = ad(numpy.array( [ 1. , 0. ] )) # direction for differentiation a_fp = a_f.forward(p, a_xp) # value of directional derivative assert a_fp[0] == 2. * a_x[1] # f_x0 (x0, x1) = 2 * x1 # evalute partial derivative of f(x0, x1) with respect to x p = 1 a_xp = ad(numpy.array( [ 0. , 1. ] )) # the x1 direction a_fp = a_f.forward(p, a_xp) assert a_fp[0] == 2. * a_x[0] # f_x1 (x0, x1) = 2 * x0 
Input File: example/forward_1.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.5: Reverse Mode: Derivative in One Range Direction

Syntax
Purpose
x_k
X(t, u)
W(t, u)
f
p
w
dw
First Order
Second Order
Example

6.5.a: Syntax
dw = f.forward(p, w)

6.5.b: Purpose
Reverse mode computes the derivative of the 6.4: forward more Taylor coefficients with respect to the domain variable $x$.

6.5.c: x_k
For $k = 0 , \ldots , p$, we use $x^{(k)}$ to denote the value of x_k in the most recent call to       f.forward(k, x_k)  We use $F : \B{R}^n \rightarrow \B{R}^m$ to denote the function corresponding to the adfun object 6.2.e: f .

6.5.d: X(t, u)
We define the function $X : \B{R} \times \B{R}^n \rightarrow \B{R}^n$ by $$X(t, u) = u + x^{(0)} + x^{(1)} * t + \cdots + x^{(p-1)} * t^{p-1}$$ Note that for $k = 0 , \ldots , p - 1$, $$x^{(k)} = \frac{1}{k !} \frac{\partial^k}{\partial t^k} X(0, 0)$$

6.5.e: W(t, u)
The function $W : \B{R} \times \B{R}^n \rightarrow \B{R}$ is defined by $$W(t, u) = w_0 * F_0 [ X(t, u) ] + \cdots + w_{m-1} * F_{m-1} [ X(t, u) ]$$ We define the function $W_k : \B{R}^n \rightarrow \B{R}$ by $$W_k ( u ) = \frac{1}{k !} \frac{\partial^k}{\partial t^k} W(0, u)$$ It follows that $$W(t, u ) = W_0 ( u ) + W_1 ( u ) * t + \cdots + W_{p-1} (u) * t^{p-1} + o( t^{p-1} )$$ where $o( t^{p-1} ) / t^{p-1} \rightarrow 0$ as $t \rightarrow 0$.

6.5.f: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.5.g: p
The argument p is a non-negative int. It specifies the order of the Taylor coefficient $W_{p-1} ( u )$ that is differentiated. Note that $W_{p-1} (u)$ corresponds a derivative of order $p-1$ of $F(x)$, so the derivative of $W_{p-1} (u)$ corresponds to a derivative of order $p$ of $F(x)$.

6.5.h: w
The argument w is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It specifies the weighting vector $w$ used in the definition of $W(t, u)$. If the AD 6.2.e.c: level for f is zero, all the elements of w must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of w must be a_float objects.

6.5.i: dw
The return value v is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It is set to the derivative $$\begin{array}{rcl} dw & = & W_{p-1}^{(1)} ( 0 ) \\ & = & \partial_u \frac{1}{(p-1) !} \frac{\partial^{p-1}}{\partial t^{p-1}} W(0, 0) \end{array}$$ If the AD 6.2.e.c: level for f is zero, all the elements of dw will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of dw will be a_float objects.

6.5.j: First Order
In the case where $p = 1$, we have $$\begin{array}{rcl} dw & = & \partial_u \frac{1}{0 !} \frac{\partial^0}{\partial t^0} W(0, 0) \\ & = & \partial_u W(0, 0) \\ & = & \partial_u \left[ w_0 * F_0 ( u + x^{(0)} ) + \cdots + w_{m-1} F_{m-1} ( u + x^{(0)} ) \right]_{u = 0} \\ & = & w_0 * F_0^{(1)} ( x^{(0)} ) + \cdots + w_{m-1} * F_{m-1}^{(1)} ( x^{(0)} ) \end{array}$$

6.5.k: Second Order
In the case where $p = 2$, we have $$\begin{array}{rcl} dw & = & \partial_u \frac{1}{1 !} \frac{\partial^1}{\partial t^1} W (0, 0) \\ & = & \partial_u \left[ w_0 * F_0^{(1)} ( u + x^{(0)} ) * x^{(1)} + \cdots + w_{m-1} * F_{m-1}^{(1)} ( u + x^{(0)} ) * x^{(1)} \right]_{u = 0} \\ & = & w_0 * ( x^{(1)} )^\R{T} * F_0^{(2)} ( x^{(0)} ) + \cdots + w_{m-1} * ( x^{(1)} )^\R{T} * F_{m-1}^{(2)} ( x^{(0)} ) \end{array}$$

6.5.l: Example
 6.5.1: reverse_1.py Reverse Order One: Example and Test 6.5.2: reverse_2.py Reverse Order Two: Example and Test

$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.5.1: Reverse Order One: Example and Test

 from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_reverse_1(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1 # evalute derivative of f(x0, x1) p = 1 # order one for first derivatives w = numpy.array( [ 1. ] ) # weight in range space fp = f.reverse(p, w) # derivaitive of weighted function assert fp[0] == 2. * x[1] # f_x0 (x0, x1) = 2 * x1 assert fp[1] == 2. * x[0] # f_x1 (x0, x1) = 2 * x0 # Example using a2float ------------------------------------------------------ def pycppad_test_reverse_1_a2(): # start record a_float operations a_x = ad(numpy.array( [ 2. , 3. ] )) # value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad(numpy.array( [ 3. , 4. ] )) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp[0] == 2. * a_x[0] * a_x[1] # f(x0, x1) = 2 * x0 * x1 # evalute derivative of f(x0, x1) p = 1 # order one for first derivatives a_w = ad(numpy.array( [ 1. ] )) # weight in range space a_fp = a_f.reverse(p, a_w) # derivaitive of weighted function assert a_fp[0] == 2. * a_x[1] # f_x0 (x0, x1) = 2 * x1 assert a_fp[1] == 2. * a_x[0] # f_x1 (x0, x1) = 2 * x0 
Input File: example/reverse_1.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.5.2: Reverse Order Two: Example and Test

 from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_reverse_2(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at same argument value p = 0 # derivative order x_p = x # zero order Taylor coefficient f_p = f.forward(0, x_p) # function value assert f_p[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative with respect to x[0] p = 1 # derivative order x_p = numpy.array( [ 1. , 0 ] ) # first order Taylor coefficient f_p = f.forward(1, x_p) # partial w.r.t. x0 assert f_p[0] == 2. * x[1] # f_x0 (x0, x1) = 2 * x1 # evaluate derivative of partial w.r.t. x[0] p = 2 # derivative order w = numpy.array( [1.] ) # weighting vector dw = f.reverse(p, w) # derivaitive of weighted function assert dw[0] == 0. # f_x0_x1 (x0, x1) = 0 assert dw[1] == 2. # f_x0_x1 (x0, x1) = 2 # Example using a2float ------------------------------------------------------ def pycppad_test_reverse_2_a2(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = ad(x) # value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at same argument value p = 0 # derivative order x_p = a_x # zero order Taylor coefficient f_p = a_f.forward(0, x_p) # function value assert f_p[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative with respect to x[0] p = 1 # derivative order x_p = ad(numpy.array([1. , 0 ])) # first order Taylor coefficient f_p = a_f.forward(1, x_p) # partial w.r.t. x0 assert f_p[0] == 2. * x[1] # f_x0 (x0, x1) = 2 * x1 # evaluate derivative of partial w.r.t. x[0] p = 2 # derivative order w = ad(numpy.array( [1.] )) # weighting vector dw = a_f.reverse(p, w) # derivaitive of weighted function assert dw[0] == 0. # f_x0_x1 (x0, x1) = 0 assert dw[1] == 2. # f_x0_x1 (x0, x1) = 2 
Input File: example/reverse_2.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.6: Driver for Computing Entire Derivative

Syntax
Purpose
f
x
J
Example

6.6.a: Syntax
J = f.jacobian(x)

6.6.b: Purpose
This routine computes the entire derivative $F^{(1)} (x)$ where $F : \B{R}^n \rightarrow \B{R}^m$ is the function corresponding to the adfun object 6.2.e: f .

6.6.c: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.6.d: x
The argument x is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of x must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x must be a_float objects.

6.6.e: J
The return value J is a numpy.array with two dimensions (i.e., a matrix). The first dimension (row size) is equal to 6.2.e.a: m (the number of range components in the function f ). The second dimension (column size) is equal to 6.2.e.b: n (the number of domain components in the function f ). It is set to the derivative; i.e., $$J = F^{(1)} (x)$$ If the AD 6.2.e.c: level for f is zero, all the elements of J will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of J will be a_float objects.

6.6.f: Example
The file 6.6.1: jacobian.py contains an example and test of this operation.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.6.1: Entire Derivative: Example and Test

 from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_jacobian(): delta = 10. * numpy.finfo(float).eps x = numpy.array( [ 0., 0. ] ) a_x = independent(x) a_y = numpy.array( [ a_x[0] * exp(a_x[1]) , a_x[0] * sin(a_x[1]) , a_x[0] * cos(a_x[1]) ] ) f = adfun(a_x, a_y) x = numpy.array( [ 2., 3. ] ) J = f.jacobian(x) assert abs( J[0,0] - exp(x[1]) ) < delta assert abs( J[0,1] - x[0] * exp(x[1]) ) < delta assert abs( J[1,0] - sin(x[1]) ) < delta assert abs( J[1,1] - x[0] * cos(x[1]) ) < delta assert abs( J[2,0] - cos(x[1]) ) < delta assert abs( J[2,1] + x[0] * sin(x[1]) ) < delta # Example using a2float ----------------------------------------------------- def pycppad_test_jacobian_a2(): delta = 10. * numpy.finfo(float).eps a_x = ad( numpy.array( [ 0., 0. ] ) ) a2x = independent(a_x) a2y = numpy.array( [ a2x[0] * exp(a2x[1]) , a2x[0] * sin(a2x[1]) , a2x[0] * cos(a2x[1]) ] ) a_f = adfun(a2x, a2y) x = numpy.array( [2., 3.] ) a_x = ad(x) a_J = a_f.jacobian(a_x) assert abs( a_J[0,0] - exp(x[1]) ) < delta assert abs( a_J[0,1] - x[0] * exp(x[1]) ) < delta assert abs( a_J[1,0] - sin(x[1]) ) < delta assert abs( a_J[1,1] - x[0] * cos(x[1]) ) < delta assert abs( a_J[2,0] - cos(x[1]) ) < delta assert abs( a_J[2,1] + x[0] * sin(x[1]) ) < delta 
Input File: example/jacobian.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.7: Driver for Computing Hessian in a Range Direction

Syntax
Purpose
f
x
w
H
Example

6.7.a: Syntax
H = f.hessian(x, w)

6.7.b: Purpose
This routine computes the Hessian of the weighted sum $$w_0 * F_0 (x) + \cdots + w_{m-1} * F_{m-1} (x)$$ where $F : \B{R}^n \rightarrow \B{R}^m$ is the function corresponding to the adfun object 6.2.e: f .

6.7.c: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.7.d: x
The argument x is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of x must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x must be a_float objects.

6.7.e: w
The argument w is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of w must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of w must be a_float objects.

6.7.f: H
The return value H is a numpy.array with two dimensions (i.e., a matrix). Both its first and second dimension size (row and column size) are equal to 6.2.e.b: n (the number of domain components in the function f ). It is set to the Hessian; i.e., $$H = w_0 * F_0^{(2)} (x) + \cdots + w_{m-1} * F_{m-1}^{(2)} (x)$$ If the AD 6.2.e.c: level for f is zero, all the elements of H will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of H will be a_float objects.

6.7.g: Example
The file 6.7.1: hessian.py contains an example and test of this operation.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.7.1: Hessian Driver: Example and Test

 from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_hessian(): delta = 10. * numpy.finfo(float).eps x = numpy.array( [ 0., 0. ] ) a_x = independent(x) a_y = numpy.array( [ a_x[0] * exp(a_x[1]) , a_x[0] * sin(a_x[1]) , a_x[0] * cos(a_x[1]) ] ) f = adfun(a_x, a_y) x = numpy.array( [ 2., 3. ] ) w = numpy.array( [ 0., 1., 0. ] ) # compute Hessian of x0 * sin(x1) H = f.hessian(x, w) assert abs( H[0,0] - 0. ) < delta assert abs( H[0,1] - cos(x[1]) ) < delta assert abs( H[1,0] - cos(x[1]) ) < delta assert abs( H[1,1] + x[0] * sin(x[1]) ) < delta # Example using a2float ----------------------------------------------------- def pycppad_test_hessian_a2(): delta = 10. * numpy.finfo(float).eps a_x = ad( numpy.array( [ 0., 0. ] ) ) a2x = independent(a_x) a2y = numpy.array( [ a2x[0] * exp(a2x[1]) , a2x[0] * sin(a2x[1]) , a2x[0] * cos(a2x[1]) ] ) a_f = adfun(a2x, a2y) x = numpy.array( [ 2., 3. ] ) a_x = ad(x) a_w = ad( numpy.array( [ 0., 1., 0. ] ) ) # compute Hessian of x0 * sin(x1) a_H = a_f.hessian(a_x, a_w) assert abs( a_H[0,0] - 0. ) < delta assert abs( a_H[0,1] - cos(x[1]) ) < delta assert abs( a_H[1,0] - cos(x[1]) ) < delta assert abs( a_H[1,1] + x[0] * sin(x[1]) ) < delta 
Input File: example/hessian.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.8: Optimize an AD Function Object Tape

Syntax
Purpose
f
Efficiency
Example

6.8.a: Syntax
f.optimize()

6.8.b: Purpose
The operation sequence corresponding to an 6.2: ADFun object can be very large and involve many operations. The f.optimize procedure reduces the number of operations, and thereby the time and memory, required to compute function and derivative values.

6.8.c: f
The object f is an 6.2: adfun object.

6.8.d: Efficiency
The optimize member function may greatly reduce the size of the operation sequence corresponding to f .

6.8.e: Example
The file 6.8.1: optimize.py contains an example and test of this operation.
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
6.8.1: Optimize Function Object: Example and Test

 from pycppad import * import time # Example using a_float ----------------------------------------------------- def pycppad_test_optimize(): # create function with many variables that are get removed by optimize n_sum = 10000 x = numpy.array( [ 0. ] ) a_x = independent(x) a_sum = 0. for i in range(n_sum) : a_sum = a_sum + a_x[0]; a_y = numpy.array( [ a_sum ] ) f = adfun(a_x, a_y) # time for a forward operations before optimize x = numpy.array( [ 1. ] ) t0 = time.time() sum_before = f.forward(0, x) sec_before = time.time() - t0 # time for a forward operations after optimize f.optimize() t0 = time.time() sum_after = f.forward(0, x) sec_after = time.time() - t0 assert sum_before == float(n_sum) assert sum_after == float(n_sum) # expect sec_before to be less than 2 times sec_after assert( sec_after * 1.5 <= sec_before ) 
Input File: example/optimize.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
7: Using Two Levels of AD: Example and Test

Purpose
F(u)
G(x)

7.a: Purpose
This example is intended to demonstrate how a_float and a2float can be used together to compute derivatives of functions that are defined in terms of derivatives of other functions.

7.b: F(u)
For this example, the function $F : \B{R}^2 \rightarrow \B{R}$ is defined by $$F(u) = u_0^2 + u_1^2$$ It follows that $$\begin{array}{rcl} \partial_{u(0)} F(u) = 2 * u_0 \\ \partial_{u(1)} F(u) = 2 * u_1 \end{array}$$

7.c: G(x)
For this example, the function $G : \B{R}^2 \rightarrow \B{R}$ is defined by $$G(x) = x_1 * \partial_{u(0)} F(x_0 , 1) + x_0 * \partial_{u(1)} F(x_0, 1)$$ where $\partial{u(j)} F(a, b)$ denotes the partial of $F$ with respect to $u_j$ and evaluated at $u = (a, b)$. It follows that $$\begin{array}{rcl} G (x) & = & 2 * x_1 * x_0 + 2 * x_0 \\ \partial_{x(0)} G (x) & = & 2 * x_1 + 2 \\ \partial_{x(1)} G (x) & = & 2 * x_0 \end{array}$$  from pycppad import * def pycppad_test_two_levels(): # start recording a_float operations x = numpy.array( [ 2. , 3. ] ) a_x = independent(x) # start recording a2float operations a_u = numpy.array( [a_x[0] , ad(1) ] ) a2u = independent(a_u) # stop a2float recording and store operations if f a2v = numpy.array( [ a2u[0] * a2u[0] + a2u[1] * a2u[1] ] ) a_f = adfun(a2u, a2v) # F(u0, u1) = u0 * u0 + u1 * u1 # evaluate the gradient of F a_J = a_f.jacobian(a_u) # stop a_float recording and store operations in g a_y = numpy.array( [ a_x[1] * a_J[0,0] + a_x[0] * a_J[0,1] ] ) g = adfun(a_x, a_y) # G(x0, x1) = x1 * F_u0(x0, 1) + x0 * F_u1(x0, 1) # evaluate the gradient of G J = g.jacobian(x) assert J[0,0] == 2. * x[1] + 2 assert J[0,1] == 2. * x[0] 
Input File: example/two_levels.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
8: Fourth Order Runge Kutta

Syntax
Purpose
f
ti
yi
dt
yf
Motivation
Source Code
Example

8.a: Syntax
 yf = runge_kutta_4(f, ti, yi, dt)

8.b: Purpose
See 8.h: Motivation below as well as the purpose described here. We are given a function $f : \B{R}^n \rightarrow \B{R}^n$, and a point $yi \in \B{R}^n$ such that an unknown function $y : \B{R} \rightarrow \B{R}^n$ satisfies the equations $$\begin{array}{rcl} y( ti ) & = & yi \\ y'(t) & = & f[t, y(t) ] \\ \end{array}$$ We use the Fourth order Runge-Kutta formula (see equation 16.1.2 of Numerical Recipes in Fortran, 2nd ed.) wish to approximate the value of $$yf = y( ti + \Delta t )$$

8.c: f
If t is a scalar and y is a vector with size $n$,       k = f(t, y)  returns a vector of size $n$ that is the value of $f(t, y)$ at the specified values.

8.d: ti
is a scalar that specifies the value of $ti$ in the problem above.

8.e: yi
is a vector of size $n$ that specifies the value of $yi$ in the problem above.

8.f: dt
is a scalar that specifies the value of $\Delta t$ in the problem above.

8.g: yf
is a vector of size $n$ that is the approximation for $y( t + \Delta t )$.

8.h: Motivation
This routine makes very few assumptions about the objects used to do these calculations. Thus, smart objects can be used for all sorts of purposes; for example, computing derivatives of the solution of an ODE. The table below lists the assumptions on the objects passed into runge_kutta_4. In this table, s and t are scalars, d is a decimal number (i.e., a float) and u and v are vectors with size $n$.
 operation result d * s scalar s + t scalar s * u vector with size $n$ d * u vector with size $n$ s * u vector with size $n$ u + v vector with size $n$

8.i: Source Code  def runge_kutta_4(f, ti, yi, dt) : k1 = dt * f(ti , yi) k2 = dt * f(ti + .5*dt , yi + .5*k1) k3 = dt * f(ti + .5*dt , yi + .5*k2) k4 = dt * f(ti + dt , yi + k3) yf = yi + (1./6.) * ( k1 + 2.*k2 + 2.*k3 + k4 ) return yf 
8.j: Example
1. The file 8.1: runge_kutta_4_correct.py contains an example and test of using runge_kutta_4 to solve an ODE.
2. The file 8.2: runge_kutta_4_ad.py contains an example and test of differentiating the numerical solution of an ODE.
3. The file 8.3: runge_kutta_4_cpp.py contains an example and test of using pycppad 6.2: adfun object to evaluate python functions with C++ speed of execution.

$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
8.1: runge_kutta_4 A Correctness Example and Test

Discussion
Source Code

8.1.a: Discussion
Define $y : \B{R} \rightarrow \B{R}^n$ by $$y_j (t) = t^{j+1}$$ It follows that the derivative of $y(t)$ satisfies the 8: runge_kutta_4 ODE equation where $y(0) = 0$ and $f(t, y)$ is given by $$f(t , y)_j = y_j '(t) = \left\{ \begin{array}{ll} 1 & {\; \rm if \;} j = 0 \\ (j+1) y_{j-1} (t) & {\; \rm otherwise } \end{array} \right.$$

8.1.b: Source Code  from pycppad import * def pycppad_test_runge_kutta_4_correct() : def fun(t , y) : n = y.size f = numpy.zeros(n) f[0] = 1. index = numpy.array( range(n-1) ) + 1 f[index] = (index + 1) * y[index-1] return f n = 5 # size of y(t) (order of method plus 1) ti = 0. # initial time dt = 2. # a very large time step size to test correctness yi = numpy.zeros(n) # initial value for y(t); i.e., y(0) # take one 4-th order Runge-Kutta integration step of size dt yf = runge_kutta_4(fun, ti, yi, dt) # check the results t_jp = 1. # t^0 at t = dt for j in range(n-1) : t_jp = t_jp * dt # t^(j+1) at t = dt assert abs( yf[j] - t_jp ) < 1e-10 # check yf[j] = t^(j+1) 
Input File: example/runge_kutta_4_correct.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
8.2: runge_kutta_4 An AD Example and Test

Discussion
Source Code

8.2.a: Discussion
Define $y : \B{R} \times \B{R} \rightarrow \B{R}^n$ by $$y_j (x, t) = x t^{j+1}$$ It follows that the derivative of $y(t)$ satisfies the 8: runge_kutta_4 ODE equation where $y(0) = 0$ and $f(t, y)$ is given by $$f(t , y)_j = \partial_t y_j (x, t) = \left\{ \begin{array}{ll} x & {\; \rm if \;} j = 0 \\ (j+1) y_{j-1} (x, t) & {\; \rm otherwise } \end{array} \right.$$ It also follows that $$\partial_x y_j (x, t) = t^{j+1}$$

8.2.b: Source Code  from pycppad import * def pycppad_test_runge_kutta_4_ad() : def fun(t , y) : n = y.size f = ad( numpy.zeros(n) ) f[0] = a_x[0] index = numpy.array( range(n-1) ) + 1 f[index] = (index + 1) * y[index-1] return f n = 5 # size of y(t) (order of method plus 1) ti = 0. # initial time dt = 2. # a very large time step size (method is exact) # initial value for y(t); i.e., y(0) a_yi = ad( numpy.zeros(n) ) # declare a_x to be the independent variable vector x = numpy.array( [.5] ) a_x = independent( numpy.array( x ) ) # take one 4-th order Runge-Kutta integration step of size dt a_yf = runge_kutta_4(fun, ti, a_yi, dt) # define the AD function g : x -> yf g = adfun(a_x, a_yf) # compute the derivative of g w.r.t x at x equals .5 dg = g.jacobian(x) # check the result is as expected t_jp = 1 # t^0 for j in range(n-1) : t_jp = t_jp * dt # t^(j+1) at t = dt assert abs( a_yf[j] - x[0]*t_jp ) < 1e-10 # check yf[j] = x*t^(j+1_ assert abs( dg[j,0] - t_jp ) < 1e-10 # check dg[j] = t^(j+1) 
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
8.3: runge_kutta_4 With C++ Speed: Example and Test

Purpose
Discussion
Source Code

8.3.a: Purpose
In this example we demonstrate how a Python function can be recorded in a pycppad function object and then evaluated at much higher speeds than the Python evaluation.

8.3.b: Discussion
Define $y : \B{R}^2 \times \B{R} \rightarrow \B{R}^n$ by $$\begin{array}{rcl} y(x, 0) & = & x_0 \\ \partial_t y(x, t) & = & x_1 y(x, t) \end{array}$$ It follows that $$y(x, t) = x_0 \exp ( x_1 t )$$ Suppose we want to compute values for the function $g : \B{R}^2 \rightarrow \B{R}$ defined by $$g(x) = y(x, 1)$$ In this example we compare the execution time for doing this in pure Python with using a pycppad function object to compute $g(x)$ in C++.

8.3.c: Source Code  from pycppad import * import time def pycppad_test_runge_kutta_4_cpp() : x_1 = 0; # use this variable to switch x_1 between float and ad(float) def fun(t , y) : f = x_1 * y return f # Number of Runge-Kutta times steps to include in the function object M = 100 # Start time for recording the pycppad function object s0 = time.time() # Declare three independent variables. The operation sequence does not # depend on x, so we could use any value here. x = numpy.array( [.1, .1, .1] ) a_x = independent( numpy.array( x ) ) # First independent variables, x[0], is the value of y(0) a_y = numpy.array( [ a_x[0] ] ) # Make x_1 a variable so can use rk4 with various coefficients. x_1 = a_x[1] # Make dt a variable so can use rk4 with various step sizes. dt = a_x[2] # f(t, y) does not depend on t, so no need to make t a variable. t = ad(0.) # Record the operations for 10 time step for k in range(M) : a_y = runge_kutta_4(fun, t, a_y, dt) t = t + dt # define the AD function rk4 : x -> y rk4 = adfun(a_x, a_y) # amount of time it took to tape this function object tape_sec = time.time() - s0 # make the fucntion object more efficient s0 = time.time() rk4.optimize() opt_sec = time.time() - s0 ti = 0. # initial time tf = 1. # final time N = M * 100 # number of time steps dt = (tf - ti) / N # size of time step x_0 = 2. # use this for initial value of y(t) x_1 = .5 # use this for coefficient in ODE # python version of integrator with float values s0 = time.time() t = ti y = numpy.array( [ x_0 ] ); for k in range(N) : y = runge_kutta_4(fun, t, y, dt) t = t + dt # number of seconds to solve the ODE using python float python_sec = time.time() - s0 # check solution is correct assert( abs( y[0] - x_0 * exp( x_1 * tf ) ) < 1e-10 ) # pycppad function object version of integrator s0 = time.time() t = ti x = numpy.array( [ x_0 , x_1 , dt ] ) for k in range(N/M) : y = rk4.forward(0, x); x[0] = y[0]; # number of seconds to solve the ODE using pycppad function object cpp_sec = time.time() - s0 # check solution is correct assert( abs( y[0] - x_0 * exp( x_1 * tf ) ) < 1e-10 ) # Uncomment the print statement below to see actual times on your machine format = 'cpp_sec = %8f,\n' format = format + 'python_sec/cpp_sec = %5.1f\n' format = format + 'tape_sec/cpp_sec = %5.1f\n' format = format + 'opt_sec/cpp_sec = %5.1f' s = cpp_sec # print format % (s, python_sec/s, tape_sec/s, opt_sec/s ) # check that C++ is always more than 30 times faster (on all systems) assert( 30. * cpp_sec <= python_sec ) 
Input File: example/runge_kutta_4_cpp.py
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
9: Extensions, Bug Fixes, and Changes

2014-07-10
2014-05-29
Long Ago

9.a: 2014-07-10
Suppress warnings about using deprecated features of the numpy C-API; see numpy c-api deprecation (http://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html)

9.b: 2014-05-29
Add support for the 5.4.2: future divide operator .

9.c: Long Ago
9.1: 2010 , 9.2: 2010 , 9.3: 2010 , 9.4: 2008
Input File: omh/whats_new.omh
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
9.1: Extensions, Bug Fixes, and Changes During 2012

10-19
10-01
04-17
04-16

9.1.a: 10-19
The package pycppad.cppad_ was missing form the installation (after testing). This is due to be a bug in the python distutils (http://docs.python.org/library/distutils.html) package when the --inplace option is specified. The 2.h: install instructions have been changed to avoid this problem.

9.1.b: 10-01
Change the 8.3: runge_kutta_4_cpp.py test from asserting that C++ is 100 times faster than Python to 75 times faster.

9.1.c: 04-17
1. Add missing --inplace flag in 2.f: build instructions.
2. Add with_debugging option to 2.g: testing so that test_more.py does not fail when you build without debugging.

9.1.d: 04-16
1. Simplify the setup.py script used during the 2: install process. This included having the user to download and install CppAD, instead of having setup.py do this task.
2. Document the change to the derivative of the 5.8: abs that occurred in CppAD on 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) .

Input File: omh/whats_new_12.omh
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$
9.2: Extensions, Bug Fixes, and Changes During 2011

10-18
10-17
10-16
03-23
03-20

9.2.a: 10-18
1. Fix problem with import cppad_ in 20111017 version.
2. Change 8.3: runge_kutta_4_cpp.py to use 6.8: optimize and to check that using pycppad is always more than 100 times faster than straight python for this test case.

9.2.b: 10-17
Add tape optimization; see 6.8: optimize .

9.2.c: 10-16
1. Add the conditional expressions; see 5.9: condexp .
2. Use newer version of CppAD that fixes warnings found by g++ version 4.6.1.

9.2.d: 03-23
The --undef option is not available to ./setup.py build so ./setup.py build was changed to ./setup.py build_ext in the 2.f: build instructions .

9.2.e: 03-20
1. There was a problem with the pycppad 2: install instructions. To be specific, the debug build was not being installed because of the presence of --inplace in the example setup.py command. This has been fixed in the new 2.f.a: debug build instructions .
2. Change from using cppad-20100101.5 to using cppad-20110101.2.gpl.tgz; see CppAD whats_new (http://www.coin-or.org/CppAD/Doc/whats_new.htm) .
3. Use external sub directory of current working directory, instead of $HOME/install, for default directory that holds a local copy of CppAD; see cppad_parent_dir under 2.e: Required Setup Information . 4. Add boost_python_include_dir to list of 2.e: Required Setup Information . Input File: omh/whats_new_11.omh $\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$ 9.3: Extensions, Bug Fixes, and Changes During 2010 12-12 12-05 11-30 03-18 02-10 01-31 01-30 01-29 01-28 9.3.a: 12-12 The 6.1: independent(x) and 6.2: adfun(x,y) functions require the elements of x and y to all have the same type. If this was not the case, pycppad might crash with out a useful error message. This has been fixed. 9.3.b: 12-05 First version that works (and installs properly) under MS windows using the cygwin (www.cygwin.com) system; see special pycppad instructions for installing under 2.f.d: cygwin . 9.3.c: 11-30 Upgraded from using cppad-20100101.2 to cppad-20100101.5. This fixes a problem when installing pycppad with version 1.44.0 of boost-python (http://www.boost.org/doc/libs/1_44_0/libs/python/doc/index.html) . 9.3.d: 03-18 The upstream tarball for CppAD was missing from The 2: install process was failing because the upstream tarball was missing from http://www.coin-or.org/download/source/CppAD/ . This has been fixed, and a newer upstream release has been used. 9.3.e: 02-10 1. Added --inplace to the 2.f.b: optimized build instructions. Improved the 2.h: install instructions and the discussion of the 2.i: python path . 2. Changes exit to sys.exit in 2.g: test_example.py and test_more.py (required on some systems). 3. Fixed the 5.3.1: ad_unary.py example so it displays the correct source code and changed all to numpy.all in that example (required on some systems). 9.3.f: 01-31 Add the 6.3: abort_recording function. 9.3.g: 01-30 Change setup.py do that it patches the CppAD distribution (this fixes a problem with the 2.f.b: optimized build). 9.3.h: 01-29 Added the python ODE solver 8: runge_kutta_4 which can be used with smart' floating point types (e.g., AD types). 9.3.i: 01-28 Upgraded from using cppad-20090909.0 to cppad-20100101.0. Input File: omh/whats_new_10.omh $\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$ 9.4: Extensions, Bug Fixes, and Changes During 2009 06-21 06-13 05-13 05-12 02-13 02-09 01-31 01-30 01-29 01-28 01-26 9.4.a: 06-21 The Boost Python library name was not being used correctly and hence one would get the error message: "Cannot find the Boost Python library" ... This has been fixed. 9.4.b: 06-13 The setup.py file (used during the 2.f: build step of the install) had /home/bradbell/install as the value of cppad_parent_dir. This has been changed so that the default is $HOME/install.

9.4.c: 05-13
Changed to a new version of CppAD, (20090303.0).

9.4.d: 05-12
The Boost Python library libboost_python-py26.so had a problem with ad(x) where x was of type int. This has been fixed.

9.4.e: 02-13
Changed to new version of CppAD, (20090131.0) so that tanh is now included in 5.7: std_math (it was documented but missing from the actual pycppad implementation before this date).

9.4.f: 02-09
Added cross reference links from the examples to the corresponding section. For example, when independent appears in example code, it is linked in the following way: 6.1: independent .

9.4.g: 01-31
2. Remove tanh from standard math functions because previous release of CppAD did not include it (a new release of CppAD is being built so that tanh can be included in pycppad).
3. The two tests in example/ad_unary.py had the same name and hence only one was being run. This has been fixed.

9.4.h: 01-30
1. Organize the documentation by grouping the 5: ad_variable and 6: ad_function operations.
2. Add a 3: get_started.py section.
3. Add the 5.3: unary plus and minus operators .
4. Sort the installation 2.g: testing results in alphabetical order.
5. Use the ** exponentiation operator in 3: get_started.py .
6. Put this whats new section in inverse order by date.
7. Improve the error reporting done by the 5.2: value function.

9.4.i: 01-29
A list of 4: examples has been added to the documentation. The code for the 7: two_levels.py example was missing, this has been fixed.  The pycppad tests no longer require any package to run; i.e., it is no longer necessary to install --       py.test
-
(- http://codespeak.net/py/dist/) for testing pycppad.

9.4.j: 01-28
Change from an general AD package to just one package; i.e. from pyad-yyyymmdd.tar.gz to pycppad-yyyymmdd.tar.gz . This is a change in API requiring user code to change import pyad.cppad to import pycppad.

9.4.k: 01-26
Released first version, pyad-20090126.  Add build inplace, and omitting the setup.py install step, to 2: install documentation.
Input File: omh/whats_new_09.omh
$\newcommand{\B}[1]{{\bf #1}} \newcommand{\R}[1]{{\rm #1}}$

-------------------------------------------------------------------------------
Authors: Sebastian F. Walter and Bradley M. Bell.

as it was on 2009-01-24 with the following substutions:
<YEAR>         = 2008-2009
<OWNER>        = Bradley M. Bell and Sebastian F. Walter
<ORGANIZATION> = contributors' organizations
In addition,   "Neither the name of the contributors' organizations"
was changed to "Neither the names of the contributors' organizations"
-------------------------------------------------------------------------------

Copyright (c) 2008-2009, Bradley M. Bell and Seastian F. Walter

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the contributors' organizations nor the names of
its contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
`

11: Alphabetic Listing of Cross Reference Tags

A
C
E
F
G
H
I
J
L
O
P
R
S
T
V
W

12: Keyword Index

A
B
C
D
E
F
G
H
I
J
L
M
N
O
P
R
S
T
U
V
Z
!= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
* 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
** 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
*= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
+
binary 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
unary 5.3.1: Unary Plus and Minus Operators: Example and Test
unary 5.3: Unary Plus and Minus Operators
+= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
-
binary 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
unary 5.3.1: Unary Plus and Minus Operators: Example and Test
unary 5.3: Unary Plus and Minus Operators
-= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
/ 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
future example 5.4.2: Future Division Operator: Example and Test
/= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
2009
feature 9.4: Extensions, Bug Fixes, and Changes During 2009
< 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
<= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
== 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
> 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
>= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
A
abort recording 6.3: Abort a Recording of AD Operations
binary numeric operator 5.4: Binary Numeric Operators With an AD Result
decrease level 5.2: Create an Object With One Lower Level of AD
increase level 5.1: Create an Object With One Higher Level of AD
python : pycppad-20140710: A Python Algorithm Derivative Package
two levels 7: Using Two Levels of AD: Example and Test
unary operator 5.3: Unary Plus and Minus Operators
a2float 7: Using Two Levels of AD: Example and Test
create 5.1.d: Create an Object With One Higher Level of AD: a_x
a_float
comparison operator 5.6.1: a_float Comparison Operators: Example and Test
create 5.1.d: Create an Object With One Higher Level of AD: a_x
abort
AD recording 6.3: Abort a Recording of AD Operations
abort_recording
example 6.3.1: abort_recording: Example and Test
abs 5.9: Conditional Expressions
5.8: Absolute Value Functions
example 5.8.1: abs: Example and Test
ad 5.1: Create an Object With One Higher Level of AD
example 5.1.1: ad: Example and Test
variable 5: AD Variable Methods
example 6.2.1: adfun: Example and Test
algorithmic differentiation
python : pycppad-20140710: A Python Algorithm Derivative Package
arccos 5.7: Standard Math Unary Functions
arcsin 5.7: Standard Math Unary Functions
arctan 5.7: Standard Math Unary Functions
assignment
assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
computed operator 5.5: Computed Assignment Operators
automatic differentiation
python : pycppad-20140710: A Python Algorithm Derivative Package
B
binary
AD numeric operator 5.4: Binary Numeric Operators With an AD Result
bool a_float operator example 5.6.1: a_float Comparison Operators: Example and Test
bool operator 5.6: Binary Comparison Operators
comparison operator 5.6: Binary Comparison Operators
numeric operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
bool
a_float comparison operator example 5.6.1: a_float Comparison Operators: Example and Test
binary operator 5.6: Binary Comparison Operators
bug
fix 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
python distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
build
debugging 2.f.a: Installing pycppad: Building.With Debugging
optimize 2.f.b: Installing pycppad: Building.Optimized
C
C++ speed
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
comparison
a_float operator 5.6.1: a_float Comparison Operators: Example and Test
binary operator 5.6: Binary Comparison Operators
computed
assignment operator 5.5: Computed Assignment Operators
assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
condexp
example 5.9.1: condexp: Example and Test
conditional
expression 5.9.1: condexp: Example and Test
cos 5.7: Standard Math Unary Functions
cosh 5.7: Standard Math Unary Functions
create
a2float 5.1.d: Create an Object With One Higher Level of AD: a_x
a_float 5.1.d: Create an Object With One Higher Level of AD: a_x
D
debugging
build 2.f.a: Installing pycppad: Building.With Debugging
dependent
variables 6.2: Create an AD Function Object
derivative
domain direction 6.4: Forward Mode: Derivative in One Domain Direction
entire driver 6.6: Driver for Computing Entire Derivative
entire example 6.6.1: Entire Derivative: Example and Test
ODE solution 8.2: runge_kutta_4 An AD Example and Test
range direction 6.5: Reverse Mode: Derivative in One Range Direction
direction
domain derivative 6.4: Forward Mode: Derivative in One Domain Direction
range derivative 6.5: Reverse Mode: Derivative in One Range Direction
distutils
python bug 2.f.c: Installing pycppad: Building.Bug in Python Distutils
divide 5.4: Binary Numeric Operators With an AD Result
domain
direction derivative 6.4: Forward Mode: Derivative in One Domain Direction
driver
entire derivative 6.6: Driver for Computing Entire Derivative
hessian 6.7: Driver for Computing Hessian in a Range Direction
E
entire
derivative driver 6.6: Driver for Computing Entire Derivative
derivative example 6.6.1: Entire Derivative: Example and Test
example
!= 5.6.1: a_float Comparison Operators: Example and Test
* 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
** 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
*= 5.5.1: Computed Assignment Operators: Example and Test
+ 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
+ 5.3.1: Unary Plus and Minus Operators: Example and Test
+= 5.5.1: Computed Assignment Operators: Example and Test
- 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
- 5.3.1: Unary Plus and Minus Operators: Example and Test
-= 5.5.1: Computed Assignment Operators: Example and Test
/ 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
/= 5.5.1: Computed Assignment Operators: Example and Test
< 5.6.1: a_float Comparison Operators: Example and Test
<= 5.6.1: a_float Comparison Operators: Example and Test
== 5.6.1: a_float Comparison Operators: Example and Test
> 5.6.1: a_float Comparison Operators: Example and Test
>= 5.6.1: a_float Comparison Operators: Example and Test
abort_recording 6.3.1: abort_recording: Example and Test
abs 5.8.1: abs: Example and Test
bool a_float binary operator 5.6.1: a_float Comparison Operators: Example and Test
computed assignment operator 5.5.1: Computed Assignment Operators: Example and Test
condexp 5.9.1: condexp: Example and Test
entire derivative 6.6.1: Entire Derivative: Example and Test
future / 5.4.2: Future Division Operator: Example and Test
get_started 3: get_started: Example and Test
hessian 6.7.1: Hessian Driver: Example and Test
independent 6.1.1: independent: Example and Test
list 4: List of All the pycppad Examples
numeric binary operator 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
optimize 6.8.1: Optimize Function Object: Example and Test
order one forward 6.4.2: Forward Order One: Example and Test
order one reverse 6.5.1: Reverse Order One: Example and Test
order two reverse 6.5.2: Reverse Order Two: Example and Test
order zero forward 6.4.1: Forward Order Zero: Example and Test
runge_kutta_8.3: runge_kutta_4 With C++ Speed: Example and Test
runge_kutta_8.2: runge_kutta_4 An AD Example and Test
runge_kutta_8.1: runge_kutta_4 A Correctness Example and Test
standard math 5.7.1: Standard Math Unary Functions: Example and Test
value 5.2.1: value: Example and Test
exp 5.7: Standard Math Unary Functions
exponentiation 5.4: Binary Numeric Operators With an AD Result
extract
F
feature
version 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
fix
bug 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
forward 6.4: Forward Mode: Derivative in One Domain Direction
order one example 6.4.2: Forward Order One: Example and Test
order zero example 6.4.1: Forward Order Zero: Example and Test
future /
example 5.4.2: Future Division Operator: Example and Test
G
get_started
example 3: get_started: Example and Test
H
hessian
driver 6.7: Driver for Computing Hessian in a Range Direction
example 6.7.1: Hessian Driver: Example and Test
Lagrangian 6.7: Driver for Computing Hessian in a Range Direction
I
independent
abort recording 6.3: Abort a Recording of AD Operations
example 6.1.1: independent: Example and Test
variables 6.1.a: Create an Independent Variable Vector: Syntax
install
J
jacobian 6.6: Driver for Computing Entire Derivative
example 6.6.1: Entire Derivative: Example and Test
L
Lagrangian
hessian 6.7: Driver for Computing Hessian in a Range Direction
level
decrease AD 5.2: Create an Object With One Lower Level of AD
increase AD 5.1: Create an Object With One Higher Level of AD
levels
two AD 7: Using Two Levels of AD: Example and Test
list
example 4: List of All the pycppad Examples
log 5.7: Standard Math Unary Functions
log10 5.7: Standard Math Unary Functions
M
math
example standard 5.7.1: Standard Math Unary Functions: Example and Test
memory
optimize 6.8: Optimize an AD Function Object Tape
minus
binary 5.4: Binary Numeric Operators With an AD Result
unary 5.3: Unary Plus and Minus Operators
N
numeric
AD binary operator 5.4: Binary Numeric Operators With an AD Result
binary operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
O
ODE solution
correct 8.1: runge_kutta_4 A Correctness Example and Test
derivative 8.2: runge_kutta_4 An AD Example and Test
ODE solver
Runge-Kutta 8: Fourth Order Runge Kutta
one
order forward example 6.4.2: Forward Order One: Example and Test
order reverse example 6.5.1: Reverse Order One: Example and Test
operation
optimize sequence 6.8.1: Optimize Function Object: Example and Test
operations
optimize sequence 6.8: Optimize an AD Function Object Tape
operator
AD numeric binary 5.4: Binary Numeric Operators With an AD Result
AD unary 5.3: Unary Plus and Minus Operators
bool a_float binary example 5.6.1: a_float Comparison Operators: Example and Test
bool binary 5.6: Binary Comparison Operators
computed assignment 5.5: Computed Assignment Operators
computed assignment example 5.5.1: Computed Assignment Operators: Example and Test
numeric binary example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
optimize 6.8: Optimize an AD Function Object Tape
build 2.f.b: Installing pycppad: Building.Optimized
example 6.8.1: Optimize Function Object: Example and Test
order
forward one example 6.4.2: Forward Order One: Example and Test
forward zero example 6.4.1: Forward Order Zero: Example and Test
reverse one example 6.5.1: Reverse Order One: Example and Test
reverse two example 6.5.2: Reverse Order Two: Example and Test
P
path
python 2.i: Installing pycppad: Python Path
plus
binary 5.4: Binary Numeric Operators With an AD Result
unary 5.3: Unary Plus and Minus Operators
pycppad : pycppad-20140710: A Python Algorithm Derivative Package
build 2.f: Installing pycppad: Building
extract 2.d: Installing pycppad: Unpacking
install 2.h: Installing pycppad: Installing
install 2: Installing pycppad
test 2.g: Installing pycppad: Testing
unpack 2.d: Installing pycppad: Unpacking
python
AD : pycppad-20140710: A Python Algorithm Derivative Package
bug distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
path 2.i: Installing pycppad: Python Path
R
Runge-Kutta
ODE solver 8: Fourth Order Runge Kutta
range
direction derivative 6.5: Reverse Mode: Derivative in One Range Direction
recording
abort AD 6.3: Abort a Recording of AD Operations
start 6.1.a: Create an Independent Variable Vector: Syntax
stop 6.2: Create an AD Function Object
reverse 6.5: Reverse Mode: Derivative in One Range Direction
order one example 6.5.1: Reverse Order One: Example and Test
order two example 6.5.2: Reverse Order Two: Example and Test
runge_kutta_4
correct 8.1: runge_kutta_4 A Correctness Example and Test
evaluate solution 8.3: runge_kutta_4 With C++ Speed: Example and Test
example 8.2: runge_kutta_4 An AD Example and Test
S
sequence
optimize operation 6.8.1: Optimize Function Object: Example and Test
optimize operations 6.8: Optimize an AD Function Object Tape
sin 5.7: Standard Math Unary Functions
sinh 5.7: Standard Math Unary Functions
speed
optimize 6.8: Optimize an AD Function Object Tape
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
sqrt 5.7: Standard Math Unary Functions
standard
example math 5.7.1: Standard Math Unary Functions: Example and Test
start
recording 6.1.a: Create an Independent Variable Vector: Syntax
stop
recording 6.2: Create an AD Function Object
T
tan 5.7: Standard Math Unary Functions
tanh 5.7: Standard Math Unary Functions
tape
optimize 6.8: Optimize an AD Function Object Tape
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
test
times 5.4: Binary Numeric Operators With an AD Result
two
AD levels 7: Using Two Levels of AD: Example and Test
order reverse example 6.5.2: Reverse Order Two: Example and Test
U
unary
+ 5.3.1: Unary Plus and Minus Operators: Example and Test
- 5.3.1: Unary Plus and Minus Operators: Example and Test
AD operator 5.3: Unary Plus and Minus Operators
unpack
V
value 5.2: Create an Object With One Lower Level of AD
example 5.2.1: value: Example and Test
variable