pycppad-20140710: A Python Algorithm Derivative Package

Syntax
Purpose
Simple Example
Features
Contents
 Manual split into sections `  ` Manual as one web page Math displayed using Latex `  ` pycppad.htm `  ` _printable.htm Math displayed using MathML `  ` pycppad.xml `  ` _printable.xml

10: BSD

b: Syntax
`from pycppad import *`

c: Purpose

d: Simple Example ``` from pycppad import * def pycppad_test_get_started() : def F(x) : # function to be differentiated return exp(-(x**2. + x**2.) / 2.) # is Gaussian density x = numpy.array( [ 1., 2.] ) a_x = independent(x) a_y = numpy.array( [ F(a_x) ] ) f = adfun(a_x, a_y) J = f.jacobian(x) # J = F'(x) assert abs( J[0, 0] + F(x) * x ) < 1e-10 # J[0,0] ~= - F(x) * x assert abs( J[0, 1] + F(x) * x ) < 1e-10 # J[0,1] ~= - F(x) * x ```

e: Features
Operator overloading is used to store the operation sequence corresponding to a python algorithm. The operation sequence can be evaluated to obtain new function values or derivatives of arbitrary order. In addition, multiple levels of AD are supported. This means that AD derivatives can be used in the definition of a function which in turn can be differentiated using AD. See 9: whats_new for a list of recent extensions and bug fixes.

f: Contents
 _contents: 1 Table of Contents install: 2 Installing pycppad get_started.py: 3 get_started: Example and Test example: 4 List of All the pycppad Examples ad_variable: 5 AD Variable Methods ad_function: 6 AD Function Methods two_levels.py: 7 Using Two Levels of AD: Example and Test runge_kutta_4: 8 Fourth Order Runge Kutta whats_new: 9 Extensions, Bug Fixes, and Changes license: 10 License _reference: 11 Alphabetic Listing of Cross Reference Tags _index: 12 Keyword Index _external: 13 External Internet References

Input File: doc.omh

```pycppad-20140710: A Python Algorithm Derivative Package: : pycppad
get_started: Example and Test: 3: get_started.py
List of All the pycppad Examples: 4: example
Create an Object With One Lower Level of AD: 5.2: value
value: Example and Test: 5.2.1: value.py
Unary Plus and Minus Operators: 5.3: ad_unary
Unary Plus and Minus Operators: Example and Test: 5.3.1: ad_unary.py
Binary Numeric Operators With an AD Result: Example and Test: 5.4.1: ad_numeric.py
Future Division Operator: Example and Test: 5.4.2: future_div_op.py
Computed Assignment Operators: 5.5: assign_op
Computed Assignment Operators: Example and Test: 5.5.1: assign_op.py
Binary Comparison Operators: 5.6: compare_op
a_float Comparison Operators: Example and Test: 5.6.1: compare_op.py
Standard Math Unary Functions: 5.7: std_math
Standard Math Unary Functions: Example and Test: 5.7.1: std_math.py
Absolute Value Functions: 5.8: abs
abs: Example and Test: 5.8.1: abs.py
Conditional Expressions: 5.9: condexp
condexp: Example and Test: 5.9.1: condexp.py
Create an Independent Variable Vector: 6.1: independent
independent: Example and Test: 6.1.1: independent.py
Abort a Recording of AD Operations: 6.3: abort_recording
abort_recording: Example and Test: 6.3.1: abort_recording.py
Forward Mode: Derivative in One Domain Direction: 6.4: forward
Forward Order Zero: Example and Test: 6.4.1: forward_0.py
Forward Order One: Example and Test: 6.4.2: forward_1.py
Reverse Mode: Derivative in One Range Direction: 6.5: reverse
Reverse Order One: Example and Test: 6.5.1: reverse_1.py
Reverse Order Two: Example and Test: 6.5.2: reverse_2.py
Driver for Computing Entire Derivative: 6.6: jacobian
Entire Derivative: Example and Test: 6.6.1: jacobian.py
Driver for Computing Hessian in a Range Direction: 6.7: hessian
Hessian Driver: Example and Test: 6.7.1: hessian.py
Optimize an AD Function Object Tape: 6.8: optimize
Optimize Function Object: Example and Test: 6.8.1: optimize.py
Using Two Levels of AD: Example and Test: 7: two_levels.py
Fourth Order Runge Kutta: 8: runge_kutta_4
runge_kutta_4 A Correctness Example and Test: 8.1: runge_kutta_4_correct.py
runge_kutta_4 With C++ Speed: Example and Test: 8.3: runge_kutta_4_cpp.py
Extensions, Bug Fixes, and Changes: 9: whats_new
Extensions, Bug Fixes, and Changes During 2012: 9.1: whats_new_12
Extensions, Bug Fixes, and Changes During 2011: 9.2: whats_new_11
Extensions, Bug Fixes, and Changes During 2010: 9.3: whats_new_10
Extensions, Bug Fixes, and Changes During 2009: 9.4: whats_new_09
Alphabetic Listing of Cross Reference Tags: 11: _reference
Keyword Index: 12: _index
External Internet References: 13: _external
```

Other Instructions
Requires
Unpacking
Required Setup Information
Building
With Debugging
Optimized
Bug in Python Distutils
Cygwin
Testing
Error
Installing
Uninstalling
Python Path

2.a: Other Instructions

2.b: Requires
This package requires the following other packages:
1. The Boost Python (development version) must be installed.
2. The python numpy library must be installed.

2.d: Unpacking
On unix, you could use the command ``` tar -xvzf pycppad-20140710.tar.gz ``` which would create the directory `pycppad-20140710`.

2.e: Required Setup Information
The value of the following setup variables, in the file ``` pycppad-20140710/setup.py ``` must be set to agree with your system: ``` # Directory where CppAD include files are located # cppad_include_dir = [ '/usr/include' ] cppad_include_dir = [ os.environ['HOME'] + '/prefix/cppad/include' ] # # Directory where Boost Python include files are located boost_python_include_dir = [ '/usr/include' ] # # Director whre Boost Python library is located # boost_python_lib_dir = [ '/usr/lib' ] boost_python_lib_dir = [ '/usr/lib64' ] # # Name of the Boost Python library in boost_python_lib_dir. # boost_python_lib = [ 'boost_python-mt' ] boost_python_lib = [ 'boost_python' ] ``` Note that Boost Python and CppAD must be installed before you can properly set this information.

2.f: Building

2.f.a: With Debugging
Change into the directory `pycppad-20140710` and execute the command ``` ./setup.py build_ext --inplace --debug --undef NDEBUG ``` to compile and link a version of the CppAD extension module with debugging (improved error messaging).

2.f.b: Optimized
You can compile and link an optimized version of the CppAD extension module with the command ``` ./setup.py build_ext --inplace ``` Note that in the optimized version, certain error checking is not done and improper use of `pycppad.cpp` may lead to a segmentation fault.

2.f.c: Bug in Python Distutils
When using the GNU C++ compiler `gcc`, the warning ``` cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++ ``` will be printed. This is not requested by `setup.py` but is rather a bug in the Python distutils (http://docs.python.org/distutils/) package.

2.f.d: Cygwin
If python crashes during the build procedure under cygwin, you may need to run the `rebaseall` program. For example, execute the following steps:
1. Check which cygwin processes are running using the `ps -e` command, and shut them all down.
2. Open an DOS command window, for example, select `Start | Run` and then enter `cmd` in as the program to run.
3. Change into the cygwin `bin` directory. If you installed cygwin in the default location, the following command will do this: ``` cd c:\cygwin\bin ```
4. Start the `ash` shell by executing the command ``` ash.exe ```
5. The execute the command ``` /usr/bin/rebaseall ``` This will take a few minutes to execute. When it is done, you can close the command window by executing the command `exit` twice.

2.g: Testing
You can test of all the 4: examples in the `pycppad` documentation. Change into the directory `pycppad-20140710` and execute the command ``` python test_example.py ``` You can run some more tests with the command ```      python test_more.py with_debugging ``` where `with_debugging` is `True` or `False` depending on if you built 2.f.a: with debugging .

2.g.a: Error
If you get a message about `cppad_` being missing, it is probably because you did not use the `--inplace` flag when you 2.f: built `pycppad`.

2.h: Installing
Installing `pycppad` copies it to the standard location for you system. You may or may not preform this step: ``` ```Change into the directory `pycppad-20140710` and execute the command ```      ./setup.py build_ext --debug --undef NDEBUG install --prefix=prefix ``` or ```      ./setup.py build_ext install --prefix=prefix ``` where `prefix` is the prefix for the location where you wish to install `pycppad`. Note that some common choices for `prefix` are
 `\$HOME` if you are not the system administrator for this machine `/usr/local` you're administrator and the system has a package manager (like `yum`, `apt-get`). `/usr` you're the administrator but the system does not have a package manager.

2.h.a: Uninstalling
Unfortunately, python's `distutils` package does not provide an uninstall command. You can uninstall the `pycppad` package by removing the entries ```      prefix/lib/pythonmajor.minor/site-packages/pycppad      prefix/lib/pythonmajor.minor/site-packages/pycppad-20140710.egg-info      prefix/share/doc/pycppad ``` where `major` and `minor` are the major and minor version numbers printed by the command ``` python --version ```
2.i: Python Path
If you have not installed `pycppad`, you will not be able to use the python command ```      import pycppad ``` unless the distribution directory ``` pycppad-20140710 ``` is in your python path. If you have installed `pycppad`, the installation directory ```      prefix/lib/pythonmajor.minor/site-packages ``` must be in you python path (to use the `import pycppad` command). You can check your python path with the following commands ``` python import sys sys.path ``` If the required directory is not yet there, you could add the directory above to your python path using the command ```      sys.path.append('prefix/lib/pythonmajor.minor/site-packages') ``` You can avoid having to do this every time you load python by adding the path to your environment variable `PYTHONPATH`. For example, if you are using the `bash` shell, you could add the command ```      export PYTHONPATH="prefix/lib/pythonmajor.minor/site-packages" ``` to your `\$HOME/.bashrc` file.

The documentation for `pycppad` starts out in the directory ``` pycppad-20140710/doc ``` During the installation process, it is copied to the directory ```      prefix/share/doc/pycppad ```
Input File: omh/install.omh
3: get_started: Example and Test

For our getting started example, we consider the Gaussian density for two independent random variables $\mathit{F}:{\mathbf{R}}^{2}\to \mathbf{R}$ and its partial derivatives : $\begin{array}{ccc}\hfill \mathit{F}\left(\mathit{x}\right)& \hfill =\hfill & \mathrm{exp}\left[-\left({\mathit{x}}_{0}^{2}+{\mathit{x}}_{1}^{2}\right)/2.\right]\hfill \\ \hfill {\partial }_{\mathit{x}\left(0\right)}\mathit{F}\left(\mathit{x}\right)& \hfill =\hfill & -\mathit{F}\left(\mathit{x}\right)*{\mathit{x}}_{0}\hfill \\ \hfill {\partial }_{\mathit{x}\left(1\right)}\mathit{F}\left(\mathit{x}\right)& \hfill =\hfill & -\mathit{F}\left(\mathit{x}\right)*{\mathit{x}}_{1}\hfill \end{array}$ The following Python code computes these derivatives using `pycppad` and then checks the results for correctness: ``` from pycppad import * def pycppad_test_get_started() : def F(x) : # function to be differentiated return exp(-(x**2. + x**2.) / 2.) # is Gaussian density x = numpy.array( [ 1., 2.] ) a_x = independent(x) a_y = numpy.array( [ F(a_x) ] ) f = adfun(a_x, a_y) J = f.jacobian(x) # J = F'(x) assert abs( J[0, 0] + F(x) * x ) < 1e-10 # J[0,0] ~= - F(x) * x assert abs( J[0, 1] + F(x) * x ) < 1e-10 # J[0,1] ~= - F(x) * x ```
Input File: example/get_started.py
4: List of All the pycppad Examples

 6.3.1: abort_recording.py abort_recording: Example and Test 5.8.1: abs.py abs: Example and Test 5.1.1: ad.py ad: Example and Test 6.2.1: adfun.py adfun: Example and Test 5.4.1: ad_numeric.py Binary Numeric Operators With an AD Result: Example and Test 5.3.1: ad_unary.py Unary Plus and Minus Operators: Example and Test 5.5.1: assign_op.py Computed Assignment Operators: Example and Test 5.9.1: condexp.py condexp: Example and Test 5.6.1: compare_op.py a_float Comparison Operators: Example and Test 5.4.2: future_div_op.py Future Division Operator: Example and Test 6.4.1: forward_0.py Forward Order Zero: Example and Test 6.4.2: forward_1.py Forward Order One: Example and Test 3: get_started.py get_started: Example and Test 6.7.1: hessian.py Hessian Driver: Example and Test 6.1.1: independent.py independent: Example and Test 6.6.1: jacobian.py Entire Derivative: Example and Test 6.8.1: optimize.py Optimize Function Object: Example and Test 6.5.1: reverse_1.py Reverse Order One: Example and Test 6.5.2: reverse_2.py Reverse Order Two: Example and Test 8.2: runge_kutta_4_ad.py runge_kutta_4 An AD Example and Test 8.3: runge_kutta_4_cpp.py runge_kutta_4 With C++ Speed: Example and Test 8.1: runge_kutta_4_correct.py runge_kutta_4 A Correctness Example and Test 5.7.1: std_math.py Standard Math Unary Functions: Example and Test 7: two_levels.py Using Two Levels of AD: Example and Test 5.2.1: value.py value: Example and Test

Input File: omh/example.omh

Contents

5.a: Contents
 ad: 5.1 Create an Object With One Higher Level of AD value: 5.2 Create an Object With One Lower Level of AD ad_unary: 5.3 Unary Plus and Minus Operators ad_numeric: 5.4 Binary Numeric Operators With an AD Result assign_op: 5.5 Computed Assignment Operators compare_op: 5.6 Binary Comparison Operators std_math: 5.7 Standard Math Unary Functions abs: 5.8 Absolute Value Functions condexp: 5.9 Conditional Expressions

5.1: Create an Object With One Higher Level of AD

Syntax
Purpose
x
a_x
Example

5.1.a: Syntax
`a_x = ad(x)`

5.1.b: Purpose
Creates an AD object `a_x` that records floating point operations. An 6.2: adfun object can later use this recording to evaluate function values and derivatives. These later evaluations are done using the same type as `x` (except when `x` is an instance of `int`, the later evaluations are done using `float` operations).

5.1.c: x
The argument `x` can be an instance of an `int` (AD level 0), or an instance of `float` (AD level 0), or an `a_float` (AD level 1). The argument `x` may also be a `numpy.array` with one of the element types listed in the previous sentence.

5.1.d: a_x
If `x` is an instance of `int` or `float`, `a_x` is an `a_float` (AD level 1). If `x` is an `a_float`, `a_x` is an `a2float` (AD level 2). If `x` is an `numpy.array`, `a_x` is also an `numpy.array` with the same shape as `x` .

5.1.e: Example
The file 5.1.1: ad.py contains an example and test of this function.

``` from pycppad import * import numpy def pycppad_test_ad() : x = 1 a_x = ad(x) a2x = ad(a_x) # assert type(a_x) == a_float and a_x == x assert type(a2x) == a2float and a2x == x # x = numpy.array( [ 1 , 2 , 3 ] ) a_x = ad(x) a2x = ad(a_x) # for i in range( len(a_x) ) : assert type(a_x[i]) == a_float and a_x[i] == x[i] for i in range( len(a2x) ) : assert type(a2x[i]) == a2float and a2x[i] == x[i] ```
5.2: Create an Object With One Lower Level of AD

Syntax
Purpose
a_x
x
Example

5.2.a: Syntax
`x = value(a_x)`

5.2.b: Purpose
Returns an object with one lower level of AD recording.

5.2.c: a_x
The argument `a_x` must be an `a_float` (AD level 1), or an `a2float` (AD level 2). The argument `a_x` may also be a `numpy.array` with one of the element types listed in the previous sentence.

5.2.d: x
If `a_x` is an `a_float`, `x` is a `float` (AD level 0). If `a_x` is an `a2float`, `x` is an `a_float` (AD level 1). If `a_x` is an `numpy.array`, `x` is also an `numpy.array` with the same shape as `a_x` .

5.2.e: Example
The file 5.2.1: value.py contains an example and test of this function.
5.2.1: value: Example and Test

``` from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_value() : x = 2 a_x = ad(x) # assert type(value(a_x)) == float and value(a_x) == x # x = numpy.array( [ 1 , 2 , 3 ] ) a_x = ad(x) # for i in range( len(a_x) ) : xi = value(a_x[i]) assert type(xi) == float and xi == x[i] # Example using a2float ------------------------------------------------------ def pycppad_test_value_a2() : x = 2 a2x = ad(ad(x)) # assert type(value(a2x)) == a_float and value(a2x) == x # x = numpy.array( [ 1 , 2 , 3 ] ) a2x = ad(ad(x)) # for i in range( len(a2x) ) : a_xi = value(a2x[i]) assert type(a_xi) == a_float and a_xi == x[i] ```
Input File: example/value.py
5.3: Unary Plus and Minus Operators

Syntax
Purpose
Type
Arrays
Example

5.3.a: Syntax
```y = + x ``` ```y = - x ```
5.3.b: Purpose
The operator `+` ( `-` ) above results in `z` equal to `x` (minus `x` ).

5.3.c: Type
The argument `x` can be `a_float` or `a2float` and the result `z` will have the same type as `x` .

5.3.d: Arrays
The argument `x` may be a `numpy.array` with elements of type `a_float` or `a2float`. In this case, the result `z` is an array with the same shape and element type as `x` .

5.3.e: Example
The file 5.3.1: ad_unary.py contains an example and test of these functions.
5.3.1: Unary Plus and Minus Operators: Example and Test

``` from pycppad import * import numpy # Example using a_float ------------------------------------------------------ def pycppad_test_ad_unary() : x = ad(2.) plus_x = + x minus_x = - x # test using corresponding unary float operators assert value(plus_x) == + value(x) assert value(minus_x) == - value(x) # x = ad( numpy.array( [ 1. , 2. ] ) ) plus_x = + x minus_x = - x # test using corresponding unary float operators assert numpy.all( value(plus_x) == + value(x) ) assert numpy.all( value(minus_x) == - value(x) ) # Example using a2float ------------------------------------------------------ def pycppad_test_ad_unary_a2() : x = ad( ad(2.) ) plus_x = + x minus_x = - x # test using corresponding unary a_float operators assert value(plus_x) == + value(x) assert value(minus_x) == - value(x) # x = ad( ad( numpy.array( [ 1. , 2. ] ) ) ) plus_x = + x minus_x = - x # test using corresponding unary float operators assert numpy.all( value(plus_x) == + value(x) ) assert numpy.all( value(minus_x) == - value(x) ) ```
5.4: Binary Numeric Operators With an AD Result

Syntax
Purpose
op
Types
Arrays
Example

5.4.a: Syntax
`z = x op y`

5.4.b: Purpose
Sets `z` to the result of the binary operation defined by `op` and with `x` as the left operand and `y` as the right operand.

5.4.c: op
The possible values for `op` are
 `op` `  ` Meaning `+` addition `-` subtraction `*` multiplication `/` division `**` exponentiation

5.4.d: Types
The following table lists the possible types for `x` and `y` and the corresponding result type for `z` . ```                       y  x           float    a_float   a2float          -------------------------------  float   -   float    a_float   a2float a_float  -  a_float   a_float a2float  -  a2float             a2float ``` The type `float` does not need to be matched exactly but rather as an instance of `float`.

5.4.e: Arrays
Either `x` or `y` or both may be a `numpy.array` with elements that match one of possible type choices above. If both `x` and `y` are arrays, they must have the same shape. When either `x` or `y` is an array, the result `z` is an array with the same shape. The type of the elements of `z` correspond to the table above (when the result type is a `float`, this only refers to the element types matching as instances).

5.4.f: Example
The file 5.4.1: ad_numeric.py contains an example and test of these operations. The file 5.4.2: future_div_op.py contains an example and test of the future division operator (http://legacy.python.org/dev/peps/pep-0238/)
5.4.1: Binary Numeric Operators With an AD Result: Example and Test

``` from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_ad_numeric() : x = 2. y = 3. a_x = ad(x) a_y = ad(y) # assert a_x + a_y == x + y assert a_x + y == x + y assert x + a_y == x + y # assert a_x - a_y == x - y assert a_x - y == x - y assert x - a_y == x - y # assert a_x * a_y == x * y assert a_x * y == x * y assert x * a_y == x * y # assert a_x / a_y == x / y assert a_x / y == x / y assert x / a_y == x / y # assert a_x ** a_y == x ** y assert a_x ** y == x ** y assert x ** a_y == x ** y # # Example using a2float ----------------------------------------------------- def pycppad_test_ad_numeric_a2() : x = 2. y = 3. a2x = ad(ad(x)) a2y = ad(ad(y)) # assert a2x + a2y == x + y assert a2x + y == x + y assert x + a2y == x + y # assert a2x - a2y == x - y assert a2x - y == x - y assert x - a2y == x - y # assert a2x * a2y == x * y assert a2x * y == x * y assert x * a2y == x * y # assert a2x / a2y == x / y assert a2x / y == x / y assert x / a2y == x / y # assert a2x ** a2y == x ** y assert a2x ** y == x ** y assert x ** a2y == x ** y # ```
5.4.2: Future Division Operator: Example and Test

``` # Example using a_float ------------------------------------------------------ from __future__ import division from pycppad import * def pycppad_test_future_div_op() : x = 2. y = 3. # tmp1 = ad(x) tmp2 = ad(y) tmp3 = tmp1/tmp2 tmp4 = x/tmp2 tmp5 = tmp1/y assert tmp3 == x / y assert tmp4 == x / y assert tmp5 == x / y ```
Input File: example/future_div_op.py
5.5: Computed Assignment Operators

Syntax
Purpose
op
Types
Arrays
Example

5.5.a: Syntax
`u op= x`

5.5.b: Purpose
We use `y` ( `z` ) to refer to the value of `u` before (after) the operation. This operation sets `z` equal to ```      y op x ``` .

5.5.c: op
The possible values for `op` are
 `op` `  ` Meaning `+` addition `-` subtraction `*` multiplication `/` division

5.5.d: Types
The following table lists the possible types for `x` and `y` (the value of `u` before the operation) and the corresponding `z` (the value of `u` after the operation). ```                       y  x           float    a_float   a2float          -------------------------------  float   -   float    a_float   a2float a_float  -  a_float   a_float a2float  -  a2float             a2float ``` The type `float` does not need to be matched exactly but rather as an instance of `float`.

5.5.e: Arrays
Either `x` or `y` or both may be a `numpy.array` with elements that match one of possible type choices above. If both `x` and `y` are arrays, they must have the same shape. When either `x` or `y` is an array, the result `z` is an array with the same shape. The type of the elements of `z` correspond to the table above (when the result type is a `float`, this only refers to the element types matching as instances).

5.5.f: Example
The file 5.5.1: assign_op.py contains an example and test of these operations.
5.5.1: Computed Assignment Operators: Example and Test

``` # Example using a_float ------------------------------------------------------ from pycppad import * def pycppad_test_assign_op() : x = 2. y = 3. # tmp = ad(x) tmp += ad(y) assert tmp == x + y tmp = ad(x) tmp += y assert tmp == x + y # tmp = ad(x) tmp -= ad(y) assert tmp == x - y tmp = ad(x) tmp -= y assert tmp == x - y # tmp = ad(x) tmp *= ad(y) assert tmp == x * y tmp = ad(x) tmp *= y assert tmp == x * y # tmp = ad(x) tmp /= ad(y) assert tmp == x / y tmp = ad(x) tmp /= y assert tmp == x / y # Example using a2float ------------------------------------------------------ from pycppad import * def pycppad_test_assign_op_a2() : x = 2. y = 3. # tmp = ad(ad(x)) tmp += ad(ad(y)) assert tmp == x + y tmp = ad(ad(x)) tmp += y assert tmp == x + y # tmp = ad(ad(x)) tmp -= ad(ad(y)) assert tmp == x - y tmp = ad(ad(x)) tmp -= y assert tmp == x - y # tmp = ad(ad(x)) tmp *= ad(ad(y)) assert tmp == x * y tmp = ad(ad(x)) tmp *= y assert tmp == x * y # tmp = ad(ad(x)) tmp /= ad(ad(y)) assert tmp == x / y tmp = ad(ad(x)) tmp /= y assert tmp == x / y ```
Input File: example/assign_op.py
5.6: Binary Comparison Operators

Syntax
Purpose
op
Types
Arrays
Example

5.6.a: Syntax
`z = x op y`

5.6.b: Purpose
Sets `z` to the result of the binary operation defined by `op` and with `x` as the left operand and `y` as the right operand.

5.6.c: op
The possible values for `op` are
 `op` `  ` Meaning `>` greater than `>` less than `>=` greater than or equal `<=` less than or equal `==` equal `!=` not equal

5.6.d: Types
The following table lists the possible (yes) and impossible (no) types for `x` and `y` . The corresponding result type for `z` is always `bool`. ```                       y  x           float    a_float   a2float          -------------------------------  float   -    yes      yes       yes    a_float  -    yes      yes       no a2float  -    yes      no        yes  ``` The type `float` does not need to be matched exactly but rather as an instance of `float`.

5.6.e: Arrays
Either `x` or `y` or both may be a `numpy.array` with elements that match one of possible type choices above. If both `x` and `y` are arrays, they must have the same shape. When either `x` or `y` is an array, the result `z` is an array with the same shape. The type of the elements of `z` correspond to the table above (when the result type is a `float`, this only refers to the element types matching as instances).

5.6.f: Example
The file 5.6.1: compare_op.py contains an example and test of these operations.
5.6.1: a_float Comparison Operators: Example and Test

``` from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_compare_op(): x = ad(2.) y = ad(3.) z = ad(2.) # assert comparisons that should be true assert x == x assert x == z assert x != y assert x <= x assert x <= z assert x <= y assert x < y # assert comparisons that should be false assert not x == y assert not x != z assert not x != x assert not x >= y assert not x > y # Example using a2float ------------------------------------------------------ def pycppad_test_compare_op_a2(): x = ad(ad(2.)) y = ad(ad(3.)) z = ad(ad(2.)) # assert comparisons that should be true assert x == x assert x == z assert x != y assert x <= x assert x <= z assert x <= y assert x < y # assert comparisons that should be false assert not x == y assert not x != z assert not x != x assert not x >= y assert not x > y ```
Input File: example/compare_op.py
5.7: Standard Math Unary Functions

Syntax
Purpose
x
y
fun
Example

5.7.a: Syntax
`y = fun(x)`

5.7.b: Purpose
Evaluate the standard math function `fun` where `fun` has one argument.

5.7.c: x
The argument `x` can be an instance of `float`, an `a_float`, an `a2float`, or a `numpy.array` of such objects.

5.7.d: y
If `x` is an instance of `float`, `y` will also be an instance of `float`. Otherwise `y` will have the same type as `x` . ``` ```In the case where `x` is an array, `y` will the same shape as `x` and the elements of `y` will have the same type as the elements of `x` .

5.7.e: fun
The function `fun` can be any of the following: `arccos`, `arcsin`, `arctan`, `cos`, `cosh`, `exp`, `log`, `log10`, `sin`, `sinh`, `sqrt`, `tan`, or `tanh`.

5.7.f: Example
The file 5.7.1: std_math.py contains an example and test of these functions.
5.7.1: Standard Math Unary Functions: Example and Test

``` from pycppad import * import numpy import math # Example using a_float ---------------------------------------------------- def pycppad_test_std_math(): delta = 10. * numpy.finfo(float).eps pi = numpy.pi x = pi / 6 a_x = ad(x) # all the a_float unary standard math functions assert abs( arccos(a_x) - math.acos(x) ) < delta assert abs( arcsin(a_x) - math.asin(x) ) < delta assert abs( arctan(a_x) - math.atan(x) ) < delta assert abs( cos(a_x) - math.cos(x) ) < delta assert abs( cosh(a_x) - math.cosh(x) ) < delta assert abs( exp(a_x) - math.exp(x) ) < delta assert abs( log(a_x) - math.log(x) ) < delta assert abs( log10(a_x) - math.log10(x) ) < delta assert abs( sin(a_x) - math.sin(x) ) < delta assert abs( sinh(a_x) - math.sinh(x) ) < delta assert abs( sqrt(a_x) - math.sqrt(x) ) < delta assert abs( tan(a_x) - math.tan(x) ) < delta assert abs( tanh(a_x) - math.tanh(x) ) < delta # example array and derivative calculation n = 5 x = numpy.array( [2 * pi * j / n for j in range(n) ] ) a_x = independent(x) a_y = sin(a_x) f = adfun(a_x, a_y) J = f.jacobian(x) for j in range(n) : for k in range(n) : if j == k : assert abs( J[j][k] - cos( x[j] ) ) < delta else : assert J[j][k] == 0. # Example using a2float ---------------------------------------------------- def pycppad_test_std_math_a2(): n = 10 delta = 10. * numpy.finfo(float).eps pi = numpy.pi x = pi / 6 a2x = ad(ad(x)) # all the a2float unary standard math functions assert abs( arccos(a2x) - math.acos(x) ) < delta assert abs( arcsin(a2x) - math.asin(x) ) < delta assert abs( arctan(a2x) - math.atan(x) ) < delta assert abs( cos(a2x) - math.cos(x) ) < delta assert abs( cosh(a2x) - math.cosh(x) ) < delta assert abs( exp(a2x) - math.exp(x) ) < delta assert abs( log(a2x) - math.log(x) ) < delta assert abs( log10(a2x) - math.log10(x) ) < delta assert abs( sin(a2x) - math.sin(x) ) < delta assert abs( sinh(a2x) - math.sinh(x) ) < delta assert abs( sqrt(a2x) - math.sqrt(x) ) < delta assert abs( tan(a2x) - math.tan(x) ) < delta assert abs( tanh(a2x) - math.tanh(x) ) < delta # example array and derivative calculation n = 5 x = numpy.array( [2 * pi * j / n for j in range(n) ] ) a_x = ad(x) a2x = independent(a_x) a2y = sin(a2x) a_f = adfun(a2x, a2y) a_J = a_f.jacobian(a_x) for j in range(n) : for k in range(n) : if j == k : assert abs( a_J[j][k] - cos( x[j] ) ) < delta else : assert a_J[j][k] == 0. ```
Input File: example/std_math.py
5.8: Absolute Value Functions

Syntax
Purpose
x
y
Derivative
Directional Derivative
Example

5.8.a: Syntax
`y = abs(x)`

5.8.b: Purpose
Sets `y` equal to the absolute value of $\mathit{x}$ .

5.8.c: x
The argument `x` can be an instance of `float`, an `a_float`, an `a2float`, or an `numpy.array` of such objects.

5.8.d: y
If `x` is an instance of `float`, `y` will also be an instance of `float` . Otherwise `y` will have the same type as `x` . ``` ```In the case where `x` is an array, `y` will the same shape as `x` and the elements of `y` will have the same type as the elements of `x` .

5.8.e: Derivative
pycppad defines the derivative of the absolute value function by ${\mathrm{abs}}^{\left(1\right)}\left(\mathit{x}\right)=\mathrm{sign}\left(\mathit{x}\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\phantom{\rule{.3em}{0ex}}\mathit{x}>0\hfill \\ 0\hfill & \mathrm{if}\phantom{\rule{.3em}{0ex}}\mathit{x}=0\hfill \\ -1\hfill & \mathrm{if}\phantom{\rule{.3em}{0ex}}\mathit{x}<0\hfill \end{array}$
5.8.f: Directional Derivative
Prior to 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) , 6.4: forward mode computed the directional derivative of the absolute value function which is defined by ${\mathrm{abs}}^{\circ }\left(\mathit{x},\mathit{d}\right)=\underset{\mathrm{\lambda }↓0}{\mathrm{lim}}\frac{\mathrm{abs}\left(\mathit{x}+\mathrm{\lambda }\mathit{d}\right)-\mathrm{abs}\left(\mathit{x}\right)}{\mathrm{\lambda }}$ For $\mathit{x}\ne 0$ , ${\mathrm{abs}}^{\circ }\left(\mathit{x},\mathit{d}\right)={\mathrm{abs}}^{\left(1\right)}\left(\mathit{x}\right)*\mathit{d}$ and ${\mathrm{abs}}^{\circ }\left(0,\mathit{d}\right)=|\mathit{d}|$ .

5.8.g: Example
The file 5.8.1: abs.py contains an example and test of this function.
5.8.1: abs: Example and Test

``` # Example using a_float ---------------------------------------------------- from pycppad import * def pycppad_test_abs() : x = numpy.array( [ -1., 0., 1.] ) n = len(x) a_x = independent(x) a_y = abs( a_x ) f = adfun(a_x, a_y) f.forward(0, x) dx = numpy.zeros(n, dtype=float) for i in range( n ) : dx[i] = 1. df = f.forward(1, dx) if x[i] > 0. : assert df[i] == +1. elif x[i] < 0. : assert df[i] == -1. else : # There was a change in the CppAD specifictions for the abs function # see 12-30 on http://www.coin-or.org/CppAD/Doc/whats_new_11.htm assert df[i] == +1. or df[i] == 0. dx[i] = -1. df = f.forward(1, dx) if x[i] > 0. : assert df[i] == -1. elif x[i] < 0. : assert df[i] == +1. else : assert df[i] == +1 or df[i] == 0. dx[i] = 0. # Example using a2float ---------------------------------------------------- def pycppad_test_abs_a2() : x = ad( numpy.array( [-1, 0, 1] ) ) n = len(x) a_x = independent(x) a_y = abs( a_x ) f = adfun(a_x, a_y) f.forward(0, x) dx = numpy.array( list( ad(0) for i in range(n) ) ) for i in range( n ) : dx[i] = ad(0) for i in range( n ) : dx[i] = ad(1) df = f.forward(1, dx) if x[i] > 0. : assert df[i] == +1. elif x[i] < 0. : assert df[i] == -1. else : assert df[i] == +1. or df[i] == 0. dx[i] = ad(-1) df = f.forward(1, dx) if x[i] > 0. : assert df[i] == -1. elif x[i] < 0. : assert df[i] == +1. else: assert df[i] == +1. or df[i] == 0. dx[i] = ad(0) ```
Input File: example/abs.py
5.9: Conditional Expressions

Syntax
Purpose
rel
left
right
if_true
if_false
result
Example

5.9.a: Syntax
`result = condexp_rel(left, right, if_true, if_false)`

5.9.b: Purpose
Record, as part of an operation sequence, the conditional result ```      if( left op right )           result = if_true      else result = if_false ``` The relation `rel%` , and operator `op` , have the following correspondence: ```      rel   lt   le   eq   ge   gt       op    <   <=   ==    >   >= ```
5.9.c: rel
In the syntax above, the relation `rel` represents one of the following two characters: `lt`, `le`, `eq`, `ge`, `gt`. As in the table above, `rel` determines which comparison operator `op` is used when comparing `left` and `right` .

5.9.d: left
The argument `left` must have type `a_float` or `a2float`. It specifies the value for the left side of the comparison operator.

5.9.e: right
The argument `right` must have the same type as `left` . It specifies the value for the right side of the comparison operator.

5.9.f: if_true
The argument `if_true` must have the same type as `left` . It specifies the return value if the result of the comparison is true.

5.9.g: if_false
The argument `if_false` must have the same type as `left` . It specifies the return value if the result of the comparison is false.

5.9.h: result
This result has the same type as `left` .

5.9.i: Example
The file 5.9.1: condexp.py contains an example and test of these functions.
5.9.1: condexp: Example and Test

``` # Example using a_float ---------------------------------------------------- from pycppad import * def pycppad_test_condexp() : x = numpy.array( [1. , 1., 3., 4. ] ) a_x = independent(x) a_left = a_x; a_right = a_x; a_if_true = a_x; a_if_false = a_x; a_y_lt = condexp_lt(a_left, a_right, a_if_true, a_if_false); a_y_le = condexp_le(a_left, a_right, a_if_true, a_if_false); a_y_eq = condexp_eq(a_left, a_right, a_if_true, a_if_false); a_y_ge = condexp_ge(a_left, a_right, a_if_true, a_if_false); a_y_gt = condexp_gt(a_left, a_right, a_if_true, a_if_false); a_y = numpy.array( [ a_y_lt, a_y_le, a_y_eq, a_y_ge, a_y_gt ] ); f = adfun(a_x, a_y) y = f.forward(0, x) assert ( y == 4. ) # 1 < 1 is false so result is 4 assert ( y == 3. ) # 1 <= 1 is true so result is 3 assert ( y == 3. ) # 1 == 1 is true so result is 3 assert ( y == 3. ) # 1 >= 1 is true so result is 3 assert ( y == 4. ) # 1 > 2 is false so result is 4 x = numpy.array( [4., 3., 2., 1.] ) y = f.forward(0, x) assert ( y == 1. ) # 4 < 3 is false so result is 1 assert ( y == 1. ) # 4 <= 3 is false so result is 1 assert ( y == 1. ) # 4 == 3 is false so result is 1 assert ( y == 2. ) # 4 >= 3 is true so result is 2 assert ( y == 2. ) # 4 > 3 is true so result is 2 # Example using a2float ---------------------------------------------------- def pycppad_test_condexp_a2() : x = numpy.array( [1. , 1., 3., 4. ] ) a_x = ad(x) # begin level two recording of conditional expression a2x = independent(a_x) a2left = a2x; a2right = a2x; a2if_true = a2x; a2if_false = a2x; a2y_lt = condexp_lt(a2left, a2right, a2if_true, a2if_false); a2y_le = condexp_le(a2left, a2right, a2if_true, a2if_false); a2y_eq = condexp_eq(a2left, a2right, a2if_true, a2if_false); a2y_ge = condexp_ge(a2left, a2right, a2if_true, a2if_false); a2y_gt = condexp_gt(a2left, a2right, a2if_true, a2if_false); a2y = numpy.array( [ a2y_lt, a2y_le, a2y_eq, a2y_ge, a2y_gt ] ); a_f = adfun(a2x, a2y) # begin level one recording of conditional expression a_x = independent(x) a_y = a_f.forward(0, a_x) f = adfun(a_x, a_y) y = f.forward(0, x) assert ( y == 4. ) # 1 < 1 is false so result is 4 assert ( y == 3. ) # 1 <= 1 is true so result is 3 assert ( y == 3. ) # 1 == 1 is true so result is 3 assert ( y == 3. ) # 1 >= 1 is true so result is 3 assert ( y == 4. ) # 1 > 2 is false so result is 4 x = numpy.array( [4., 3., 2., 1.] ) y = f.forward(0, x) assert ( y == 1. ) # 4 < 3 is false so result is 1 assert ( y == 1. ) # 4 <= 3 is false so result is 1 assert ( y == 1. ) # 4 == 3 is false so result is 1 assert ( y == 2. ) # 4 >= 3 is true so result is 2 assert ( y == 2. ) # 4 > 3 is true so result is 2 ```
Input File: example/condexp.py

Contents

6.a: Contents
 independent: 6.1 Create an Independent Variable Vector adfun: 6.2 Create an AD Function Object abort_recording: 6.3 Abort a Recording of AD Operations forward: 6.4 Forward Mode: Derivative in One Domain Direction reverse: 6.5 Reverse Mode: Derivative in One Range Direction jacobian: 6.6 Driver for Computing Entire Derivative hessian: 6.7 Driver for Computing Hessian in a Range Direction optimize: 6.8 Optimize an AD Function Object Tape

6.1: Create an Independent Variable Vector

Syntax
Purpose
x
a_x
Example

6.1.a: Syntax
`a_x = independent(x)`

6.1.b: Purpose
Creates an independent variable vector and starts recording operations involving objects that are instances of `type(a_x)` . You must create an 6.2: adfun object, or use 6.3: abort_recording , to stop the recording before making another call to `independent`,

6.1.c: x
The argument `x` must be a `numpy.array` with one dimension (i.e., a vector). All the elements of `x` must all be of the same type and instances of either `int`, `float` or ` a_float`.

6.1.d: a_x
The return value `a_x` is a `numpy.array` with the same shape as `x` . If the elements of `x` are instances of `int` or `float` the elements of `a_x` are instances of `a_float`. If the elements of `x` are instances of `a_float` the elements of `a_x` are instances of `a2float`. The 5.2: value of the elements of `a_x` are equal to the corresponding elements of `x` .

6.1.e: Example
The file 6.1.1: independent.py contains an example and test of this function.
6.1.1: independent: Example and Test

``` from pycppad import * # Example using a_float --------------------------------------------------- def pycppad_test_independent() : x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # level 1 independent variables and start recording assert type(a_x) == numpy.ndarray for j in range(len(x)) : assert isinstance(x[j], float) assert isinstance(a_x[j], a_float) assert a_x[j] == x[j] f = adfun(a_x, a_x) # stop level 1 recording # Example using a2float --------------------------------------------------- def pycppad_test_independent_a2() : x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # level 1 independent variables and start recording a2x = independent(a_x) # level 2 independent variables and start recording assert type(a_x) == numpy.ndarray assert type(a2x) == numpy.ndarray for j in range(len(x)) : assert isinstance(x[j], float) assert isinstance(a_x[j], a_float) assert isinstance(a2x[j], a2float) assert a_x[j] == x[j] assert a2x[j] == x[j] a_f = adfun(a2x, a2x) # stop level 2 recording f = adfun(a_x, a_x) # stop level 1 recording ```
Input File: example/independent.py
6.2: Create an AD Function Object

Syntax
Purpose
a_x
a_y
f
m
n
level
Example

6.2.a: Syntax
`f = adfun(a_x, a_y)`

6.2.b: Purpose
The function object `f` will store the `type( a_x )` operation sequence that mapped the independent variable vector `a_x` to the dependent variable vector `a_y` .

6.2.c: a_x
The argument `a_x` is the `numpy.array` returned by the previous call to 6.1: independent . Neither the size of `a_x` , or the value it its elements, may change between calling ```      a_x = independent(x) ``` and ```      f = adfun(a_x, a_y) ``` The length of the vector `a_x` determines the domain size $\mathit{n}$ for the function $\mathit{y}=\mathit{F}\left(\mathit{x}\right)$ below.

6.2.d: a_y
The argument `a_y` specifies the dependent variables. It must be a `numpy.array` with one dimension (i.e., a vector) and with the same type of elements as `a_x` . The object `f` stores the `type( a_x )` operations that mapped the vector `a_x` to the vector `a_y` . The length of the vector `a_y` determines the range size $\mathit{m}$ for the function $\mathit{y}=\mathit{F}\left(\mathit{x}\right)$ below.

6.2.e: f
The return value `f` can be used to evaluate the function $\mathit{F}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{m}}$ and its derivatives, where $\mathit{y}=\mathit{F}\left(\mathit{x}\right)$ corresponds to the operation sequence mentioned above.

6.2.e.a: m
The range size $\mathit{m}$ is equal to the length of the vector `a_y` .

6.2.e.b: n
The domain size $\mathit{n}$ is equal to the length of the vector `a_x` .

6.2.e.c: level
The 5.1: ad level for the object `f` is one less than the AD level for the arguments `a_x` and `a_y` ; i.e., if `type( a_x )` is `a_float` (`a2float`) the corresponding AD level for `f` is zero (one).

6.2.f: Example
The file 6.2.1: adfun.py contains an example and test of this function.

``` from pycppad import * def pycppad_test_adfun() : # record operations at x = (0, 0, 0) x = numpy.array( [ 0., 0., 0. ] ) a_x = independent(x) # declare independent variables and start recording a_y0 = a_x; a_y1 = a_x * a_x; a_y2 = a_x * a_x * a_x; a_y = numpy.array( [ a_y0, a_y1, a_y2 ] ) f = adfun(a_x, a_y) # declare dependent variables and stop recording # evaluate function at x = (1, 2, 3) x = numpy.array( [ 1., 2., 3. ] ) y = f.forward(0, x) assert y == x assert y == x * x assert y == x * x * x ```
6.3: Abort a Recording of AD Operations

Syntax
Purpose
Example

6.3.a: Syntax
`abort_recording()`

6.3.b: Purpose
Sometimes it is necessary to abort the recording of AD operations that started with a call of the form ```      a_x = independent(x) ``` If such a recording is currently in progress, this will stop the recording and delete the corresponding information. Otherwise, `abort_recording` has no effect.

6.3.c: Example
The file 6.3.1: abort_recording.py contains an example and test of this operation. It returns true if it succeeds and false otherwise.
6.3.1: abort_recording: Example and Test

``` from pycppad import * # Example using a_float --------------------------------------------------- def pycppad_test_abort_recording() : from numpy import array try : x = numpy.array( [ 1., 2., 3. ] ) a_x = independent(x) # start first level recording a2_x = independent(a_x) # start second level recording a_y = array([sum(a_x)]) # record some operations if a_y > 2 : raise ValueError except ValueError : # Pretend that we are not sure if there are any active recordings # and use this call to terminate any that may exist. abort_recording() a_x = independent(x) # test starting a level 1 recording a2_x = independent(a_x) # test starting a level 2 recording a_y = array([sum(a_x)]) # record some level 1 operations f = adfun(a_x, a_y) # terminate level 1 recording y = f.forward(0, x) # evaluate the function at original x value assert( y == 6. ) # check the value abort_recording() # abort the level 2 recording ```
Input File: example/abort_recording.py
6.4: Forward Mode: Derivative in One Domain Direction

Syntax
Purpose
x_k
y_k
f
p
x_p
y_p
Example

6.4.a: Syntax
`y_p = f.forward(p, x_p)`

6.4.b: Purpose
We use $\mathit{F}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{m}}$ to denote the function corresponding to the `adfun` object 6.2.e: f . Given the `p`-th order Taylor expansion for a function $\mathit{X}:\mathbf{R}\to {\mathbf{R}}^{\mathit{n}}$ , this function can be used to compute the `p`-th order Taylor expansion for the function $\mathit{Y}:\mathbf{R}\to {\mathbf{R}}^{\mathit{m}}$ defined by $\mathit{Y}\left(\mathit{t}\right)=\mathit{F}\left[\mathit{X}\left(\mathit{t}\right)\right]$
6.4.c: x_k
For $\mathit{k}=0,\dots ,\mathit{p}$ , we use ${\mathit{x}}^{\left(\mathit{k}\right)}$ to denote the value of `x_k` in the most recent call to ```      f.forward(k, x_k) ``` including ${\mathit{x}}^{\left(\mathit{p}\right)}$ as the value `x_p` in this call. We define the function $\mathit{X}\left(\mathit{t}\right)$ by $\mathit{X}\left(\mathit{t}\right)={\mathit{x}}^{\left(0\right)}+{\mathit{x}}^{\left(1\right)}*\mathit{t}+\cdots +{\mathit{x}}^{\left(\mathit{p}\right)}*{\mathit{t}}^{\mathit{p}}$
6.4.d: y_k
For $\mathit{k}=0,\dots ,\mathit{p}$ , we use ${\mathit{y}}^{\left(\mathit{k}\right)}$ to denote the Taylor coefficients for $\mathit{Y}\left(\mathit{t}\right)=\mathit{F}\left[\mathit{X}\left(\mathit{t}\right)\right]$ expanded about zero; i.e., $\begin{array}{ccc}\hfill {\mathit{y}}^{\left(\mathit{k}\right)}& \hfill =\hfill & {\mathit{Y}}^{\left(\mathit{k}\right)}\left(0\right)/\mathit{k}!\hfill \\ \hfill \mathit{Y}\left(\mathit{t}\right)& \hfill =\hfill & {\mathit{y}}^{\left(0\right)}+{\mathit{y}}^{\left(1\right)}*\mathit{t}+\cdots +{\mathit{y}}^{\left(\mathit{p}\right)}*{\mathit{t}}^{\mathit{p}}+\mathit{o}\left({\mathit{t}}^{\mathit{p}}\right)\hfill \end{array}$ where $\mathit{o}\left({\mathit{t}}^{\mathit{p}}\right)/{\mathit{t}}^{\mathit{p}}\to 0$ as $\mathit{t}\to 0$ . The coefficient ${\mathit{y}}^{\left(\mathit{p}\right)}$ is equal to the value `y_p` returned by this call.

6.4.e: f
The object `f` must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.4.f: p
The argument `p` is a non-negative `int`. It specifies the order of the Taylor coefficient for $\mathit{Y}\left(\mathit{t}\right)$ that is computed.

6.4.g: x_p
The argument `x_p` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function `f` . It specifies the `p`-th order Taylor coefficient for $\mathit{X}\left(\mathit{t}\right)$ . If the AD 6.2.e.c: level for `f` is zero, all the elements of `x_p` must be either `int` or instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `x_p` must be `a_float` objects.

6.4.h: y_p
The return value `y_p` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function `f` . It is set to the `p`-th order Taylor coefficient for $\mathit{Y}\left(\mathit{t}\right)$ . If the AD 6.2.e.c: level for `f` is zero, all the elements of `y_p` will be instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `y_p` will be `a_float` objects.

6.4.i: Example
 6.4.1: forward_0.py Forward Order Zero: Example and Test 6.4.2: forward_1.py Forward Order One: Example and Test

6.4.1: Forward Order Zero: Example and Test

``` from pycppad import * # Example using a_float ---------------------------------------------------- def pycppad_test_forward_0() : # start record a_float operations x = numpy.array( [ 2., 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x * a_x ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp == 2. * x * x # f(x0, x1) = 2 * x0 * x1 # Example using a2float ---------------------------------------------------- def pycppad_test_forward_0_a2() : # start record a_float operations a_x = ad(numpy.array( [ 2., 3. ] )) # a_float value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x * a2x ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad( numpy.array( [ 3. , 4. ] ) ) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp == 2. * a_x * a_x # f(x0, x1) = 2 * x0 * x1 ```
Input File: example/forward_0.py
6.4.2: Forward Order One: Example and Test

``` from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_forward_1() : # start record a_float operations x = numpy.array( [ 2., 3. ] ) # value of independent variables a_x = independent(x) # declare independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x * a_x ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp == 2. * x * x # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative of f(x0, x1) with respect to x0 p = 1 # order one for first derivatives xp = numpy.array( [ 1. , 0. ] ) # direction for differentiation fp = f.forward(p, xp) # value of directional derivative assert fp == 2. * x # f_x0 (x0, x1) = 2 * x1 # evalute partial derivative of f(x0, x1) with respect to x p = 1 xp = numpy.array( [ 0. , 1. ] ) # the x1 direction fp = f.forward(p, xp) assert fp == 2. * x # f_x1 (x0, x1) = 2 * x0 # Example using a2float ----------------------------------------------------- def pycppad_test_forward_1_a2() : # start record a_float operations a_x = ad(numpy.array( [ 2., 3. ] )) # a_float value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x * a2x ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad(numpy.array( [ 3. , 4. ] )) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp == 2. * a_x * a_x # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative of f(x0, x1) with respect to x0 p = 1 # order one for first derivatives a_xp = ad(numpy.array( [ 1. , 0. ] )) # direction for differentiation a_fp = a_f.forward(p, a_xp) # value of directional derivative assert a_fp == 2. * a_x # f_x0 (x0, x1) = 2 * x1 # evalute partial derivative of f(x0, x1) with respect to x p = 1 a_xp = ad(numpy.array( [ 0. , 1. ] )) # the x1 direction a_fp = a_f.forward(p, a_xp) assert a_fp == 2. * a_x # f_x1 (x0, x1) = 2 * x0 ```
Input File: example/forward_1.py
6.5: Reverse Mode: Derivative in One Range Direction

Syntax
Purpose
x_k
X(t, u)
W(t, u)
f
p
w
dw
First Order
Second Order
Example

6.5.a: Syntax
`dw = f.forward(p, w)`

6.5.b: Purpose
Reverse mode computes the derivative of the 6.4: forward more Taylor coefficients with respect to the domain variable $\mathit{x}$ .

6.5.c: x_k
For $\mathit{k}=0,\dots ,\mathit{p}$ , we use ${\mathit{x}}^{\left(\mathit{k}\right)}$ to denote the value of `x_k` in the most recent call to ```      f.forward(k, x_k) ``` We use $\mathit{F}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{m}}$ to denote the function corresponding to the `adfun` object 6.2.e: f .

6.5.d: X(t, u)
We define the function $\mathit{X}:\mathbf{R}×{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{n}}$ by $\mathit{X}\left(\mathit{t},\mathit{u}\right)=\mathit{u}+{\mathit{x}}^{\left(0\right)}+{\mathit{x}}^{\left(1\right)}*\mathit{t}+\cdots +{\mathit{x}}^{\left(\mathit{p}-1\right)}*{\mathit{t}}^{\mathit{p}-1}$ Note that for $\mathit{k}=0,\dots ,\mathit{p}-1$ , ${\mathit{x}}^{\left(\mathit{k}\right)}=\frac{1}{\mathit{k}!}\frac{{\partial }^{\mathit{k}}}{\partial {\mathit{t}}^{\mathit{k}}}\mathit{X}\left(0,0\right)$
6.5.e: W(t, u)
The function $\mathit{W}:\mathbf{R}×{\mathbf{R}}^{\mathit{n}}\to \mathbf{R}$ is defined by $\mathit{W}\left(\mathit{t},\mathit{u}\right)={\mathit{w}}_{0}*{\mathit{F}}_{0}\left[\mathit{X}\left(\mathit{t},\mathit{u}\right)\right]+\cdots +{\mathit{w}}_{\mathit{m}-1}*{\mathit{F}}_{\mathit{m}-1}\left[\mathit{X}\left(\mathit{t},\mathit{u}\right)\right]$ We define the function ${\mathit{W}}_{\mathit{k}}:{\mathbf{R}}^{\mathit{n}}\to \mathbf{R}$ by ${\mathit{W}}_{\mathit{k}}\left(\mathit{u}\right)=\frac{1}{\mathit{k}!}\frac{{\partial }^{\mathit{k}}}{\partial {\mathit{t}}^{\mathit{k}}}\mathit{W}\left(0,\mathit{u}\right)$ It follows that $\mathit{W}\left(\mathit{t},\mathit{u}\right)={\mathit{W}}_{0}\left(\mathit{u}\right)+{\mathit{W}}_{1}\left(\mathit{u}\right)*\mathit{t}+\cdots +{\mathit{W}}_{\mathit{p}-1}\left(\mathit{u}\right)*{\mathit{t}}^{\mathit{p}-1}+\mathit{o}\left({\mathit{t}}^{\mathit{p}-1}\right)$ where $\mathit{o}\left({\mathit{t}}^{\mathit{p}-1}\right)/{\mathit{t}}^{\mathit{p}-1}\to 0$ as $\mathit{t}\to 0$ .

6.5.f: f
The object `f` must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.5.g: p
The argument `p` is a non-negative `int`. It specifies the order of the Taylor coefficient ${\mathit{W}}_{\mathit{p}-1}\left(\mathit{u}\right)$ that is differentiated. Note that ${\mathit{W}}_{\mathit{p}-1}\left(\mathit{u}\right)$ corresponds a derivative of order $\mathit{p}-1$ of $\mathit{F}\left(\mathit{x}\right)$ , so the derivative of ${\mathit{W}}_{\mathit{p}-1}\left(\mathit{u}\right)$ corresponds to a derivative of order $\mathit{p}$ of $\mathit{F}\left(\mathit{x}\right)$ .

6.5.h: w
The argument `w` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function `f` . It specifies the weighting vector $\mathit{w}$ used in the definition of $\mathit{W}\left(\mathit{t},\mathit{u}\right)$ . If the AD 6.2.e.c: level for `f` is zero, all the elements of `w` must be either `int` or instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `w` must be `a_float` objects.

6.5.i: dw
The return value `v` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function `f` . It is set to the derivative $\begin{array}{ccc}\hfill \mathit{dw}& \hfill =\hfill & {\mathit{W}}_{\mathit{p}-1}^{\left(1\right)}\left(0\right)\hfill \\ \hfill & \hfill =\hfill & {\partial }_{\mathit{u}}\frac{1}{\left(\mathit{p}-1\right)!}\frac{{\partial }^{\mathit{p}-1}}{\partial {\mathit{t}}^{\mathit{p}-1}}\mathit{W}\left(0,0\right)\hfill \end{array}$ If the AD 6.2.e.c: level for `f` is zero, all the elements of `dw` will be instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `dw` will be `a_float` objects.

6.5.j: First Order
In the case where $\mathit{p}=1$ , we have $\begin{array}{ccc}\hfill \mathit{dw}& \hfill =\hfill & {\partial }_{\mathit{u}}\frac{1}{0!}\frac{{\partial }^{0}}{\partial {\mathit{t}}^{0}}\mathit{W}\left(0,0\right)\hfill \\ \hfill & \hfill =\hfill & {\partial }_{\mathit{u}}\mathit{W}\left(0,0\right)\hfill \\ \hfill & \hfill =\hfill & {\partial }_{\mathit{u}}{\left[{\mathit{w}}_{0}*{\mathit{F}}_{0}\left(\mathit{u}+{\mathit{x}}^{\left(0\right)}\right)+\cdots +{\mathit{w}}_{\mathit{m}-1}{\mathit{F}}_{\mathit{m}-1}\left(\mathit{u}+{\mathit{x}}^{\left(0\right)}\right)\right]}_{\mathit{u}=0}\hfill \\ \hfill & \hfill =\hfill & {\mathit{w}}_{0}*{\mathit{F}}_{0}^{\left(1\right)}\left({\mathit{x}}^{\left(0\right)}\right)+\cdots +{\mathit{w}}_{\mathit{m}-1}*{\mathit{F}}_{\mathit{m}-1}^{\left(1\right)}\left({\mathit{x}}^{\left(0\right)}\right)\hfill \end{array}$
6.5.k: Second Order
In the case where $\mathit{p}=2$ , we have $\begin{array}{ccc}\hfill \mathit{dw}& \hfill =\hfill & {\partial }_{\mathit{u}}\frac{1}{1!}\frac{{\partial }^{1}}{\partial {\mathit{t}}^{1}}\mathit{W}\left(0,0\right)\hfill \\ \hfill & \hfill =\hfill & {\partial }_{\mathit{u}}{\left[{\mathit{w}}_{0}*{\mathit{F}}_{0}^{\left(1\right)}\left(\mathit{u}+{\mathit{x}}^{\left(0\right)}\right)*{\mathit{x}}^{\left(1\right)}+\cdots +{\mathit{w}}_{\mathit{m}-1}*{\mathit{F}}_{\mathit{m}-1}^{\left(1\right)}\left(\mathit{u}+{\mathit{x}}^{\left(0\right)}\right)*{\mathit{x}}^{\left(1\right)}\right]}_{\mathit{u}=0}\hfill \\ \hfill & \hfill =\hfill & {\mathit{w}}_{0}*\left({\mathit{x}}^{\left(1\right)}{\right)}^{\mathrm{T}}*{\mathit{F}}_{0}^{\left(2\right)}\left({\mathit{x}}^{\left(0\right)}\right)+\cdots +{\mathit{w}}_{\mathit{m}-1}*\left({\mathit{x}}^{\left(1\right)}{\right)}^{\mathrm{T}}*{\mathit{F}}_{\mathit{m}-1}^{\left(2\right)}\left({\mathit{x}}^{\left(0\right)}\right)\hfill \end{array}$
6.5.l: Example
 6.5.1: reverse_1.py Reverse Order One: Example and Test 6.5.2: reverse_2.py Reverse Order Two: Example and Test

6.5.1: Reverse Order One: Example and Test

``` from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_reverse_1(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x * a_x ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values x = numpy.array( [ 3. , 4. ] ) # argument value fp = f.forward(p, x) # function value assert fp == 2. * x * x # f(x0, x1) = 2 * x0 * x1 # evalute derivative of f(x0, x1) p = 1 # order one for first derivatives w = numpy.array( [ 1. ] ) # weight in range space fp = f.reverse(p, w) # derivaitive of weighted function assert fp == 2. * x # f_x0 (x0, x1) = 2 * x1 assert fp == 2. * x # f_x1 (x0, x1) = 2 * x0 # Example using a2float ------------------------------------------------------ def pycppad_test_reverse_1_a2(): # start record a_float operations a_x = ad(numpy.array( [ 2. , 3. ] )) # value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x * a2x ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at a different argument value p = 0 # order zero for function values a_x = ad(numpy.array( [ 3. , 4. ] )) # argument value a_fp = a_f.forward(p, a_x) # function value assert a_fp == 2. * a_x * a_x # f(x0, x1) = 2 * x0 * x1 # evalute derivative of f(x0, x1) p = 1 # order one for first derivatives a_w = ad(numpy.array( [ 1. ] )) # weight in range space a_fp = a_f.reverse(p, a_w) # derivaitive of weighted function assert a_fp == 2. * a_x # f_x0 (x0, x1) = 2 * x1 assert a_fp == 2. * a_x # f_x1 (x0, x1) = 2 * x0 ```
Input File: example/reverse_1.py
6.5.2: Reverse Order Two: Example and Test

``` from pycppad import * # Example using a_float ------------------------------------------------------ def pycppad_test_reverse_2(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = independent(x) # declare a_float independent variables # stop recording and store operations in the function object f a_y = numpy.array( [ 2. * a_x * a_x ] ) # dependent variables f = adfun(a_x, a_y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at same argument value p = 0 # derivative order x_p = x # zero order Taylor coefficient f_p = f.forward(0, x_p) # function value assert f_p == 2. * x * x # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative with respect to x p = 1 # derivative order x_p = numpy.array( [ 1. , 0 ] ) # first order Taylor coefficient f_p = f.forward(1, x_p) # partial w.r.t. x0 assert f_p == 2. * x # f_x0 (x0, x1) = 2 * x1 # evaluate derivative of partial w.r.t. x p = 2 # derivative order w = numpy.array( [1.] ) # weighting vector dw = f.reverse(p, w) # derivaitive of weighted function assert dw == 0. # f_x0_x1 (x0, x1) = 0 assert dw == 2. # f_x0_x1 (x0, x1) = 2 # Example using a2float ------------------------------------------------------ def pycppad_test_reverse_2_a2(): # start record a_float operations x = numpy.array( [ 2. , 3. ] ) # value of independent variables a_x = ad(x) # value of independent variables a2x = independent(a_x) # declare a2float independent variables # stop recording and store operations in the function object f a2y = numpy.array( [ 2. * a2x * a2x ] ) # dependent variables a_f = adfun(a2x, a2y) # f(x0, x1) = 2 * x0 * x1 # evaluate the function at same argument value p = 0 # derivative order x_p = a_x # zero order Taylor coefficient f_p = a_f.forward(0, x_p) # function value assert f_p == 2. * x * x # f(x0, x1) = 2 * x0 * x1 # evalute partial derivative with respect to x p = 1 # derivative order x_p = ad(numpy.array([1. , 0 ])) # first order Taylor coefficient f_p = a_f.forward(1, x_p) # partial w.r.t. x0 assert f_p == 2. * x # f_x0 (x0, x1) = 2 * x1 # evaluate derivative of partial w.r.t. x p = 2 # derivative order w = ad(numpy.array( [1.] )) # weighting vector dw = a_f.reverse(p, w) # derivaitive of weighted function assert dw == 0. # f_x0_x1 (x0, x1) = 0 assert dw == 2. # f_x0_x1 (x0, x1) = 2 ```
Input File: example/reverse_2.py
6.6: Driver for Computing Entire Derivative

Syntax
Purpose
f
x
J
Example

6.6.a: Syntax
`J = f.jacobian(x)`

6.6.b: Purpose
This routine computes the entire derivative ${\mathit{F}}^{\left(1\right)}\left(\mathit{x}\right)$ where $\mathit{F}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{m}}$ is the function corresponding to the `adfun` object 6.2.e: f .

6.6.c: f
The object `f` must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.6.d: x
The argument `x` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function `f` . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for `f` is zero, all the elements of `x` must be either `int` or instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `x` must be `a_float` objects.

6.6.e: J
The return value `J` is a `numpy.array` with two dimensions (i.e., a matrix). The first dimension (row size) is equal to 6.2.e.a: m (the number of range components in the function `f` ). The second dimension (column size) is equal to 6.2.e.b: n (the number of domain components in the function `f` ). It is set to the derivative; i.e., $\mathit{J}={\mathit{F}}^{\left(1\right)}\left(\mathit{x}\right)$ If the AD 6.2.e.c: level for `f` is zero, all the elements of `J` will be instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `J` will be `a_float` objects.

6.6.f: Example
The file 6.6.1: jacobian.py contains an example and test of this operation.
6.6.1: Entire Derivative: Example and Test

``` from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_jacobian(): delta = 10. * numpy.finfo(float).eps x = numpy.array( [ 0., 0. ] ) a_x = independent(x) a_y = numpy.array( [ a_x * exp(a_x) , a_x * sin(a_x) , a_x * cos(a_x) ] ) f = adfun(a_x, a_y) x = numpy.array( [ 2., 3. ] ) J = f.jacobian(x) assert abs( J[0,0] - exp(x) ) < delta assert abs( J[0,1] - x * exp(x) ) < delta assert abs( J[1,0] - sin(x) ) < delta assert abs( J[1,1] - x * cos(x) ) < delta assert abs( J[2,0] - cos(x) ) < delta assert abs( J[2,1] + x * sin(x) ) < delta # Example using a2float ----------------------------------------------------- def pycppad_test_jacobian_a2(): delta = 10. * numpy.finfo(float).eps a_x = ad( numpy.array( [ 0., 0. ] ) ) a2x = independent(a_x) a2y = numpy.array( [ a2x * exp(a2x) , a2x * sin(a2x) , a2x * cos(a2x) ] ) a_f = adfun(a2x, a2y) x = numpy.array( [2., 3.] ) a_x = ad(x) a_J = a_f.jacobian(a_x) assert abs( a_J[0,0] - exp(x) ) < delta assert abs( a_J[0,1] - x * exp(x) ) < delta assert abs( a_J[1,0] - sin(x) ) < delta assert abs( a_J[1,1] - x * cos(x) ) < delta assert abs( a_J[2,0] - cos(x) ) < delta assert abs( a_J[2,1] + x * sin(x) ) < delta ```
Input File: example/jacobian.py
6.7: Driver for Computing Hessian in a Range Direction

Syntax
Purpose
f
x
w
H
Example

6.7.a: Syntax
`H = f.hessian(x, w)`

6.7.b: Purpose
This routine computes the Hessian of the weighted sum ${\mathit{w}}_{0}*{\mathit{F}}_{0}\left(\mathit{x}\right)+\cdots +{\mathit{w}}_{\mathit{m}-1}*{\mathit{F}}_{\mathit{m}-1}\left(\mathit{x}\right)$ where $\mathit{F}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{m}}$ is the function corresponding to the `adfun` object 6.2.e: f .

6.7.c: f
The object `f` must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.7.d: x
The argument `x` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function `f` . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for `f` is zero, all the elements of `x` must be either `int` or instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `x` must be `a_float` objects.

6.7.e: w
The argument `w` is a `numpy.array` with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function `f` . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for `f` is zero, all the elements of `w` must be either `int` or instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `w` must be `a_float` objects.

6.7.f: H
The return value `H` is a `numpy.array` with two dimensions (i.e., a matrix). Both its first and second dimension size (row and column size) are equal to 6.2.e.b: n (the number of domain components in the function `f` ). It is set to the Hessian; i.e., $\mathit{H}={\mathit{w}}_{0}*{\mathit{F}}_{0}^{\left(2\right)}\left(\mathit{x}\right)+\cdots +{\mathit{w}}_{\mathit{m}-1}*{\mathit{F}}_{\mathit{m}-1}^{\left(2\right)}\left(\mathit{x}\right)$ If the AD 6.2.e.c: level for `f` is zero, all the elements of `H` will be instances of `float`. If the AD 6.2.e.c: level for `f` is one, all the elements of `H` will be `a_float` objects.

6.7.g: Example
The file 6.7.1: hessian.py contains an example and test of this operation.
6.7.1: Hessian Driver: Example and Test

``` from pycppad import * # Example using a_float ----------------------------------------------------- def pycppad_test_hessian(): delta = 10. * numpy.finfo(float).eps x = numpy.array( [ 0., 0. ] ) a_x = independent(x) a_y = numpy.array( [ a_x * exp(a_x) , a_x * sin(a_x) , a_x * cos(a_x) ] ) f = adfun(a_x, a_y) x = numpy.array( [ 2., 3. ] ) w = numpy.array( [ 0., 1., 0. ] ) # compute Hessian of x0 * sin(x1) H = f.hessian(x, w) assert abs( H[0,0] - 0. ) < delta assert abs( H[0,1] - cos(x) ) < delta assert abs( H[1,0] - cos(x) ) < delta assert abs( H[1,1] + x * sin(x) ) < delta # Example using a2float ----------------------------------------------------- def pycppad_test_hessian_a2(): delta = 10. * numpy.finfo(float).eps a_x = ad( numpy.array( [ 0., 0. ] ) ) a2x = independent(a_x) a2y = numpy.array( [ a2x * exp(a2x) , a2x * sin(a2x) , a2x * cos(a2x) ] ) a_f = adfun(a2x, a2y) x = numpy.array( [ 2., 3. ] ) a_x = ad(x) a_w = ad( numpy.array( [ 0., 1., 0. ] ) ) # compute Hessian of x0 * sin(x1) a_H = a_f.hessian(a_x, a_w) assert abs( a_H[0,0] - 0. ) < delta assert abs( a_H[0,1] - cos(x) ) < delta assert abs( a_H[1,0] - cos(x) ) < delta assert abs( a_H[1,1] + x * sin(x) ) < delta ```
Input File: example/hessian.py
6.8: Optimize an AD Function Object Tape

Syntax
Purpose
f
Efficiency
Example

6.8.a: Syntax
`f.optimize()`

6.8.b: Purpose
The operation sequence corresponding to an 6.2: ADFun object can be very large and involve many operations. The `f.optimize` procedure reduces the number of operations, and thereby the time and memory, required to compute function and derivative values.

6.8.c: f
The object `f` is an 6.2: adfun object.

6.8.d: Efficiency
The `optimize` member function may greatly reduce the size of the operation sequence corresponding to `f` .

6.8.e: Example
The file 6.8.1: optimize.py contains an example and test of this operation.
6.8.1: Optimize Function Object: Example and Test

``` from pycppad import * import time # Example using a_float ----------------------------------------------------- def pycppad_test_optimize(): # create function with many variables that are get removed by optimize n_sum = 10000 x = numpy.array( [ 0. ] ) a_x = independent(x) a_sum = 0. for i in range(n_sum) : a_sum = a_sum + a_x; a_y = numpy.array( [ a_sum ] ) f = adfun(a_x, a_y) # time for a forward operations before optimize x = numpy.array( [ 1. ] ) t0 = time.time() sum_before = f.forward(0, x) sec_before = time.time() - t0 # time for a forward operations after optimize f.optimize() t0 = time.time() sum_after = f.forward(0, x) sec_after = time.time() - t0 assert sum_before == float(n_sum) assert sum_after == float(n_sum) # expect sec_before to be less than 2 times sec_after assert( sec_after * 1.5 <= sec_before ) ```
Input File: example/optimize.py
7: Using Two Levels of AD: Example and Test

Purpose
F(u)
G(x)

7.a: Purpose
This example is intended to demonstrate how `a_float` and `a2float` can be used together to compute derivatives of functions that are defined in terms of derivatives of other functions.

7.b: F(u)
For this example, the function $\mathit{F}:{\mathbf{R}}^{2}\to \mathbf{R}$ is defined by $\mathit{F}\left(\mathit{u}\right)={\mathit{u}}_{0}^{2}+{\mathit{u}}_{1}^{2}$ It follows that $\begin{array}{c}\hfill {\partial }_{\mathit{u}\left(0\right)}\mathit{F}\left(\mathit{u}\right)=2*{\mathit{u}}_{0}\\ \hfill {\partial }_{\mathit{u}\left(1\right)}\mathit{F}\left(\mathit{u}\right)=2*{\mathit{u}}_{1}\end{array}$
7.c: G(x)
For this example, the function $\mathit{G}:{\mathbf{R}}^{2}\to \mathbf{R}$ is defined by $\mathit{G}\left(\mathit{x}\right)={\mathit{x}}_{1}*{\partial }_{\mathit{u}\left(0\right)}\mathit{F}\left({\mathit{x}}_{0},1\right)+{\mathit{x}}_{0}*{\partial }_{\mathit{u}\left(1\right)}\mathit{F}\left({\mathit{x}}_{0},1\right)$ where $\partial \mathit{u}\left(\mathit{j}\right)\mathit{F}\left(\mathit{a},\mathit{b}\right)$ denotes the partial of $\mathit{F}$ with respect to ${\mathit{u}}_{\mathit{j}}$ and evaluated at $\mathit{u}=\left(\mathit{a},\mathit{b}\right)$ . It follows that $\begin{array}{ccc}\hfill \mathit{G}\left(\mathit{x}\right)& \hfill =\hfill & 2*{\mathit{x}}_{1}*{\mathit{x}}_{0}+2*{\mathit{x}}_{0}\hfill \\ \hfill {\partial }_{\mathit{x}\left(0\right)}\mathit{G}\left(\mathit{x}\right)& \hfill =\hfill & 2*{\mathit{x}}_{1}+2\hfill \\ \hfill {\partial }_{\mathit{x}\left(1\right)}\mathit{G}\left(\mathit{x}\right)& \hfill =\hfill & 2*{\mathit{x}}_{0}\hfill \end{array}$ ``` from pycppad import * def pycppad_test_two_levels(): # start recording a_float operations x = numpy.array( [ 2. , 3. ] ) a_x = independent(x) # start recording a2float operations a_u = numpy.array( [a_x , ad(1) ] ) a2u = independent(a_u) # stop a2float recording and store operations if f a2v = numpy.array( [ a2u * a2u + a2u * a2u ] ) a_f = adfun(a2u, a2v) # F(u0, u1) = u0 * u0 + u1 * u1 # evaluate the gradient of F a_J = a_f.jacobian(a_u) # stop a_float recording and store operations in g a_y = numpy.array( [ a_x * a_J[0,0] + a_x * a_J[0,1] ] ) g = adfun(a_x, a_y) # G(x0, x1) = x1 * F_u0(x0, 1) + x0 * F_u1(x0, 1) # evaluate the gradient of G J = g.jacobian(x) assert J[0,0] == 2. * x + 2 assert J[0,1] == 2. * x ```
Input File: example/two_levels.py
8: Fourth Order Runge Kutta

Syntax
Purpose
f
ti
yi
dt
yf
Motivation
Source Code
Example

8.a: Syntax
` yf = runge_kutta_4(f, ti, yi, dt)`

8.b: Purpose
See 8.h: Motivation below as well as the purpose described here. We are given a function $\mathit{f}:{\mathbf{R}}^{\mathit{n}}\to {\mathbf{R}}^{\mathit{n}}$ , and a point $\mathit{yi}\in {\mathbf{R}}^{\mathit{n}}$ such that an unknown function $\mathit{y}:\mathbf{R}\to {\mathbf{R}}^{\mathit{n}}$ satisfies the equations $\begin{array}{ccc}\hfill \mathit{y}\left(\mathit{ti}\right)& \hfill =\hfill & \mathit{yi}\hfill \\ \hfill \mathit{y}\text{'}\left(\mathit{t}\right)& \hfill =\hfill & \mathit{f}\left[\mathit{t},\mathit{y}\left(\mathit{t}\right)\right]\hfill \\ \hfill \end{array}$ We use the Fourth order Runge-Kutta formula (see equation 16.1.2 of Numerical Recipes in Fortran, 2nd ed.) wish to approximate the value of $\mathit{yf}=\mathit{y}\left(\mathit{ti}+\mathrm{\Delta }\mathit{t}\right)$
8.c: f
If `t` is a scalar and `y` is a vector with size $\mathit{n}$ , ```      k = f(t, y) ``` returns a vector of size $\mathit{n}$ that is the value of $\mathit{f}\left(\mathit{t},\mathit{y}\right)$ at the specified values.

8.d: ti
is a scalar that specifies the value of $\mathit{ti}$ in the problem above.

8.e: yi
is a vector of size $\mathit{n}$ that specifies the value of $\mathit{yi}$ in the problem above.

8.f: dt
is a scalar that specifies the value of $\mathrm{\Delta }\mathit{t}$ in the problem above.

8.g: yf
is a vector of size $\mathit{n}$ that is the approximation for $\mathit{y}\left(\mathit{t}+\mathrm{\Delta }\mathit{t}\right)$ .

8.h: Motivation
This routine makes very few assumptions about the objects used to do these calculations. Thus, smart objects can be used for all sorts of purposes; for example, computing derivatives of the solution of an ODE. The table below lists the assumptions on the objects passed into `runge_kutta_4`. In this table, `s` and `t` are scalars, `d` is a decimal number (i.e., a `float`) and `u` and `v` are vectors with size $\mathit{n}$ .
 operation result `d * s` scalar `s + t` scalar `s * u` vector with size $\mathit{n}$ `d * u` vector with size $\mathit{n}$ `s * u` vector with size $\mathit{n}$ `u + v` vector with size $\mathit{n}$

8.i: Source Code ``` def runge_kutta_4(f, ti, yi, dt) : k1 = dt * f(ti , yi) k2 = dt * f(ti + .5*dt , yi + .5*k1) k3 = dt * f(ti + .5*dt , yi + .5*k2) k4 = dt * f(ti + dt , yi + k3) yf = yi + (1./6.) * ( k1 + 2.*k2 + 2.*k3 + k4 ) return yf ```
8.j: Example
1. The file 8.1: runge_kutta_4_correct.py contains an example and test of using `runge_kutta_4` to solve an ODE.
2. The file 8.2: runge_kutta_4_ad.py contains an example and test of differentiating the numerical solution of an ODE.
3. The file 8.3: runge_kutta_4_cpp.py contains an example and test of using pycppad 6.2: adfun object to evaluate python functions with C++ speed of execution.

8.1: runge_kutta_4 A Correctness Example and Test

Discussion
Source Code

8.1.a: Discussion
Define $\mathit{y}:\mathbf{R}\to {\mathbf{R}}^{\mathit{n}}$ by ${\mathit{y}}_{\mathit{j}}\left(\mathit{t}\right)={\mathit{t}}^{\mathit{j}+1}$ It follows that the derivative of $\mathit{y}\left(\mathit{t}\right)$ satisfies the 8: runge_kutta_4 ODE equation where $\mathit{y}\left(0\right)=0$ and $\mathit{f}\left(\mathit{t},\mathit{y}\right)$ is given by $\mathit{f}\left(\mathit{t},\mathit{y}{\right)}_{\mathit{j}}={\mathit{y}}_{\mathit{j}}\text{'}\left(\mathit{t}\right)=\left\{\begin{array}{cc}1\hfill & \phantom{\rule{.3em}{0ex}}\mathrm{if}\phantom{\rule{.3em}{0ex}}\mathit{j}=0\hfill \\ \left(\mathit{j}+1\right){\mathit{y}}_{\mathit{j}-1}\left(\mathit{t}\right)\hfill & \phantom{\rule{.3em}{0ex}}\mathrm{otherwise}\hfill \end{array}$
8.1.b: Source Code ``` from pycppad import * def pycppad_test_runge_kutta_4_correct() : def fun(t , y) : n = y.size f = numpy.zeros(n) f = 1. index = numpy.array( range(n-1) ) + 1 f[index] = (index + 1) * y[index-1] return f n = 5 # size of y(t) (order of method plus 1) ti = 0. # initial time dt = 2. # a very large time step size to test correctness yi = numpy.zeros(n) # initial value for y(t); i.e., y(0) # take one 4-th order Runge-Kutta integration step of size dt yf = runge_kutta_4(fun, ti, yi, dt) # check the results t_jp = 1. # t^0 at t = dt for j in range(n-1) : t_jp = t_jp * dt # t^(j+1) at t = dt assert abs( yf[j] - t_jp ) < 1e-10 # check yf[j] = t^(j+1) ```
Input File: example/runge_kutta_4_correct.py
8.2: runge_kutta_4 An AD Example and Test

Discussion
Source Code

8.2.a: Discussion
Define $\mathit{y}:\mathbf{R}×\mathbf{R}\to {\mathbf{R}}^{\mathit{n}}$ by ${\mathit{y}}_{\mathit{j}}\left(\mathit{x},\mathit{t}\right)=\mathit{x}{\mathit{t}}^{\mathit{j}+1}$ It follows that the derivative of $\mathit{y}\left(\mathit{t}\right)$ satisfies the 8: runge_kutta_4 ODE equation where $\mathit{y}\left(0\right)=0$ and $\mathit{f}\left(\mathit{t},\mathit{y}\right)$ is given by $\mathit{f}\left(\mathit{t},\mathit{y}{\right)}_{\mathit{j}}={\partial }_{\mathit{t}}{\mathit{y}}_{\mathit{j}}\left(\mathit{x},\mathit{t}\right)=\left\{\begin{array}{cc}\mathit{x}\hfill & \phantom{\rule{.3em}{0ex}}\mathrm{if}\phantom{\rule{.3em}{0ex}}\mathit{j}=0\hfill \\ \left(\mathit{j}+1\right){\mathit{y}}_{\mathit{j}-1}\left(\mathit{x},\mathit{t}\right)\hfill & \phantom{\rule{.3em}{0ex}}\mathrm{otherwise}\hfill \end{array}$ It also follows that ${\partial }_{\mathit{x}}{\mathit{y}}_{\mathit{j}}\left(\mathit{x},\mathit{t}\right)={\mathit{t}}^{\mathit{j}+1}$
8.2.b: Source Code ``` from pycppad import * def pycppad_test_runge_kutta_4_ad() : def fun(t , y) : n = y.size f = ad( numpy.zeros(n) ) f = a_x index = numpy.array( range(n-1) ) + 1 f[index] = (index + 1) * y[index-1] return f n = 5 # size of y(t) (order of method plus 1) ti = 0. # initial time dt = 2. # a very large time step size (method is exact) # initial value for y(t); i.e., y(0) a_yi = ad( numpy.zeros(n) ) # declare a_x to be the independent variable vector x = numpy.array( [.5] ) a_x = independent( numpy.array( x ) ) # take one 4-th order Runge-Kutta integration step of size dt a_yf = runge_kutta_4(fun, ti, a_yi, dt) # define the AD function g : x -> yf g = adfun(a_x, a_yf) # compute the derivative of g w.r.t x at x equals .5 dg = g.jacobian(x) # check the result is as expected t_jp = 1 # t^0 for j in range(n-1) : t_jp = t_jp * dt # t^(j+1) at t = dt assert abs( a_yf[j] - x*t_jp ) < 1e-10 # check yf[j] = x*t^(j+1_ assert abs( dg[j,0] - t_jp ) < 1e-10 # check dg[j] = t^(j+1) ```
8.3: runge_kutta_4 With C++ Speed: Example and Test

Purpose
Discussion
Source Code

8.3.a: Purpose
In this example we demonstrate how a Python function can be recorded in a `pycppad` function object and then evaluated at much higher speeds than the Python evaluation.

8.3.b: Discussion
Define $\mathit{y}:{\mathbf{R}}^{2}×\mathbf{R}\to {\mathbf{R}}^{\mathit{n}}$ by $\begin{array}{ccc}\hfill \mathit{y}\left(\mathit{x},0\right)& \hfill =\hfill & {\mathit{x}}_{0}\hfill \\ \hfill {\partial }_{\mathit{t}}\mathit{y}\left(\mathit{x},\mathit{t}\right)& \hfill =\hfill & {\mathit{x}}_{1}\mathit{y}\left(\mathit{x},\mathit{t}\right)\hfill \end{array}$ It follows that $\mathit{y}\left(\mathit{x},\mathit{t}\right)={\mathit{x}}_{0}\mathrm{exp}\left({\mathit{x}}_{1}\mathit{t}\right)$ Suppose we want to compute values for the function $\mathit{g}:{\mathbf{R}}^{2}\to \mathbf{R}$ defined by $\mathit{g}\left(\mathit{x}\right)=\mathit{y}\left(\mathit{x},1\right)$ In this example we compare the execution time for doing this in pure Python with using a pycppad function object to compute $\mathit{g}\left(\mathit{x}\right)$ in C++.

8.3.c: Source Code ``` from pycppad import * import time def pycppad_test_runge_kutta_4_cpp() : x_1 = 0; # use this variable to switch x_1 between float and ad(float) def fun(t , y) : f = x_1 * y return f # Number of Runge-Kutta times steps to include in the function object M = 100 # Start time for recording the pycppad function object s0 = time.time() # Declare three independent variables. The operation sequence does not # depend on x, so we could use any value here. x = numpy.array( [.1, .1, .1] ) a_x = independent( numpy.array( x ) ) # First independent variables, x, is the value of y(0) a_y = numpy.array( [ a_x ] ) # Make x_1 a variable so can use rk4 with various coefficients. x_1 = a_x # Make dt a variable so can use rk4 with various step sizes. dt = a_x # f(t, y) does not depend on t, so no need to make t a variable. t = ad(0.) # Record the operations for 10 time step for k in range(M) : a_y = runge_kutta_4(fun, t, a_y, dt) t = t + dt # define the AD function rk4 : x -> y rk4 = adfun(a_x, a_y) # amount of time it took to tape this function object tape_sec = time.time() - s0 # make the fucntion object more efficient s0 = time.time() rk4.optimize() opt_sec = time.time() - s0 ti = 0. # initial time tf = 1. # final time N = M * 100 # number of time steps dt = (tf - ti) / N # size of time step x_0 = 2. # use this for initial value of y(t) x_1 = .5 # use this for coefficient in ODE # python version of integrator with float values s0 = time.time() t = ti y = numpy.array( [ x_0 ] ); for k in range(N) : y = runge_kutta_4(fun, t, y, dt) t = t + dt # number of seconds to solve the ODE using python float python_sec = time.time() - s0 # check solution is correct assert( abs( y - x_0 * exp( x_1 * tf ) ) < 1e-10 ) # pycppad function object version of integrator s0 = time.time() t = ti x = numpy.array( [ x_0 , x_1 , dt ] ) for k in range(N/M) : y = rk4.forward(0, x); x = y; # number of seconds to solve the ODE using pycppad function object cpp_sec = time.time() - s0 # check solution is correct assert( abs( y - x_0 * exp( x_1 * tf ) ) < 1e-10 ) # Uncomment the print statement below to see actual times on your machine format = 'cpp_sec = %8f,\n' format = format + 'python_sec/cpp_sec = %5.1f\n' format = format + 'tape_sec/cpp_sec = %5.1f\n' format = format + 'opt_sec/cpp_sec = %5.1f' s = cpp_sec # print format % (s, python_sec/s, tape_sec/s, opt_sec/s ) # check that C++ is always more than 30 times faster (on all systems) assert( 30. * cpp_sec <= python_sec ) ```
Input File: example/runge_kutta_4_cpp.py
9: Extensions, Bug Fixes, and Changes

2014-07-10
2014-05-29
Long Ago

9.a: 2014-07-10
Suppress warnings about using deprecated features of the numpy C-API; see numpy c-api deprecation (http://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html)

9.b: 2014-05-29
Add support for the 5.4.2: future divide operator .

9.c: Long Ago
9.1: 2010 , 9.2: 2010 , 9.3: 2010 , 9.4: 2008
Input File: omh/whats_new.omh
9.1: Extensions, Bug Fixes, and Changes During 2012

10-19
10-01
04-17
04-16

9.1.a: 10-19
The package `pycppad.cppad_` was missing form the installation (after testing). This is due to be a bug in the python distutils (http://docs.python.org/library/distutils.html) package when the `--inplace` option is specified. The 2.h: install instructions have been changed to avoid this problem.

9.1.b: 10-01
Change the 8.3: runge_kutta_4_cpp.py test from asserting that C++ is 100 times faster than Python to 75 times faster.

9.1.c: 04-17
1. Add missing `--inplace` flag in 2.f: build instructions.
2. Add `with_debugging` option to 2.g: testing so that `test_more.py` does not fail when you build without debugging.

9.1.d: 04-16
1. Simplify the `setup.py` script used during the 2: install process. This included having the user to download and install CppAD, instead of having `setup.py` do this task.
2. Document the change to the derivative of the 5.8: abs that occurred in CppAD on 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) .

Input File: omh/whats_new_12.omh
9.2: Extensions, Bug Fixes, and Changes During 2011

10-18
10-17
10-16
03-23
03-20

9.2.a: 10-18
1. Fix problem with `import cppad_` in `20111017` version.
2. Change 8.3: runge_kutta_4_cpp.py to use 6.8: optimize and to check that using `pycppad` is always more than 100 times faster than straight python for this test case.

9.2.b: 10-17
Add tape optimization; see 6.8: optimize .

9.2.c: 10-16
1. Add the conditional expressions; see 5.9: condexp .
2. Use newer version of CppAD that fixes warnings found by `g++` version 4.6.1.

9.2.d: 03-23
The `--undef` option is not available to `./setup.py build` so `./setup.py build` was changed to `./setup.py build_ext` in the 2.f: build instructions .

9.2.e: 03-20
1. There was a problem with the pycppad 2: install instructions. To be specific, the debug build was not being installed because of the presence of `--inplace` in the example `setup.py` command. This has been fixed in the new 2.f.a: debug build instructions .
2. Change from using `cppad-20100101.5` to using `cppad-20110101.2.gpl.tgz`; see CppAD whats_new (http://www.coin-or.org/CppAD/Doc/whats_new.htm) .
3. Use `external` sub directory of current working directory, instead of `\$HOME/install`, for default directory that holds a local copy of CppAD; see `cppad_parent_dir` under 2.e: Required Setup Information .
4. Add `boost_python_include_dir` to list of 2.e: Required Setup Information .

Input File: omh/whats_new_11.omh
9.3: Extensions, Bug Fixes, and Changes During 2010

12-12
12-05
11-30
03-18
02-10
01-31
01-30
01-29
01-28

9.3.a: 12-12
The 6.1: independent(x) and 6.2: adfun(x,y) functions require the elements of `x` and `y` to all have the same type. If this was not the case, `pycppad` might crash with out a useful error message. This has been fixed.

9.3.b: 12-05
First version that works (and installs properly) under MS windows using the cygwin (www.cygwin.com) system; see special pycppad instructions for installing under 2.f.d: cygwin .

9.3.c: 11-30
Upgraded from using `cppad-20100101.2` to `cppad-20100101.5`. This fixes a problem when installing pycppad with version 1.44.0 of boost-python (http://www.boost.org/doc/libs/1_44_0/libs/python/doc/index.html) .

9.3.d: 03-18
The upstream tarball for CppAD was missing from The 2: install process was failing because the upstream tarball was missing from http://www.coin-or.org/download/source/CppAD/ . This has been fixed, and a newer upstream release has been used.

9.3.e: 02-10
1. Added `--inplace` to the 2.f.b: optimized build instructions. Improved the 2.h: install instructions and the discussion of the 2.i: python path .
2. Changes `exit` to `sys.exit` in 2.g: test_example.py and `test_more.py` (required on some systems).
3. Fixed the 5.3.1: ad_unary.py example so it displays the correct source code and changed `all` to `numpy.all` in that example (required on some systems).

9.3.f: 01-31

9.3.g: 01-30
Change `setup.py` do that it patches the CppAD distribution (this fixes a problem with the 2.f.b: optimized build).

9.3.h: 01-29
Added the python ODE solver 8: runge_kutta_4 which can be used with `smart' floating point types (e.g., AD types).

9.3.i: 01-28
Upgraded from using `cppad-20090909.0` to `cppad-20100101.0`.
Input File: omh/whats_new_10.omh
9.4: Extensions, Bug Fixes, and Changes During 2009

06-21
06-13
05-13
05-12
02-13
02-09
01-31
01-30
01-29
01-28
01-26

9.4.a: 06-21
The Boost Python library name was not being used correctly and hence one would get the error message: "Cannot find the Boost Python library" ... This has been fixed.

9.4.b: 06-13
The `setup.py` file (used during the 2.f: build step of the install) had `/home/bradbell/install` as the value of `cppad_parent_dir`. This has been changed so that the default is `\$HOME/install`.

9.4.c: 05-13
Changed to a new version of CppAD, (20090303.0).

9.4.d: 05-12
The Boost Python library `libboost_python-py26.so` had a problem with `ad(x)` where `x` was of type `int`. This has been fixed.

9.4.e: 02-13
Changed to new version of CppAD, (20090131.0) so that `tanh` is now included in 5.7: std_math (it was documented but missing from the actual pycppad implementation before this date).

9.4.f: 02-09
Added cross reference links from the examples to the corresponding section. For example, when `independent` appears in example code, it is linked in the following way: 6.1: independent .

9.4.g: 01-31
2. Remove `tanh` from standard math functions because previous release of CppAD did not include it (a new release of CppAD is being built so that `tanh` can be included in `pycppad`).
3. The two tests in `example/ad_unary.py` had the same name and hence only one was being run. This has been fixed.

9.4.h: 01-30
1. Organize the documentation by grouping the 5: ad_variable and 6: ad_function operations.
2. Add a 3: get_started.py section.
3. Add the 5.3: unary plus and minus operators .
4. Sort the installation 2.g: testing results in alphabetical order.
5. Use the `**` exponentiation operator in 3: get_started.py .
6. Put this whats new section in inverse order by date.
7. Improve the error reporting done by the 5.2: value function.

9.4.i: 01-29
A list of 4: examples has been added to the documentation. The code for the 7: two_levels.py example was missing, this has been fixed. ``` ```The `pycppad` tests no longer require any package to run; i.e., it is no longer necessary to install --       py.test
-
(- http://codespeak.net/py/dist/) for testing `pycppad`.

9.4.j: 01-28
Change from an general AD package to just one package; i.e. from `pyad-yyyymmdd.tar.gz` to `pycppad-yyyymmdd.tar.gz` . This is a change in API requiring user code to change `import pyad.cppad` to `import pycppad`.

9.4.k: 01-26
Released first version, `pyad-20090126`. ``` ```Add build inplace, and omitting the `setup.py` install step, to 2: install documentation.
Input File: omh/whats_new_09.omh

```-------------------------------------------------------------------------------
Authors: Sebastian F. Walter and Bradley M. Bell.

as it was on 2009-01-24 with the following substutions:
<YEAR>         = 2008-2009
<OWNER>        = Bradley M. Bell and Sebastian F. Walter
<ORGANIZATION> = contributors' organizations
In addition,   "Neither the name of the contributors' organizations"
was changed to "Neither the names of the contributors' organizations"
-------------------------------------------------------------------------------

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the contributors' organizations nor the names of
its contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```

11: Alphabetic Listing of Cross Reference Tags

A
C
E
F
G
H
I
J
L
O
P
R
S
T
V
W

12: Keyword Index

A
B
C
D
E
F
G
H
I
J
L
M
N
O
P
R
S
T
U
V
Z
!= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
* 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
** 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
*= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
+
binary 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
unary 5.3.1: Unary Plus and Minus Operators: Example and Test
unary 5.3: Unary Plus and Minus Operators
+= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
-
binary 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
unary 5.3.1: Unary Plus and Minus Operators: Example and Test
unary 5.3: Unary Plus and Minus Operators
-= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
/ 5.4: Binary Numeric Operators With an AD Result
example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
future example 5.4.2: Future Division Operator: Example and Test
/= 5.5: Computed Assignment Operators
example 5.5.1: Computed Assignment Operators: Example and Test
2009
feature 9.4: Extensions, Bug Fixes, and Changes During 2009
< 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
<= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
== 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
> 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
>= 5.6: Binary Comparison Operators
example 5.6.1: a_float Comparison Operators: Example and Test
A
abort recording 6.3: Abort a Recording of AD Operations
binary numeric operator 5.4: Binary Numeric Operators With an AD Result
decrease level 5.2: Create an Object With One Lower Level of AD
increase level 5.1: Create an Object With One Higher Level of AD
python : pycppad-20140710: A Python Algorithm Derivative Package
two levels 7: Using Two Levels of AD: Example and Test
unary operator 5.3: Unary Plus and Minus Operators
a2float 7: Using Two Levels of AD: Example and Test
create 5.1.d: Create an Object With One Higher Level of AD: a_x
a_float
comparison operator 5.6.1: a_float Comparison Operators: Example and Test
create 5.1.d: Create an Object With One Higher Level of AD: a_x
abort
abort_recording
example 6.3.1: abort_recording: Example and Test
abs 5.9: Conditional Expressions
5.8: Absolute Value Functions
example 5.8.1: abs: Example and Test
example 5.1.1: ad: Example and Test
example 6.2.1: adfun: Example and Test
algorithmic differentiation
python : pycppad-20140710: A Python Algorithm Derivative Package
arccos 5.7: Standard Math Unary Functions
arcsin 5.7: Standard Math Unary Functions
arctan 5.7: Standard Math Unary Functions
assignment
assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
computed operator 5.5: Computed Assignment Operators
automatic differentiation
python : pycppad-20140710: A Python Algorithm Derivative Package
B
binary
bool a_float operator example 5.6.1: a_float Comparison Operators: Example and Test
bool operator 5.6: Binary Comparison Operators
comparison operator 5.6: Binary Comparison Operators
numeric operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
bool
a_float comparison operator example 5.6.1: a_float Comparison Operators: Example and Test
binary operator 5.6: Binary Comparison Operators
bug
fix 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
python distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
build
debugging 2.f.a: Installing pycppad: Building.With Debugging
C
C++ speed
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
comparison
a_float operator 5.6.1: a_float Comparison Operators: Example and Test
binary operator 5.6: Binary Comparison Operators
computed
assignment operator 5.5: Computed Assignment Operators
assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
condexp
example 5.9.1: condexp: Example and Test
conditional
expression 5.9.1: condexp: Example and Test
cos 5.7: Standard Math Unary Functions
cosh 5.7: Standard Math Unary Functions
create
a2float 5.1.d: Create an Object With One Higher Level of AD: a_x
a_float 5.1.d: Create an Object With One Higher Level of AD: a_x
D
debugging
build 2.f.a: Installing pycppad: Building.With Debugging
dependent
variables 6.2: Create an AD Function Object
derivative
domain direction 6.4: Forward Mode: Derivative in One Domain Direction
entire driver 6.6: Driver for Computing Entire Derivative
entire example 6.6.1: Entire Derivative: Example and Test
ODE solution 8.2: runge_kutta_4 An AD Example and Test
range direction 6.5: Reverse Mode: Derivative in One Range Direction
direction
domain derivative 6.4: Forward Mode: Derivative in One Domain Direction
range derivative 6.5: Reverse Mode: Derivative in One Range Direction
distutils
python bug 2.f.c: Installing pycppad: Building.Bug in Python Distutils
divide 5.4: Binary Numeric Operators With an AD Result
domain
direction derivative 6.4: Forward Mode: Derivative in One Domain Direction
driver
entire derivative 6.6: Driver for Computing Entire Derivative
hessian 6.7: Driver for Computing Hessian in a Range Direction
E
entire
derivative driver 6.6: Driver for Computing Entire Derivative
derivative example 6.6.1: Entire Derivative: Example and Test
example
!= 5.6.1: a_float Comparison Operators: Example and Test
* 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
** 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
*= 5.5.1: Computed Assignment Operators: Example and Test
+ 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
+ 5.3.1: Unary Plus and Minus Operators: Example and Test
+= 5.5.1: Computed Assignment Operators: Example and Test
- 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
- 5.3.1: Unary Plus and Minus Operators: Example and Test
-= 5.5.1: Computed Assignment Operators: Example and Test
/ 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
/= 5.5.1: Computed Assignment Operators: Example and Test
< 5.6.1: a_float Comparison Operators: Example and Test
<= 5.6.1: a_float Comparison Operators: Example and Test
== 5.6.1: a_float Comparison Operators: Example and Test
> 5.6.1: a_float Comparison Operators: Example and Test
>= 5.6.1: a_float Comparison Operators: Example and Test
abort_recording 6.3.1: abort_recording: Example and Test
abs 5.8.1: abs: Example and Test
bool a_float binary operator 5.6.1: a_float Comparison Operators: Example and Test
computed assignment operator 5.5.1: Computed Assignment Operators: Example and Test
condexp 5.9.1: condexp: Example and Test
entire derivative 6.6.1: Entire Derivative: Example and Test
future / 5.4.2: Future Division Operator: Example and Test
get_started 3: get_started: Example and Test
hessian 6.7.1: Hessian Driver: Example and Test
independent 6.1.1: independent: Example and Test
list 4: List of All the pycppad Examples
numeric binary operator 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
optimize 6.8.1: Optimize Function Object: Example and Test
order one forward 6.4.2: Forward Order One: Example and Test
order one reverse 6.5.1: Reverse Order One: Example and Test
order two reverse 6.5.2: Reverse Order Two: Example and Test
order zero forward 6.4.1: Forward Order Zero: Example and Test
runge_kutta_8.3: runge_kutta_4 With C++ Speed: Example and Test
runge_kutta_8.2: runge_kutta_4 An AD Example and Test
runge_kutta_8.1: runge_kutta_4 A Correctness Example and Test
standard math 5.7.1: Standard Math Unary Functions: Example and Test
value 5.2.1: value: Example and Test
exp 5.7: Standard Math Unary Functions
exponentiation 5.4: Binary Numeric Operators With an AD Result
extract
F
feature
version 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
fix
bug 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
forward 6.4: Forward Mode: Derivative in One Domain Direction
order one example 6.4.2: Forward Order One: Example and Test
order zero example 6.4.1: Forward Order Zero: Example and Test
future /
example 5.4.2: Future Division Operator: Example and Test
G
get_started
example 3: get_started: Example and Test
H
hessian
driver 6.7: Driver for Computing Hessian in a Range Direction
example 6.7.1: Hessian Driver: Example and Test
Lagrangian 6.7: Driver for Computing Hessian in a Range Direction
I
independent
abort recording 6.3: Abort a Recording of AD Operations
example 6.1.1: independent: Example and Test
variables 6.1.a: Create an Independent Variable Vector: Syntax
install
J
jacobian 6.6: Driver for Computing Entire Derivative
example 6.6.1: Entire Derivative: Example and Test
L
Lagrangian
hessian 6.7: Driver for Computing Hessian in a Range Direction
level
decrease AD 5.2: Create an Object With One Lower Level of AD
increase AD 5.1: Create an Object With One Higher Level of AD
levels
list
example 4: List of All the pycppad Examples
log 5.7: Standard Math Unary Functions
log10 5.7: Standard Math Unary Functions
M
math
example standard 5.7.1: Standard Math Unary Functions: Example and Test
memory
optimize 6.8: Optimize an AD Function Object Tape
minus
binary 5.4: Binary Numeric Operators With an AD Result
unary 5.3: Unary Plus and Minus Operators
N
numeric
binary operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
O
ODE solution
correct 8.1: runge_kutta_4 A Correctness Example and Test
derivative 8.2: runge_kutta_4 An AD Example and Test
ODE solver
Runge-Kutta 8: Fourth Order Runge Kutta
one
order forward example 6.4.2: Forward Order One: Example and Test
order reverse example 6.5.1: Reverse Order One: Example and Test
operation
optimize sequence 6.8.1: Optimize Function Object: Example and Test
operations
optimize sequence 6.8: Optimize an AD Function Object Tape
operator
AD unary 5.3: Unary Plus and Minus Operators
bool a_float binary example 5.6.1: a_float Comparison Operators: Example and Test
bool binary 5.6: Binary Comparison Operators
computed assignment 5.5: Computed Assignment Operators
computed assignment example 5.5.1: Computed Assignment Operators: Example and Test
numeric binary example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
optimize 6.8: Optimize an AD Function Object Tape
example 6.8.1: Optimize Function Object: Example and Test
order
forward one example 6.4.2: Forward Order One: Example and Test
forward zero example 6.4.1: Forward Order Zero: Example and Test
reverse one example 6.5.1: Reverse Order One: Example and Test
reverse two example 6.5.2: Reverse Order Two: Example and Test
P
path
python 2.i: Installing pycppad: Python Path
plus
binary 5.4: Binary Numeric Operators With an AD Result
unary 5.3: Unary Plus and Minus Operators
python
bug distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
path 2.i: Installing pycppad: Python Path
R
Runge-Kutta
ODE solver 8: Fourth Order Runge Kutta
range
direction derivative 6.5: Reverse Mode: Derivative in One Range Direction
recording
start 6.1.a: Create an Independent Variable Vector: Syntax
stop 6.2: Create an AD Function Object
reverse 6.5: Reverse Mode: Derivative in One Range Direction
order one example 6.5.1: Reverse Order One: Example and Test
order two example 6.5.2: Reverse Order Two: Example and Test
runge_kutta_4
correct 8.1: runge_kutta_4 A Correctness Example and Test
evaluate solution 8.3: runge_kutta_4 With C++ Speed: Example and Test
example 8.2: runge_kutta_4 An AD Example and Test
S
sequence
optimize operation 6.8.1: Optimize Function Object: Example and Test
optimize operations 6.8: Optimize an AD Function Object Tape
sin 5.7: Standard Math Unary Functions
sinh 5.7: Standard Math Unary Functions
speed
optimize 6.8: Optimize an AD Function Object Tape
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
sqrt 5.7: Standard Math Unary Functions
standard
example math 5.7.1: Standard Math Unary Functions: Example and Test
start
recording 6.1.a: Create an Independent Variable Vector: Syntax
stop
recording 6.2: Create an AD Function Object
T
tan 5.7: Standard Math Unary Functions
tanh 5.7: Standard Math Unary Functions
tape
optimize 6.8: Optimize an AD Function Object Tape
python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
test
times 5.4: Binary Numeric Operators With an AD Result
two
order reverse example 6.5.2: Reverse Order Two: Example and Test
U
unary
+ 5.3.1: Unary Plus and Minus Operators: Example and Test
- 5.3.1: Unary Plus and Minus Operators: Example and Test
AD operator 5.3: Unary Plus and Minus Operators
unpack
V
value 5.2: Create an Object With One Lower Level of AD
example 5.2.1: value: Example and Test
variable