\newcommand{\B}[1]{{\bf #1}}  \newcommand{\R}[1]{{\rm #1}}
pycppad-20140710: A Python Algorithm Derivative Package
Manual split into sections    Manual as one web page
Math displayed using Latex    pycppad.htm    _printable.htm
Math displayed using MathML    pycppad.xml    _printable.xml

a: License
10: BSD

b: Syntax
from pycppad import *

c: Purpose
The command above imports a boost::python (http://www.boost.org) interface to the C++ Algorithmic Differentiation (AD) (http://en.wikipedia.org/wiki/Automatic_differentiation) package CppAD (http://www.coin-or.org/CppAD/CppAD/) .

d: Simple Example
 
from pycppad import *
def pycppad_test_get_started() :
  def F(x) :                                   # function to be differentiated
    return exp(-(x[0]**2. + x[1]**2.) / 2.)    # is Gaussian density
  x     = numpy.array( [ 1.,  2.] )
  a_x   = independent(x)
  a_y   = numpy.array( [ F(a_x) ] ) 
  f     = adfun(a_x, a_y)
  J     = f.jacobian(x)                        # J = F'(x)
  assert abs( J[0, 0] + F(x) * x[0] ) < 1e-10  # J[0,0] ~= - F(x) * x[0]
  assert abs( J[0, 1] + F(x) * x[1] ) < 1e-10  # J[0,1] ~= - F(x) * x[1]


e: Features
Operator overloading is used to store the operation sequence corresponding to a python algorithm. The operation sequence can be evaluated to obtain new function values or derivatives of arbitrary order. In addition, multiple levels of AD are supported. This means that AD derivatives can be used in the definition of a function which in turn can be differentiated using AD. See 9: whats_new for a list of recent extensions and bug fixes.

f: Contents
_contents: 1Table of Contents
install: 2Installing pycppad
get_started.py: 3get_started: Example and Test
example: 4List of All the pycppad Examples
ad_variable: 5AD Variable Methods
ad_function: 6AD Function Methods
two_levels.py: 7Using Two Levels of AD: Example and Test
runge_kutta_4: 8Fourth Order Runge Kutta
whats_new: 9Extensions, Bug Fixes, and Changes
license: 10License
_reference: 11Alphabetic Listing of Cross Reference Tags
_index: 12Keyword Index
_external: 13External Internet References

Input File: doc.omh
1: Table of Contents
pycppad-20140710: A Python Algorithm Derivative Package: : pycppad
    Table of Contents: 1: _contents
    Installing pycppad: 2: install
    get_started: Example and Test: 3: get_started.py
    List of All the pycppad Examples: 4: example
    AD Variable Methods: 5: ad_variable
        Create an Object With One Higher Level of AD: 5.1: ad
            ad: Example and Test: 5.1.1: ad.py
        Create an Object With One Lower Level of AD: 5.2: value
            value: Example and Test: 5.2.1: value.py
        Unary Plus and Minus Operators: 5.3: ad_unary
            Unary Plus and Minus Operators: Example and Test: 5.3.1: ad_unary.py
        Binary Numeric Operators With an AD Result: 5.4: ad_numeric
            Binary Numeric Operators With an AD Result: Example and Test: 5.4.1: ad_numeric.py
            Future Division Operator: Example and Test: 5.4.2: future_div_op.py
        Computed Assignment Operators: 5.5: assign_op
            Computed Assignment Operators: Example and Test: 5.5.1: assign_op.py
        Binary Comparison Operators: 5.6: compare_op
            a_float Comparison Operators: Example and Test: 5.6.1: compare_op.py
        Standard Math Unary Functions: 5.7: std_math
            Standard Math Unary Functions: Example and Test: 5.7.1: std_math.py
        Absolute Value Functions: 5.8: abs
            abs: Example and Test: 5.8.1: abs.py
        Conditional Expressions: 5.9: condexp
            condexp: Example and Test: 5.9.1: condexp.py
    AD Function Methods: 6: ad_function
        Create an Independent Variable Vector: 6.1: independent
            independent: Example and Test: 6.1.1: independent.py
        Create an AD Function Object: 6.2: adfun
            adfun: Example and Test: 6.2.1: adfun.py
        Abort a Recording of AD Operations: 6.3: abort_recording
            abort_recording: Example and Test: 6.3.1: abort_recording.py
        Forward Mode: Derivative in One Domain Direction: 6.4: forward
            Forward Order Zero: Example and Test: 6.4.1: forward_0.py
            Forward Order One: Example and Test: 6.4.2: forward_1.py
        Reverse Mode: Derivative in One Range Direction: 6.5: reverse
            Reverse Order One: Example and Test: 6.5.1: reverse_1.py
            Reverse Order Two: Example and Test: 6.5.2: reverse_2.py
        Driver for Computing Entire Derivative: 6.6: jacobian
            Entire Derivative: Example and Test: 6.6.1: jacobian.py
        Driver for Computing Hessian in a Range Direction: 6.7: hessian
            Hessian Driver: Example and Test: 6.7.1: hessian.py
        Optimize an AD Function Object Tape: 6.8: optimize
            Optimize Function Object: Example and Test: 6.8.1: optimize.py
    Using Two Levels of AD: Example and Test: 7: two_levels.py
    Fourth Order Runge Kutta: 8: runge_kutta_4
        runge_kutta_4 A Correctness Example and Test: 8.1: runge_kutta_4_correct.py
        runge_kutta_4 An AD Example and Test: 8.2: runge_kutta_4_ad.py
        runge_kutta_4 With C++ Speed: Example and Test: 8.3: runge_kutta_4_cpp.py
    Extensions, Bug Fixes, and Changes: 9: whats_new
        Extensions, Bug Fixes, and Changes During 2012: 9.1: whats_new_12
        Extensions, Bug Fixes, and Changes During 2011: 9.2: whats_new_11
        Extensions, Bug Fixes, and Changes During 2010: 9.3: whats_new_10
        Extensions, Bug Fixes, and Changes During 2009: 9.4: whats_new_09
    License: 10: license
    Alphabetic Listing of Cross Reference Tags: 11: _reference
    Keyword Index: 12: _index
    External Internet References: 13: _external

2: Installing pycppad

2.a: Other Instructions
If you are using WinPython (http://code.google.com/p/winpython/) , see building pycppad in Windows (http://list.coin-or.org/pipermail/cppad/2013q2/000309.html)

2.b: Requires
This package requires the following other packages:
  1. The Boost Python (development version) must be installed.
  2. The python numpy library must be installed.
  3. The CppAD package.


2.c: Downloading
Download the file pycppad-20140710.tar.gz (http://www.seanet.com/~bradbell/pycppad-20140710.tar.gz) and store it on your machine.

2.d: Unpacking
On unix, you could use the command
 
	tar -xvzf  pycppad-20140710.tar.gz
which would create the directory pycppad-20140710.

2.e: Required Setup Information
The value of the following setup variables, in the file
 
	pycppad-20140710/setup.py
must be set to agree with your system:
 
     # Directory where CppAD include files are located
     # cppad_include_dir        = [ '/usr/include' ]
     cppad_include_dir        = [ os.environ['HOME'] + '/prefix/cppad/include' ]
     #
     # Directory where Boost Python include files are located
     boost_python_include_dir = [ '/usr/include' ]
     #
     # Director whre Boost Python library is located
     # boost_python_lib_dir     = [ '/usr/lib' ] 
     boost_python_lib_dir     = [ '/usr/lib64' ] 
     #
     # Name of the Boost Python library in boost_python_lib_dir.
     # boost_python_lib         = [ 'boost_python-mt' ]
     boost_python_lib         = [ 'boost_python' ]
Note that Boost Python and CppAD must be installed before you can properly set this information.

2.f: Building

2.f.a: With Debugging
Change into the directory pycppad-20140710 and execute the command
 
	./setup.py build_ext --inplace --debug --undef NDEBUG
to compile and link a version of the CppAD extension module with debugging (improved error messaging).

2.f.b: Optimized
You can compile and link an optimized version of the CppAD extension module with the command
 
	./setup.py build_ext --inplace
Note that in the optimized version, certain error checking is not done and improper use of pycppad.cpp may lead to a segmentation fault.

2.f.c: Bug in Python Distutils
When using the GNU C++ compiler gcc, the warning
 
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
will be printed. This is not requested by setup.py but is rather a bug in the Python distutils (http://docs.python.org/distutils/) package.

2.f.d: Cygwin
If python crashes during the build procedure under cygwin, you may need to run the rebaseall program. For example, execute the following steps:
  1. Check which cygwin processes are running using the ps -e command, and shut them all down.
  2. Open an DOS command window, for example, select Start | Run and then enter cmd in as the program to run.
  3. Change into the cygwin bin directory. If you installed cygwin in the default location, the following command will do this:
     
    	cd c:\cygwin\bin
    
  4. Start the ash shell by executing the command
     
    	ash.exe
    
  5. The execute the command
     
    	/usr/bin/rebaseall
    
    This will take a few minutes to execute. When it is done, you can close the command window by executing the command exit twice.


2.g: Testing
You can test of all the 4: examples in the pycppad documentation. Change into the directory pycppad-20140710 and execute the command
 
	python test_example.py
You can run some more tests with the command
     python test_more.py 
with_debugging
where with_debugging is True or False depending on if you built 2.f.a: with debugging .

2.g.a: Error
If you get a message about cppad_ being missing, it is probably because you did not use the --inplace flag when you 2.f: built pycppad.

2.h: Installing
Installing pycppad copies it to the standard location for you system. You may or may not preform this step:

Change into the directory pycppad-20140710 and execute the command
     ./setup.py build_ext --debug --undef NDEBUG install --prefix=
prefix
or
     ./setup.py build_ext install --prefix=
prefix
where prefix is the prefix for the location where you wish to install pycppad. Note that some common choices for prefix are
$HOME if you are not the system administrator for this machine
/usr/local you're administrator and the system has a package manager (like yum, apt-get).
/usr you're the administrator but the system does not have a package manager.

2.h.a: Uninstalling
Unfortunately, python's distutils package does not provide an uninstall command. You can uninstall the pycppad package by removing the entries
     
prefix/lib/pythonmajor.minor/site-packages/pycppad
     
prefix/lib/pythonmajor.minor/site-packages/pycppad-20140710.egg-info
     
prefix/share/doc/pycppad
where major and minor are the major and minor version numbers printed by the command
 
	python --version


2.i: Python Path
If you have not installed pycppad, you will not be able to use the python command
     import pycppad
unless the distribution directory
 
	pycppad-20140710
is in your python path. If you have installed pycppad, the installation directory
     
prefix/lib/pythonmajor.minor/site-packages
must be in you python path (to use the import pycppad command). You can check your python path with the following commands
 
	python
	import sys
	sys.path
If the required directory is not yet there, you could add the directory above to your python path using the command
     sys.path.append('
prefix/lib/pythonmajor.minor/site-packages')
You can avoid having to do this every time you load python by adding the path to your environment variable PYTHONPATH. For example, if you are using the bash shell, you could add the command
     export PYTHONPATH="
prefix/lib/pythonmajor.minor/site-packages"
to your $HOME/.bashrc file.

2.j: pycppad Documentation
The documentation for pycppad starts out in the directory
 
	pycppad-20140710/doc
During the installation process, it is copied to the directory
     
prefix/share/doc/pycppad

Input File: omh/install.omh
3: get_started: Example and Test
For our getting started example, we consider the Gaussian density for two independent random variables  F : \B{R}^2 \rightarrow \B{R} and its partial derivatives :  \[
\begin{array}{rcl}
F(x)                 & = & \exp \left[ - ( x_0^2  + x_1^2 ) / 2. \right] \\
\partial_{x(0)} F(x) & = & - F(x) * x_0  \\
\partial_{x(1)} F(x) & = & - F(x) * x_1
\end{array}
\] 
The following Python code computes these derivatives using pycppad and then checks the results for correctness:
 
from pycppad import *
def pycppad_test_get_started() :
  def F(x) :                                   # function to be differentiated
    return exp(-(x[0]**2. + x[1]**2.) / 2.)    # is Gaussian density
  x     = numpy.array( [ 1.,  2.] )
  a_x   = independent(x)
  a_y   = numpy.array( [ F(a_x) ] ) 
  f     = adfun(a_x, a_y)
  J     = f.jacobian(x)                        # J = F'(x)
  assert abs( J[0, 0] + F(x) * x[0] ) < 1e-10  # J[0,0] ~= - F(x) * x[0]
  assert abs( J[0, 1] + F(x) * x[1] ) < 1e-10  # J[0,1] ~= - F(x) * x[1]

Input File: example/get_started.py
4: List of All the pycppad Examples
6.3.1: abort_recording.py abort_recording: Example and Test
5.8.1: abs.py abs: Example and Test
5.1.1: ad.py ad: Example and Test
6.2.1: adfun.py adfun: Example and Test
5.4.1: ad_numeric.py Binary Numeric Operators With an AD Result: Example and Test
5.3.1: ad_unary.py Unary Plus and Minus Operators: Example and Test
5.5.1: assign_op.py Computed Assignment Operators: Example and Test
5.9.1: condexp.py condexp: Example and Test
5.6.1: compare_op.py a_float Comparison Operators: Example and Test
5.4.2: future_div_op.py Future Division Operator: Example and Test
6.4.1: forward_0.py Forward Order Zero: Example and Test
6.4.2: forward_1.py Forward Order One: Example and Test
3: get_started.py get_started: Example and Test
6.7.1: hessian.py Hessian Driver: Example and Test
6.1.1: independent.py independent: Example and Test
6.6.1: jacobian.py Entire Derivative: Example and Test
6.8.1: optimize.py Optimize Function Object: Example and Test
6.5.1: reverse_1.py Reverse Order One: Example and Test
6.5.2: reverse_2.py Reverse Order Two: Example and Test
8.2: runge_kutta_4_ad.py runge_kutta_4 An AD Example and Test
8.3: runge_kutta_4_cpp.py runge_kutta_4 With C++ Speed: Example and Test
8.1: runge_kutta_4_correct.py runge_kutta_4 A Correctness Example and Test
5.7.1: std_math.py Standard Math Unary Functions: Example and Test
7: two_levels.py Using Two Levels of AD: Example and Test
5.2.1: value.py value: Example and Test

Input File: omh/example.omh
5: AD Variable Methods

5.a: Contents
ad: 5.1Create an Object With One Higher Level of AD
value: 5.2Create an Object With One Lower Level of AD
ad_unary: 5.3Unary Plus and Minus Operators
ad_numeric: 5.4Binary Numeric Operators With an AD Result
assign_op: 5.5Computed Assignment Operators
compare_op: 5.6Binary Comparison Operators
std_math: 5.7Standard Math Unary Functions
abs: 5.8Absolute Value Functions
condexp: 5.9Conditional Expressions

Input File: omh/ad_method.omh
5.1: Create an Object With One Higher Level of AD

5.1.a: Syntax
a_x = ad(x)

5.1.b: Purpose
Creates an AD object a_x that records floating point operations. An 6.2: adfun object can later use this recording to evaluate function values and derivatives. These later evaluations are done using the same type as x (except when x is an instance of int, the later evaluations are done using float operations).

5.1.c: x
The argument x can be an instance of an int (AD level 0), or an instance of float (AD level 0), or an a_float (AD level 1). The argument x may also be a numpy.array with one of the element types listed in the previous sentence.

5.1.d: a_x
If x is an instance of int or float, a_x is an a_float (AD level 1). If x is an a_float, a_x is an a2float (AD level 2). If x is an numpy.array, a_x is also an numpy.array with the same shape as x .

5.1.e: Example
The file 5.1.1: ad.py contains an example and test of this function.
Input File: pycppad/__init__.py
5.1.1: ad: Example and Test
 
from pycppad import *
import numpy
def pycppad_test_ad() :
  x   = 1
  a_x = ad(x)
  a2x = ad(a_x)
  #
  assert type(a_x) == a_float and a_x == x
  assert type(a2x) == a2float and a2x == x
  #
  x   = numpy.array( [ 1 , 2 , 3 ] )
  a_x = ad(x)
  a2x = ad(a_x)
  #
  for i in range( len(a_x) ) :  
    assert type(a_x[i]) == a_float and a_x[i] == x[i]
  for i in range( len(a2x) ) :  
    assert type(a2x[i]) == a2float and a2x[i] == x[i]

Input File: example/ad.py
5.2: Create an Object With One Lower Level of AD

5.2.a: Syntax
x = value(a_x)

5.2.b: Purpose
Returns an object with one lower level of AD recording.

5.2.c: a_x
The argument a_x must be an a_float (AD level 1), or an a2float (AD level 2). The argument a_x may also be a numpy.array with one of the element types listed in the previous sentence.

5.2.d: x
If a_x is an a_float, x is a float (AD level 0). If a_x is an a2float, x is an a_float (AD level 1). If a_x is an numpy.array, x is also an numpy.array with the same shape as a_x .

5.2.e: Example
The file 5.2.1: value.py contains an example and test of this function.
Input File: pycppad/__init__.py
5.2.1: value: Example and Test
 
from pycppad import *
# Example using a_float ------------------------------------------------------
def pycppad_test_value() :
  x   = 2
  a_x = ad(x)
  #
  assert type(value(a_x)) == float   and value(a_x) == x
  #
  x   = numpy.array( [ 1 , 2 , 3 ] )
  a_x = ad(x)
  #
  for i in range( len(a_x) ) :  
    xi = value(a_x[i])
    assert type(xi) == float and xi == x[i]

# Example using a2float ------------------------------------------------------
def pycppad_test_value_a2() :
  x   = 2
  a2x = ad(ad(x))
  #
  assert type(value(a2x)) == a_float and value(a2x) == x
  #
  x   = numpy.array( [ 1 , 2 , 3 ] )
  a2x = ad(ad(x))
  #
  for i in range( len(a2x) ) :  
    a_xi = value(a2x[i])
    assert type(a_xi) == a_float and a_xi == x[i]

Input File: example/value.py
5.3: Unary Plus and Minus Operators

5.3.a: Syntax
y = + x
y = - x

5.3.b: Purpose
The operator + ( - ) above results in z equal to x (minus x ).

5.3.c: Type
The argument x can be a_float or a2float and the result z will have the same type as x .

5.3.d: Arrays
The argument x may be a numpy.array with elements of type a_float or a2float. In this case, the result z is an array with the same shape and element type as x .

5.3.e: Example
The file 5.3.1: ad_unary.py contains an example and test of these functions.
Input File: pycppad/pycppad.cpp
5.3.1: Unary Plus and Minus Operators: Example and Test
 

from pycppad import *
import numpy
# Example using a_float ------------------------------------------------------
def pycppad_test_ad_unary() :
  x       = ad(2.)
  plus_x  = + x
  minus_x = - x
  # test using corresponding unary float operators 
  assert value(plus_x)  == + value(x)
  assert value(minus_x) == - value(x)
  #
  x       = ad( numpy.array( [ 1. , 2. ] ) )
  plus_x  = + x
  minus_x = - x
  # test using corresponding unary float operators 
  assert numpy.all( value(plus_x)  == + value(x) )
  assert numpy.all( value(minus_x) == - value(x) )

# Example using a2float ------------------------------------------------------
def pycppad_test_ad_unary_a2() :
  x       = ad( ad(2.) )
  plus_x  = + x
  minus_x = - x
  # test using corresponding unary a_float operators 
  assert value(plus_x)  == + value(x)
  assert value(minus_x) == - value(x)
  #
  x       = ad( ad( numpy.array( [ 1. , 2. ] ) ) )
  plus_x  = + x
  minus_x = - x
  # test using corresponding unary float operators 
  assert numpy.all( value(plus_x)  == + value(x) )
  assert numpy.all( value(minus_x) == - value(x) )


Input File: example/ad_unary.py
5.4: Binary Numeric Operators With an AD Result

5.4.a: Syntax
z = x op y

5.4.b: Purpose
Sets z to the result of the binary operation defined by op and with x as the left operand and y as the right operand.

5.4.c: op
The possible values for op are
op    Meaning
+ addition
- subtraction
* multiplication
/ division
** exponentiation

5.4.d: Types
The following table lists the possible types for x and y and the corresponding result type for z .
                      
y
 
x           float    a_float   a2float
         
-------------------------------
 float   
-   float    a_float   a2float
a_float  
-  a_float   a_float
a2float  
-  a2float             a2float
The type float does not need to be matched exactly but rather as an instance of float.

5.4.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.4.f: Example
The file 5.4.1: ad_numeric.py contains an example and test of these operations. The file 5.4.2: future_div_op.py contains an example and test of the future division operator (http://legacy.python.org/dev/peps/pep-0238/)
Input File: pycppad/pycppad.cpp
5.4.1: Binary Numeric Operators With an AD Result: Example and Test
 
from pycppad import *
# Example using a_float -----------------------------------------------------
def pycppad_test_ad_numeric() :
  x    = 2.
  y    = 3.
  a_x  = ad(x)
  a_y  = ad(y)
  #
  assert a_x + a_y == x + y
  assert a_x + y   == x + y
  assert x   + a_y == x + y
  #
  assert a_x - a_y == x - y
  assert a_x - y   == x - y
  assert x   - a_y == x - y
  #
  assert a_x * a_y == x * y
  assert a_x * y   == x * y
  assert x   * a_y == x * y
  #
  assert a_x / a_y == x / y
  assert a_x / y   == x / y
  assert x   / a_y == x / y
  #
  assert a_x ** a_y == x ** y
  assert a_x ** y   == x ** y
  assert x   ** a_y == x ** y
  #
# Example using a2float -----------------------------------------------------
def pycppad_test_ad_numeric_a2() :
  x    = 2.
  y    = 3.
  a2x  = ad(ad(x))
  a2y  = ad(ad(y))
  #
  assert a2x + a2y == x + y
  assert a2x + y   == x + y
  assert x   + a2y == x + y
  #
  assert a2x - a2y == x - y
  assert a2x - y   == x - y
  assert x   - a2y == x - y
  #
  assert a2x * a2y == x * y
  assert a2x * y   == x * y
  assert x   * a2y == x * y
  #
  assert a2x / a2y == x / y
  assert a2x / y   == x / y
  assert x   / a2y == x / y
  #
  assert a2x ** a2y == x ** y
  assert a2x ** y   == x ** y
  assert x   ** a2y == x ** y
  #

Input File: example/ad_numeric.py
5.4.2: Future Division Operator: Example and Test
 
# Example using a_float ------------------------------------------------------
from __future__ import division
from pycppad import *
def pycppad_test_future_div_op() :
  x = 2.
  y = 3.
  #
  tmp1 = ad(x)
  tmp2 = ad(y)
  tmp3 = tmp1/tmp2
  tmp4 = x/tmp2
  tmp5 = tmp1/y

  assert tmp3 == x / y
  assert tmp4 == x / y
  assert tmp5 == x / y


Input File: example/future_div_op.py
5.5: Computed Assignment Operators

5.5.a: Syntax
u opx

5.5.b: Purpose
We use y ( z ) to refer to the value of u before (after) the operation. This operation sets z equal to
     
y op x
.

5.5.c: op
The possible values for op are
op    Meaning
+ addition
- subtraction
* multiplication
/ division

5.5.d: Types
The following table lists the possible types for x and y (the value of u before the operation) and the corresponding z (the value of u after the operation).
                      
y
 
x           float    a_float   a2float
         
-------------------------------
 float   
-   float    a_float   a2float
a_float  
-  a_float   a_float
a2float  
-  a2float             a2float
The type float does not need to be matched exactly but rather as an instance of float.

5.5.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.5.f: Example
The file 5.5.1: assign_op.py contains an example and test of these operations.
Input File: pycppad/pycppad.cpp
5.5.1: Computed Assignment Operators: Example and Test
 
# Example using a_float ------------------------------------------------------
from pycppad import *
def pycppad_test_assign_op() :
  x = 2.
  y = 3.
  #
  tmp  = ad(x)
  tmp += ad(y)
  assert tmp == x + y
  tmp  = ad(x)
  tmp += y
  assert tmp == x + y
  #
  tmp  = ad(x)
  tmp -= ad(y)
  assert tmp == x - y
  tmp  = ad(x)
  tmp -= y
  assert tmp == x - y
  #
  tmp  = ad(x)
  tmp *= ad(y)
  assert tmp == x * y
  tmp  = ad(x)
  tmp *= y
  assert tmp == x * y
  #
  tmp  = ad(x)
  tmp /= ad(y)
  assert tmp == x / y
  tmp  = ad(x)
  tmp /= y
  assert tmp == x / y

# Example using a2float ------------------------------------------------------
from pycppad import *
def pycppad_test_assign_op_a2() :
  x = 2.
  y = 3.
  #
  tmp  = ad(ad(x))
  tmp += ad(ad(y))
  assert tmp == x + y
  tmp  = ad(ad(x))
  tmp += y
  assert tmp == x + y
  #
  tmp  = ad(ad(x))
  tmp -= ad(ad(y))
  assert tmp == x - y
  tmp  = ad(ad(x))
  tmp -= y
  assert tmp == x - y
  #
  tmp  = ad(ad(x))
  tmp *= ad(ad(y))
  assert tmp == x * y
  tmp  = ad(ad(x))
  tmp *= y
  assert tmp == x * y
  #
  tmp  = ad(ad(x))
  tmp /= ad(ad(y))
  assert tmp == x / y
  tmp  = ad(ad(x))
  tmp /= y
  assert tmp == x / y

Input File: example/assign_op.py
5.6: Binary Comparison Operators

5.6.a: Syntax
z = x op y

5.6.b: Purpose
Sets z to the result of the binary operation defined by op and with x as the left operand and y as the right operand.

5.6.c: op
The possible values for op are
op    Meaning
> greater than
> less than
>= greater than or equal
<= less than or equal
== equal
!= not equal

5.6.d: Types
The following table lists the possible (yes) and impossible (no) types for x and y . The corresponding result type for z is always bool.
                      
y
 
x           float    a_float   a2float
         
-------------------------------
 float   
-    yes      yes       yes   
a_float  
-    yes      yes       no
a2float  
-    yes      no        yes 
The type float does not need to be matched exactly but rather as an instance of float.

5.6.e: Arrays
Either x or y or both may be a numpy.array with elements that match one of possible type choices above. If both x and y are arrays, they must have the same shape. When either x or y is an array, the result z is an array with the same shape. The type of the elements of z correspond to the table above (when the result type is a float, this only refers to the element types matching as instances).

5.6.f: Example
The file 5.6.1: compare_op.py contains an example and test of these operations.
Input File: pycppad/pycppad.cpp
5.6.1: a_float Comparison Operators: Example and Test
 
from pycppad import *
# Example using a_float ------------------------------------------------------
def pycppad_test_compare_op():
  x = ad(2.)
  y = ad(3.)
  z = ad(2.)
  
  # assert comparisons that should be true
  assert x == x
  assert x == z
  assert x != y
  assert x <= x
  assert x <= z
  assert x <= y
  assert x <  y
  
  # assert comparisons that should be false
  assert not x == y
  assert not x != z
  assert not x != x
  assert not x >= y
  assert not x >  y
# Example using a2float ------------------------------------------------------
def pycppad_test_compare_op_a2():
  x = ad(ad(2.))
  y = ad(ad(3.))
  z = ad(ad(2.))
  
  # assert comparisons that should be true
  assert x == x
  assert x == z
  assert x != y
  assert x <= x
  assert x <= z
  assert x <= y
  assert x <  y
  
  # assert comparisons that should be false
  assert not x == y
  assert not x != z
  assert not x != x
  assert not x >= y
  assert not x >  y

Input File: example/compare_op.py
5.7: Standard Math Unary Functions

5.7.a: Syntax
y = fun(x)

5.7.b: Purpose
Evaluate the standard math function fun where fun has one argument.

5.7.c: x
The argument x can be an instance of float, an a_float, an a2float, or a numpy.array of such objects.

5.7.d: y
If x is an instance of float, y will also be an instance of float. Otherwise y will have the same type as x .

In the case where x is an array, y will the same shape as x and the elements of y will have the same type as the elements of x .

5.7.e: fun
The function fun can be any of the following: arccos, arcsin, arctan, cos, cosh, exp, log, log10, sin, sinh, sqrt, tan, or tanh.

5.7.f: Example
The file 5.7.1: std_math.py contains an example and test of these functions.
Input File: pycppad/pycppad.cpp
5.7.1: Standard Math Unary Functions: Example and Test
 
from pycppad import *
import numpy
import math
# Example using a_float ----------------------------------------------------
def pycppad_test_std_math(): 
  delta = 10. * numpy.finfo(float).eps
  pi    = numpy.pi
  x     = pi / 6
  a_x   = ad(x)

  # all the a_float unary standard math functions
  assert abs( arccos(a_x) - math.acos(x) )  < delta
  assert abs( arcsin(a_x) - math.asin(x) )  < delta
  assert abs( arctan(a_x) - math.atan(x) )  < delta
  assert abs( cos(a_x)    - math.cos(x) )   < delta
  assert abs( cosh(a_x)   - math.cosh(x) )  < delta
  assert abs( exp(a_x)    - math.exp(x) )   < delta
  assert abs( log(a_x)    - math.log(x) )   < delta
  assert abs( log10(a_x)  - math.log10(x) ) < delta
  assert abs( sin(a_x)    - math.sin(x) )   < delta
  assert abs( sinh(a_x)   - math.sinh(x) )  < delta
  assert abs( sqrt(a_x)   - math.sqrt(x) )  < delta
  assert abs( tan(a_x)    - math.tan(x) )   < delta
  assert abs( tanh(a_x)   - math.tanh(x) )  < delta

  # example array and derivative calculation
  n = 5
  x = numpy.array( [2 * pi * j / n for j in range(n) ] )
  a_x = independent(x)
  a_y = sin(a_x)
  f   = adfun(a_x, a_y)
  J   = f.jacobian(x)
  for j in range(n) :
    for k in range(n) :
      if j == k : assert abs( J[j][k] - cos( x[j] ) ) < delta
      else :      assert J[j][k] == 0.

# Example using a2float ----------------------------------------------------
def pycppad_test_std_math_a2(): 
  n     = 10
  delta = 10. * numpy.finfo(float).eps
  pi    = numpy.pi
  x     = pi / 6
  a2x   = ad(ad(x))

  # all the a2float unary standard math functions
  assert abs( arccos(a2x) - math.acos(x) )  < delta
  assert abs( arcsin(a2x) - math.asin(x) )  < delta
  assert abs( arctan(a2x) - math.atan(x) )  < delta
  assert abs( cos(a2x)    - math.cos(x) )   < delta
  assert abs( cosh(a2x)   - math.cosh(x) )  < delta
  assert abs( exp(a2x)    - math.exp(x) )   < delta
  assert abs( log(a2x)    - math.log(x) )   < delta
  assert abs( log10(a2x)  - math.log10(x) ) < delta
  assert abs( sin(a2x)    - math.sin(x) )   < delta
  assert abs( sinh(a2x)   - math.sinh(x) )  < delta
  assert abs( sqrt(a2x)   - math.sqrt(x) )  < delta
  assert abs( tan(a2x)    - math.tan(x) )   < delta
  assert abs( tanh(a2x)   - math.tanh(x) )  < delta

  # example array and derivative calculation
  n = 5
  x   = numpy.array( [2 * pi * j / n for j in range(n) ] )
  a_x = ad(x)
  a2x = independent(a_x)
  a2y = sin(a2x)
  a_f = adfun(a2x, a2y)
  a_J = a_f.jacobian(a_x)
  for j in range(n) :
    for k in range(n) :
      if j == k : assert abs( a_J[j][k] - cos( x[j] ) ) < delta
      else :      assert a_J[j][k] == 0.

Input File: example/std_math.py
5.8: Absolute Value Functions

5.8.a: Syntax
y = abs(x)

5.8.b: Purpose
Sets y equal to the absolute value of  x .

5.8.c: x
The argument x can be an instance of float, an a_float, an a2float, or an numpy.array of such objects.

5.8.d: y
If x is an instance of float, y will also be an instance of float . Otherwise y will have the same type as x .

In the case where x is an array, y will the same shape as x and the elements of y will have the same type as the elements of x .

5.8.e: Derivative
pycppad defines the derivative of the absolute value function by  \[
     \R{abs}^{(1)} (x) = \R{sign} (x) = \left\{ \begin{array}{ll} 
          1 & \R{if} \; x > 0
          \\
          0 & \R{if} \; x = 0
          \\
          -1 & \R{if} \; x < 0
     \end{array} \right.
\] 


5.8.f: Directional Derivative
Prior to 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) , 6.4: forward mode computed the directional derivative of the absolute value function which is defined by  \[
     \R{abs}^\circ ( x , d ) = \lim_{\lambda \downarrow 0 } 
          \frac{\R{abs}(x + \lambda d) - \R{abs}(x) }{ \lambda }
\] 
For  x \neq 0 ,  \[
     \R{abs}^\circ ( x , d ) = \R{abs}^{(1)} ( x ) * d
\] 
and  \R{abs}^\circ (0 , d) = |d| .

5.8.g: Example
The file 5.8.1: abs.py contains an example and test of this function.
Input File: pycppad/pycppad.cpp
5.8.1: abs: Example and Test
 
# Example using a_float ----------------------------------------------------
from pycppad import *
def pycppad_test_abs() :
  x   = numpy.array( [ -1.,  0.,  1.] )
  n   = len(x)
  a_x = independent(x)
  a_y = abs( a_x )
  f   = adfun(a_x, a_y)
  f.forward(0, x)
  dx  = numpy.zeros(n, dtype=float)
  for i in range( n ) :
    dx[i] = 1.
    df    = f.forward(1, dx)
    if x[i] > 0. :
      assert df[i] == +1.
    elif x[i] < 0. :
      assert df[i] == -1.
    else :
      # There was a change in the CppAD specifictions for the abs function
      # see 12-30 on http://www.coin-or.org/CppAD/Doc/whats_new_11.htm
      assert df[i] == +1. or df[i] == 0.
    dx[i] = -1.
    df    = f.forward(1, dx)
    if x[i] > 0. :
      assert df[i] == -1.
    elif x[i] < 0. :
      assert df[i] == +1.
    else :
      assert df[i] == +1 or df[i] == 0.
    dx[i] = 0.
# Example using a2float ----------------------------------------------------
def pycppad_test_abs_a2() :
  x   = ad( numpy.array( [-1,  0,  1] ) )
  n   = len(x)
  a_x = independent(x)
  a_y = abs( a_x )
  f   = adfun(a_x, a_y)
  f.forward(0, x)
  dx  = numpy.array( list( ad(0) for i in range(n) ) )
  for i in range( n ) :
    dx[i] = ad(0)
  for i in range( n ) :
    dx[i] = ad(1)
    df    = f.forward(1, dx)
    if x[i] > 0. :
      assert df[i] == +1.
    elif x[i] < 0. :
      assert df[i] == -1.
    else :
      assert df[i] == +1. or df[i] == 0.
    dx[i] = ad(-1)
    df    = f.forward(1, dx)
    if x[i] > 0. :
      assert df[i] == -1.
    elif x[i] < 0. :
      assert df[i] == +1.
    else:
      assert df[i] == +1. or df[i] == 0.
    dx[i] = ad(0)

Input File: example/abs.py
5.9: Conditional Expressions

5.9.a: Syntax
result = condexp_rel(leftrightif_trueif_false)

5.9.b: Purpose
Record, as part of an operation sequence, the conditional result
     if( 
left op right )
          
result = if_true
     else 
result = if_false
The relation rel% , and operator op , have the following correspondence:
     
rel   lt   le   eq   ge   gt
      
op    <   <=   ==    >   >=

5.9.c: rel
In the syntax above, the relation rel represents one of the following two characters: lt, le, eq, ge, gt. As in the table above, rel determines which comparison operator op is used when comparing left and right .

5.9.d: left
The argument left must have type a_float or a2float. It specifies the value for the left side of the comparison operator.

5.9.e: right
The argument right must have the same type as left . It specifies the value for the right side of the comparison operator.

5.9.f: if_true
The argument if_true must have the same type as left . It specifies the return value if the result of the comparison is true.

5.9.g: if_false
The argument if_false must have the same type as left . It specifies the return value if the result of the comparison is false.

5.9.h: result
This result has the same type as left .

5.9.i: Example
The file 5.9.1: condexp.py contains an example and test of these functions.
Input File: pycppad/pycppad.cpp
5.9.1: condexp: Example and Test
 
# Example using a_float ----------------------------------------------------
from pycppad import *
def pycppad_test_condexp() :
  x          = numpy.array( [1. , 1., 3., 4. ] )
  a_x        = independent(x)
  a_left     = a_x[0];
  a_right    = a_x[1];
  a_if_true  = a_x[2];
  a_if_false = a_x[3]; 
  a_y_lt     = condexp_lt(a_left, a_right, a_if_true, a_if_false);
  a_y_le     = condexp_le(a_left, a_right, a_if_true, a_if_false);
  a_y_eq     = condexp_eq(a_left, a_right, a_if_true, a_if_false);
  a_y_ge     = condexp_ge(a_left, a_right, a_if_true, a_if_false);
  a_y_gt     = condexp_gt(a_left, a_right, a_if_true, a_if_false);
  a_y        = numpy.array( [ a_y_lt, a_y_le, a_y_eq, a_y_ge, a_y_gt ] );
  f          = adfun(a_x, a_y)
  y          = f.forward(0, x)
  assert ( y[0] == 4. )  # 1 <  1 is false so result is 4
  assert ( y[1] == 3. )  # 1 <= 1 is true  so result is 3
  assert ( y[2] == 3. )  # 1 == 1 is true  so result is 3
  assert ( y[3] == 3. )  # 1 >= 1 is true  so result is 3
  assert ( y[4] == 4. )  # 1 >  2 is false so result is 4
  x          = numpy.array( [4., 3., 2., 1.] )
  y          = f.forward(0, x)
  assert ( y[0] == 1. )  # 4 <  3 is false so result is 1
  assert ( y[1] == 1. )  # 4 <= 3 is false so result is 1
  assert ( y[2] == 1. )  # 4 == 3 is false so result is 1
  assert ( y[3] == 2. )  # 4 >= 3 is true  so result is 2
  assert ( y[4] == 2. )  # 4 >  3 is true  so result is 2
# Example using a2float ----------------------------------------------------
def pycppad_test_condexp_a2() :
  x          = numpy.array( [1. , 1., 3., 4. ] )
  a_x        = ad(x)
  # begin level two recording of conditional expression
  a2x        = independent(a_x)
  a2left     = a2x[0];
  a2right    = a2x[1];
  a2if_true  = a2x[2];
  a2if_false = a2x[3]; 
  a2y_lt     = condexp_lt(a2left, a2right, a2if_true, a2if_false);
  a2y_le     = condexp_le(a2left, a2right, a2if_true, a2if_false);
  a2y_eq     = condexp_eq(a2left, a2right, a2if_true, a2if_false);
  a2y_ge     = condexp_ge(a2left, a2right, a2if_true, a2if_false);
  a2y_gt     = condexp_gt(a2left, a2right, a2if_true, a2if_false);
  a2y        = numpy.array( [ a2y_lt, a2y_le, a2y_eq, a2y_ge, a2y_gt ] );
  a_f        = adfun(a2x, a2y)
  # begin level one recording of conditional expression
  a_x        = independent(x)
  a_y        = a_f.forward(0, a_x)
  f          = adfun(a_x, a_y)
  y          = f.forward(0, x)
  assert ( y[0] == 4. )  # 1 <  1 is false so result is 4
  assert ( y[1] == 3. )  # 1 <= 1 is true  so result is 3
  assert ( y[2] == 3. )  # 1 == 1 is true  so result is 3
  assert ( y[3] == 3. )  # 1 >= 1 is true  so result is 3
  assert ( y[4] == 4. )  # 1 >  2 is false so result is 4
  x          = numpy.array( [4., 3., 2., 1.] )
  y          = f.forward(0, x)
  assert ( y[0] == 1. )  # 4 <  3 is false so result is 1
  assert ( y[1] == 1. )  # 4 <= 3 is false so result is 1
  assert ( y[2] == 1. )  # 4 == 3 is false so result is 1
  assert ( y[3] == 2. )  # 4 >= 3 is true  so result is 2
  assert ( y[4] == 2. )  # 4 >  3 is true  so result is 2

Input File: example/condexp.py
6: AD Function Methods

6.a: Contents
independent: 6.1Create an Independent Variable Vector
adfun: 6.2Create an AD Function Object
abort_recording: 6.3Abort a Recording of AD Operations
forward: 6.4Forward Mode: Derivative in One Domain Direction
reverse: 6.5Reverse Mode: Derivative in One Range Direction
jacobian: 6.6Driver for Computing Entire Derivative
hessian: 6.7Driver for Computing Hessian in a Range Direction
optimize: 6.8Optimize an AD Function Object Tape

Input File: omh/ad_method.omh
6.1: Create an Independent Variable Vector

6.1.a: Syntax
a_x = independent(x)

6.1.b: Purpose
Creates an independent variable vector and starts recording operations involving objects that are instances of type(a_x[0]) . You must create an 6.2: adfun object, or use 6.3: abort_recording , to stop the recording before making another call to independent,

6.1.c: x
The argument x must be a numpy.array with one dimension (i.e., a vector). All the elements of x must all be of the same type and instances of either int, float or a_float.

6.1.d: a_x
The return value a_x is a numpy.array with the same shape as x . If the elements of x are instances of int or float the elements of a_x are instances of a_float. If the elements of x are instances of a_float the elements of a_x are instances of a2float. The 5.2: value of the elements of a_x are equal to the corresponding elements of x .

6.1.e: Example
The file 6.1.1: independent.py contains an example and test of this function.
Input File: pycppad/adfun.py
6.1.1: independent: Example and Test
 
from pycppad import *
# Example using a_float ---------------------------------------------------
def pycppad_test_independent() :
  x   = numpy.array( [ 0., 0., 0. ] )
  a_x = independent(x)    # level 1 independent variables and start recording
  assert type(a_x) == numpy.ndarray
  for j in range(len(x)) :
    assert isinstance(x[j],   float)
    assert isinstance(a_x[j], a_float)
    assert a_x[j] == x[j]
  f   = adfun(a_x, a_x)   # stop level 1 recording
# Example using a2float ---------------------------------------------------
def pycppad_test_independent_a2() :
  x   = numpy.array( [ 0., 0., 0. ] )
  a_x = independent(x)    # level 1 independent variables and start recording
  a2x = independent(a_x)  # level 2 independent variables and start recording
  assert type(a_x) == numpy.ndarray
  assert type(a2x) == numpy.ndarray
  for j in range(len(x)) :
    assert isinstance(x[j],   float)
    assert isinstance(a_x[j], a_float)
    assert isinstance(a2x[j], a2float)
    assert a_x[j] == x[j]
    assert a2x[j] == x[j]
  a_f = adfun(a2x, a2x)   # stop level 2 recording
  f   = adfun(a_x, a_x)   # stop level 1 recording

Input File: example/independent.py
6.2: Create an AD Function Object

6.2.a: Syntax
f = adfun(a_xa_y)

6.2.b: Purpose
The function object f will store the type( a_x[0] ) operation sequence that mapped the independent variable vector a_x to the dependent variable vector a_y .

6.2.c: a_x
The argument a_x is the numpy.array returned by the previous call to 6.1: independent . Neither the size of a_x , or the value it its elements, may change between calling
     
a_x = independent(x)
and
     
f = adfun(a_xa_y)
The length of the vector a_x determines the domain size  n for the function  y = F(x) below.

6.2.d: a_y
The argument a_y specifies the dependent variables. It must be a numpy.array with one dimension (i.e., a vector) and with the same type of elements as a_x . The object f stores the type( a_x[0] ) operations that mapped the vector a_x to the vector a_y . The length of the vector a_y determines the range size  m for the function  y = F(x) below.

6.2.e: f
The return value f can be used to evaluate the function  \[
     F : \B{R}^n \rightarrow \B{R}^m
\] 
and its derivatives, where  y = F(x) corresponds to the operation sequence mentioned above.

6.2.e.a: m
The range size  m is equal to the length of the vector a_y .

6.2.e.b: n
The domain size  n is equal to the length of the vector a_x .

6.2.e.c: level
The 5.1: ad level for the object f is one less than the AD level for the arguments a_x and a_y ; i.e., if type( a_x[0] ) is a_float (a2float) the corresponding AD level for f is zero (one).

6.2.f: Example
The file 6.2.1: adfun.py contains an example and test of this function.
Input File: pycppad/adfun.py
6.2.1: adfun: Example and Test
 
from pycppad import *
def pycppad_test_adfun() :
  # record operations at x = (0, 0, 0)
  x    = numpy.array( [ 0., 0., 0. ] )
  a_x  = independent(x)   # declare independent variables and start recording
  a_y0 = a_x[0];
  a_y1 = a_x[0] * a_x[1];
  a_y2 = a_x[0] * a_x[1] * a_x[2];
  a_y  = numpy.array( [ a_y0, a_y1, a_y2 ] )
  f    = adfun(a_x, a_y)  # declare dependent variables and stop recording
  # evaluate function at x = (1, 2, 3)
  x    = numpy.array( [ 1., 2., 3. ] ) 
  y    = f.forward(0, x) 
  assert y[0] == x[0]
  assert y[1] == x[0] * x[1]
  assert y[2] == x[0] * x[1] * x[2]

Input File: example/adfun.py
6.3: Abort a Recording of AD Operations

6.3.a: Syntax
abort_recording()

6.3.b: Purpose
Sometimes it is necessary to abort the recording of AD operations that started with a call of the form
     
a_x = independent(x)
If such a recording is currently in progress, this will stop the recording and delete the corresponding information. Otherwise, abort_recording has no effect.

6.3.c: Example
The file 6.3.1: abort_recording.py contains an example and test of this operation. It returns true if it succeeds and false otherwise.
Input File: pycppad/adfun.py
6.3.1: abort_recording: Example and Test
 
from pycppad import *
# Example using a_float ---------------------------------------------------
def pycppad_test_abort_recording() :
	from numpy import array
	try :
		x    = numpy.array( [ 1., 2., 3. ] )
		a_x  = independent(x)    # start first level recording
		a2_x = independent(a_x)  # start second level recording
		a_y  = array([sum(a_x)]) # record some operations
		if a_y[0] > 2 :
			raise ValueError
	except ValueError :
		# Pretend that we are not sure if there are any active recordings
		# and use this call to terminate any that may exist.
		abort_recording()

	a_x  = independent(x)     # test starting a level 1 recording
	a2_x = independent(a_x)   # test starting a level 2 recording
	a_y  = array([sum(a_x)])  # record some level 1 operations
	f    = adfun(a_x, a_y)    # terminate level 1 recording
	y    = f.forward(0, x)    # evaluate the function at original x value 
	assert( y[0] == 6. )      # check the value
	abort_recording()         # abort the level 2 recording


Input File: example/abort_recording.py
6.4: Forward Mode: Derivative in One Domain Direction

6.4.a: Syntax
y_p = f.forward(px_p)

6.4.b: Purpose
We use  F : \B{R}^n \rightarrow \B{R}^m to denote the function corresponding to the adfun object 6.2.e: f . Given the p-th order Taylor expansion for a function  X : \B{R} \rightarrow \B{R}^n , this function can be used to compute the p-th order Taylor expansion for the function  Y : \B{R} \rightarrow \B{R}^m defined by  \[
     Y(t) = F [ X(t) ]
\] 


6.4.c: x_k
For  k = 0 , \ldots , p , we use  x^{(k)} to denote the value of x_k in the most recent call to
     
f.forward(kx_k)
including  x^{(p)} as the value x_p in this call. We define the function  X(t) by  \[
     X(t) =  x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p 
\] 


6.4.d: y_k
For  k = 0 , \ldots , p , we use  y^{(k)} to denote the Taylor coefficients for  Y(t) = F[ X(t) ] expanded about zero; i.e.,  \[
\begin{array}{rcl}
y^{(k)} & = & Y^{(k)} (0) / k !
\\
Y(t)    & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + o( t^p )
\end{array}
\] 
where  o( t^p ) / t^p \rightarrow 0 as  t \rightarrow 0 . The coefficient  y^{(p)} is equal to the value y_p returned by this call.

6.4.e: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.4.f: p
The argument p is a non-negative int. It specifies the order of the Taylor coefficient for  Y(t) that is computed.

6.4.g: x_p
The argument x_p is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the p-th order Taylor coefficient for  X(t) . If the AD 6.2.e.c: level for f is zero, all the elements of x_p must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x_p must be a_float objects.

6.4.h: y_p
The return value y_p is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It is set to the p-th order Taylor coefficient for  Y(t) . If the AD 6.2.e.c: level for f is zero, all the elements of y_p will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of y_p will be a_float objects.

6.4.i: Example
6.4.1: forward_0.py Forward Order Zero: Example and Test
6.4.2: forward_1.py Forward Order One: Example and Test

Input File: pycppad/adfun.cpp
6.4.1: Forward Order Zero: Example and Test
 
from pycppad import *
# Example using a_float ----------------------------------------------------
def pycppad_test_forward_0() :

  # start record a_float operations
  x   = numpy.array( [ 2., 3. ] )  # value of independent variables
  a_x = independent(x)             # declare a_float independent variables

  # stop recording and store operations in the function object f
  a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables
  f   = adfun(a_x, a_y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p  = 0                                 # order zero for function values
  x  = numpy.array( [ 3. , 4. ] )        # argument value
  fp = f.forward(p, x)                   # function value
  assert fp[0] == 2. * x[0] * x[1]       # f(x0, x1) = 2 * x0 * x1

# Example using a2float ----------------------------------------------------
def pycppad_test_forward_0_a2() :

  # start record a_float operations
  a_x = ad(numpy.array( [ 2., 3. ] )) # a_float value of independent variables
  a2x = independent(a_x)              # declare a2float independent variables

  # stop recording and store operations in the function object f
  a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables
  a_f = adfun(a2x, a2y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p  = 0                                  # order zero for function values
  a_x  = ad( numpy.array( [ 3. , 4. ] ) ) # argument value
  a_fp = a_f.forward(p, a_x)              # function value
  assert a_fp[0] == 2. * a_x[0] * a_x[1]  # f(x0, x1) = 2 * x0 * x1


Input File: example/forward_0.py
6.4.2: Forward Order One: Example and Test
 
from pycppad import *

# Example using a_float -----------------------------------------------------
def pycppad_test_forward_1() :

  # start record a_float operations
  x   = numpy.array( [ 2., 3. ] )  # value of independent variables
  a_x = independent(x)             # declare independent variables

  # stop recording and store operations in the function object f
  a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables
  f   = adfun(a_x, a_y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p  = 0                                 # order zero for function values
  x  = numpy.array( [ 3. , 4. ] )        # argument value
  fp = f.forward(p, x)                   # function value
  assert fp[0] == 2. * x[0] * x[1]       # f(x0, x1) = 2 * x0 * x1

  # evalute partial derivative of f(x0, x1) with respect to x0
  p  = 1                                 # order one for first derivatives
  xp = numpy.array( [ 1. , 0. ] )        # direction for differentiation
  fp = f.forward(p, xp)                  # value of directional derivative
  assert fp[0] == 2. * x[1]              # f_x0 (x0, x1) = 2 * x1

  # evalute partial derivative of f(x0, x1) with respect to x
  p  = 1
  xp = numpy.array( [ 0. , 1. ] )        # the x1 direction
  fp = f.forward(p, xp)
  assert fp[0] == 2. * x[0]              # f_x1 (x0, x1) = 2 * x0

# Example using a2float -----------------------------------------------------
def pycppad_test_forward_1_a2() :

  # start record a_float operations
  a_x = ad(numpy.array( [ 2., 3. ] ))  # a_float value of independent variables
  a2x = independent(a_x)               # declare a2float independent variables

  # stop recording and store operations in the function object f
  a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables
  a_f = adfun(a2x, a2y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p    = 0                               # order zero for function values
  a_x  = ad(numpy.array( [ 3. , 4. ] ))  # argument value
  a_fp = a_f.forward(p, a_x)             # function value
  assert a_fp[0] == 2. * a_x[0] * a_x[1] # f(x0, x1) = 2 * x0 * x1

  # evalute partial derivative of f(x0, x1) with respect to x0
  p  = 1                                 # order one for first derivatives
  a_xp = ad(numpy.array( [ 1. , 0. ] ))  # direction for differentiation
  a_fp = a_f.forward(p, a_xp)            # value of directional derivative
  assert a_fp[0] == 2. * a_x[1]          # f_x0 (x0, x1) = 2 * x1

  # evalute partial derivative of f(x0, x1) with respect to x
  p  = 1
  a_xp = ad(numpy.array( [ 0. , 1. ] ))  # the x1 direction
  a_fp = a_f.forward(p, a_xp)
  assert a_fp[0] == 2. * a_x[0]          # f_x1 (x0, x1) = 2 * x0


Input File: example/forward_1.py
6.5: Reverse Mode: Derivative in One Range Direction

6.5.a: Syntax
dw = f.forward(pw)

6.5.b: Purpose
Reverse mode computes the derivative of the 6.4: forward more Taylor coefficients with respect to the domain variable  x .

6.5.c: x_k
For  k = 0 , \ldots , p , we use  x^{(k)} to denote the value of x_k in the most recent call to
     
f.forward(kx_k)
We use  F : \B{R}^n \rightarrow \B{R}^m to denote the function corresponding to the adfun object 6.2.e: f .

6.5.d: X(t, u)
We define the function  X : \B{R} \times \B{R}^n \rightarrow \B{R}^n by  \[
     X(t, u) =  u + x^{(0)} + x^{(1)} * t + \cdots + x^{(p-1)} * t^{p-1} 
\] 
Note that for  k = 0 , \ldots , p - 1 ,  \[
     x^{(k)} = \frac{1}{k !} \frac{\partial^k}{\partial t^k} X(0, 0)
\] 


6.5.e: W(t, u)
The function  W : \B{R} \times \B{R}^n \rightarrow \B{R} is defined by  \[
W(t, u) = w_0 * F_0 [ X(t, u) ] + \cdots + w_{m-1} * F_{m-1} [ X(t, u) ]
\] 
We define the function  W_k : \B{R}^n \rightarrow \B{R} by  \[
     W_k ( u ) = \frac{1}{k !} \frac{\partial^k}{\partial t^k} W(0, u)
\] 
It follows that  \[
W(t, u ) = W_0 ( u ) + W_1 ( u ) * t + \cdots + W_{p-1} (u) * t^{p-1}
         + o( t^{p-1} )
\] 
where  o( t^{p-1} ) / t^{p-1} \rightarrow 0 as  t \rightarrow 0 .

6.5.f: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.5.g: p
The argument p is a non-negative int. It specifies the order of the Taylor coefficient  W_{p-1} ( u ) that is differentiated. Note that  W_{p-1} (u) corresponds a derivative of order  p-1 of  F(x) , so the derivative of  W_{p-1} (u) corresponds to a derivative of order  p of  F(x) .

6.5.h: w
The argument w is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It specifies the weighting vector  w used in the definition of  W(t, u) . If the AD 6.2.e.c: level for f is zero, all the elements of w must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of w must be a_float objects.

6.5.i: dw
The return value v is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It is set to the derivative  \[
\begin{array}{rcl}
dw & = & W_{p-1}^{(1)} ( 0 ) \\
& = &
\partial_u \frac{1}{(p-1) !} \frac{\partial^{p-1}}{\partial t^{p-1}} W(0, 0)
\end{array}
\] 
If the AD 6.2.e.c: level for f is zero, all the elements of dw will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of dw will be a_float objects.

6.5.j: First Order
In the case where  p = 1 , we have  \[
\begin{array}{rcl}
dw 
& = & \partial_u \frac{1}{0 !} \frac{\partial^0}{\partial t^0} W(0, 0)
\\
& = & \partial_u W(0, 0)
\\
& = & 
\partial_u \left[ 
w_0 * F_0 ( u + x^{(0)} ) + \cdots +  w_{m-1} F_{m-1} ( u + x^{(0)} )
\right]_{u = 0}
\\
& = & 
w_0 * F_0^{(1)} ( x^{(0)} ) + \cdots + w_{m-1} * F_{m-1}^{(1)} ( x^{(0)} )
\end{array}
\] 


6.5.k: Second Order
In the case where  p = 2 , we have  \[
\begin{array}{rcl}
dw 
& = & \partial_u \frac{1}{1 !} \frac{\partial^1}{\partial t^1} W (0, 0)
\\
& = &
\partial_u \left[
     w_0 * F_0^{(1)} ( u + x^{(0)} ) * x^{(1)}
     + \cdots + 
     w_{m-1} * F_{m-1}^{(1)} ( u + x^{(0)} ) * x^{(1)}
\right]_{u = 0}
\\
& = &
w_0 * ( x^{(1)} )^\R{T} * F_0^{(2)} ( x^{(0)} ) 
+ \cdots + 
w_{m-1} * ( x^{(1)} )^\R{T} * F_{m-1}^{(2)} ( x^{(0)} )
\end{array}
\] 


6.5.l: Example
6.5.1: reverse_1.py Reverse Order One: Example and Test
6.5.2: reverse_2.py Reverse Order Two: Example and Test

Input File: pycppad/adfun.cpp
6.5.1: Reverse Order One: Example and Test
 
from pycppad import *
# Example using a_float ------------------------------------------------------
def pycppad_test_reverse_1():

  # start record a_float operations
  x   = numpy.array( [ 2. , 3. ] )  # value of independent variables
  a_x = independent(x)              # declare a_float independent variables

  # stop recording and store operations in the function object f
  a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables
  f   = adfun(a_x, a_y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p  = 0                                 # order zero for function values
  x  = numpy.array( [ 3. , 4. ] )        # argument value
  fp = f.forward(p, x)                   # function value
  assert fp[0] == 2. * x[0] * x[1]       # f(x0, x1) = 2 * x0 * x1

  # evalute derivative of f(x0, x1) 
  p  = 1                                 # order one for first derivatives
  w  = numpy.array( [ 1. ] )             # weight in range space
  fp = f.reverse(p, w)                   # derivaitive of weighted function
  assert fp[0] == 2. * x[1]              # f_x0 (x0, x1) = 2 * x1
  assert fp[1] == 2. * x[0]              # f_x1 (x0, x1) = 2 * x0

# Example using a2float ------------------------------------------------------
def pycppad_test_reverse_1_a2():

  # start record a_float operations
  a_x = ad(numpy.array( [ 2. , 3. ] ))  # value of independent variables
  a2x = independent(a_x)                # declare a2float independent variables

  # stop recording and store operations in the function object f
  a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables
  a_f = adfun(a2x, a2y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at a different argument value
  p   = 0                                 # order zero for function values
  a_x = ad(numpy.array( [ 3. , 4. ] ))    # argument value
  a_fp = a_f.forward(p, a_x)              # function value
  assert a_fp[0] == 2. * a_x[0] * a_x[1]  # f(x0, x1) = 2 * x0 * x1

  # evalute derivative of f(x0, x1) 
  p  = 1                                 # order one for first derivatives
  a_w  = ad(numpy.array( [ 1. ] ))       # weight in range space
  a_fp = a_f.reverse(p, a_w)             # derivaitive of weighted function
  assert a_fp[0] == 2. * a_x[1]          # f_x0 (x0, x1) = 2 * x1
  assert a_fp[1] == 2. * a_x[0]          # f_x1 (x0, x1) = 2 * x0

Input File: example/reverse_1.py
6.5.2: Reverse Order Two: Example and Test
 
from pycppad import *
# Example using a_float ------------------------------------------------------
def pycppad_test_reverse_2():

  # start record a_float operations
  x   = numpy.array( [ 2. , 3. ] )  # value of independent variables
  a_x = independent(x)              # declare a_float independent variables

  # stop recording and store operations in the function object f
  a_y = numpy.array( [ 2. * a_x[0] * a_x[1] ] ) # dependent variables
  f   = adfun(a_x, a_y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at same argument value
  p   = 0                           # derivative order
  x_p = x                           # zero order Taylor coefficient
  f_p = f.forward(0, x_p)           # function value
  assert f_p[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1

  # evalute partial derivative with respect to x[0]
  p  = 1                            # derivative order
  x_p = numpy.array( [ 1. , 0 ] )   # first order Taylor coefficient
  f_p = f.forward(1, x_p)           # partial w.r.t. x0
  assert f_p[0] == 2. * x[1]        # f_x0 (x0, x1) = 2 * x1
 
  # evaluate derivative of partial w.r.t. x[0]
  p  = 2                            # derivative order
  w  = numpy.array( [1.] )          # weighting vector
  dw = f.reverse(p, w)              # derivaitive of weighted function
  assert dw[0] == 0.                # f_x0_x1 (x0, x1) = 0 
  assert dw[1] == 2.                # f_x0_x1 (x0, x1) = 2 

# Example using a2float ------------------------------------------------------
def pycppad_test_reverse_2_a2():

  # start record a_float operations
  x   = numpy.array( [ 2. , 3. ] )  # value of independent variables
  a_x = ad(x)                       # value of independent variables
  a2x = independent(a_x)            # declare a2float independent variables

  # stop recording and store operations in the function object f
  a2y = numpy.array( [ 2. * a2x[0] * a2x[1] ] ) # dependent variables
  a_f = adfun(a2x, a2y)                         # f(x0, x1) = 2 * x0 * x1

  # evaluate the function at same argument value
  p   = 0                           # derivative order
  x_p = a_x                         # zero order Taylor coefficient
  f_p = a_f.forward(0, x_p)         # function value
  assert f_p[0] == 2. * x[0] * x[1] # f(x0, x1) = 2 * x0 * x1

  # evalute partial derivative with respect to x[0]
  p  = 1                            # derivative order
  x_p = ad(numpy.array([1. , 0 ]))  # first order Taylor coefficient
  f_p = a_f.forward(1, x_p)         # partial w.r.t. x0
  assert f_p[0] == 2. * x[1]        # f_x0 (x0, x1) = 2 * x1
 
  # evaluate derivative of partial w.r.t. x[0]
  p  = 2                            # derivative order
  w  = ad(numpy.array( [1.] ))      # weighting vector
  dw = a_f.reverse(p, w)            # derivaitive of weighted function
  assert dw[0] == 0.                # f_x0_x1 (x0, x1) = 0 
  assert dw[1] == 2.                # f_x0_x1 (x0, x1) = 2 


Input File: example/reverse_2.py
6.6: Driver for Computing Entire Derivative

6.6.a: Syntax
J = f.jacobian(x)

6.6.b: Purpose
This routine computes the entire derivative  F^{(1)} (x) where  F : \B{R}^n \rightarrow \B{R}^m is the function corresponding to the adfun object 6.2.e: f .

6.6.c: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.6.d: x
The argument x is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of x must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x must be a_float objects.

6.6.e: J
The return value J is a numpy.array with two dimensions (i.e., a matrix). The first dimension (row size) is equal to 6.2.e.a: m (the number of range components in the function f ). The second dimension (column size) is equal to 6.2.e.b: n (the number of domain components in the function f ). It is set to the derivative; i.e.,  \[
     J = F^{(1)} (x)
\] 
If the AD 6.2.e.c: level for f is zero, all the elements of J will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of J will be a_float objects.

6.6.f: Example
The file 6.6.1: jacobian.py contains an example and test of this operation.
Input File: pycppad/adfun.cpp
6.6.1: Entire Derivative: Example and Test
 
from pycppad import *
# Example using a_float -----------------------------------------------------
def pycppad_test_jacobian():
  delta = 10. * numpy.finfo(float).eps
  x     = numpy.array( [ 0., 0. ] )
  a_x   = independent(x)
  a_y   = numpy.array( [ 
    a_x[0] * exp(a_x[1]) , 
    a_x[0] * sin(a_x[1]) ,
    a_x[0] * cos(a_x[1]) 
  ] )
  f   = adfun(a_x, a_y)
  x   = numpy.array( [ 2., 3. ] )
  J   = f.jacobian(x)
  assert abs( J[0,0] -        exp(x[1]) ) < delta
  assert abs( J[0,1] - x[0] * exp(x[1]) ) < delta
  assert abs( J[1,0] -        sin(x[1]) ) < delta
  assert abs( J[1,1] - x[0] * cos(x[1]) ) < delta
  assert abs( J[2,0] -        cos(x[1]) ) < delta
  assert abs( J[2,1] + x[0] * sin(x[1]) ) < delta
# Example using a2float -----------------------------------------------------
def pycppad_test_jacobian_a2():
  delta = 10. * numpy.finfo(float).eps
  a_x   = ad( numpy.array( [ 0., 0. ] ) )
  a2x   = independent(a_x)
  a2y   = numpy.array( [ 
    a2x[0] * exp(a2x[1]) , 
    a2x[0] * sin(a2x[1]) ,
    a2x[0] * cos(a2x[1]) 
  ] )
  a_f   = adfun(a2x, a2y)
  x     = numpy.array( [2., 3.] )
  a_x   = ad(x)
  a_J   = a_f.jacobian(a_x)
  assert abs( a_J[0,0] -        exp(x[1]) ) < delta
  assert abs( a_J[0,1] - x[0] * exp(x[1]) ) < delta
  assert abs( a_J[1,0] -        sin(x[1]) ) < delta
  assert abs( a_J[1,1] - x[0] * cos(x[1]) ) < delta
  assert abs( a_J[2,0] -        cos(x[1]) ) < delta
  assert abs( a_J[2,1] + x[0] * sin(x[1]) ) < delta

Input File: example/jacobian.py
6.7: Driver for Computing Hessian in a Range Direction

6.7.a: Syntax
H = f.hessian(xw)

6.7.b: Purpose
This routine computes the Hessian of the weighted sum  \[
     w_0 * F_0 (x) + \cdots + w_{m-1} * F_{m-1} (x)
\] 
where  F : \B{R}^n \rightarrow \B{R}^m is the function corresponding to the adfun object 6.2.e: f .

6.7.c: f
The object f must be an 6.2: adfun object. We use 6.2.e.c: level for the AD 5.1: ad level of this object.

6.7.d: x
The argument x is a numpy.array with one dimension (i.e., a vector) with length equal to the domain size 6.2.e.b: n for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of x must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of x must be a_float objects.

6.7.e: w
The argument w is a numpy.array with one dimension (i.e., a vector) with length equal to the range size 6.2.e.a: m for the function f . It specifies the argument value at which the derivative is computed. If the AD 6.2.e.c: level for f is zero, all the elements of w must be either int or instances of float. If the AD 6.2.e.c: level for f is one, all the elements of w must be a_float objects.

6.7.f: H
The return value H is a numpy.array with two dimensions (i.e., a matrix). Both its first and second dimension size (row and column size) are equal to 6.2.e.b: n (the number of domain components in the function f ). It is set to the Hessian; i.e.,  \[
     H = w_0 * F_0^{(2)} (x) + \cdots + w_{m-1} * F_{m-1}^{(2)} (x)
\] 
If the AD 6.2.e.c: level for f is zero, all the elements of H will be instances of float. If the AD 6.2.e.c: level for f is one, all the elements of H will be a_float objects.

6.7.g: Example
The file 6.7.1: hessian.py contains an example and test of this operation.
Input File: pycppad/adfun.cpp
6.7.1: Hessian Driver: Example and Test
 
from pycppad import *
# Example using a_float -----------------------------------------------------
def pycppad_test_hessian():
  delta = 10. * numpy.finfo(float).eps
  x     = numpy.array( [ 0., 0. ] )
  a_x   = independent(x)
  a_y   = numpy.array( [ 
    a_x[0] * exp(a_x[1]) , 
    a_x[0] * sin(a_x[1]) ,
    a_x[0] * cos(a_x[1]) 
  ] )
  f   = adfun(a_x, a_y)
  x   = numpy.array( [ 2., 3. ] )
  w   = numpy.array( [ 0., 1., 0. ] ) # compute Hessian of x0 * sin(x1)
  H   = f.hessian(x, w)
  assert abs( H[0,0] - 0.               ) < delta
  assert abs( H[0,1] - cos(x[1])        ) < delta
  assert abs( H[1,0] - cos(x[1])        ) < delta
  assert abs( H[1,1] + x[0] * sin(x[1]) ) < delta
# Example using a2float -----------------------------------------------------
def pycppad_test_hessian_a2():
  delta = 10. * numpy.finfo(float).eps
  a_x   = ad( numpy.array( [ 0., 0. ] ) )
  a2x   = independent(a_x)
  a2y   = numpy.array( [ 
    a2x[0] * exp(a2x[1]) , 
    a2x[0] * sin(a2x[1]) ,
    a2x[0] * cos(a2x[1]) 
  ] )
  a_f = adfun(a2x, a2y)
  x   = numpy.array( [ 2., 3. ] )
  a_x = ad(x)
  a_w = ad( numpy.array( [ 0., 1., 0. ] ) ) # compute Hessian of x0 * sin(x1)
  a_H = a_f.hessian(a_x, a_w)
  assert abs( a_H[0,0] - 0.               ) < delta
  assert abs( a_H[0,1] - cos(x[1])        ) < delta
  assert abs( a_H[1,0] - cos(x[1])        ) < delta
  assert abs( a_H[1,1] + x[0] * sin(x[1]) ) < delta

Input File: example/hessian.py
6.8: Optimize an AD Function Object Tape

6.8.a: Syntax
f.optimize()

6.8.b: Purpose
The operation sequence corresponding to an 6.2: ADFun object can be very large and involve many operations. The f.optimize procedure reduces the number of operations, and thereby the time and memory, required to compute function and derivative values.

6.8.c: f
The object f is an 6.2: adfun object.

6.8.d: Efficiency
The optimize member function may greatly reduce the size of the operation sequence corresponding to f .

6.8.e: Example
The file 6.8.1: optimize.py contains an example and test of this operation.
Input File: pycppad/adfun.cpp
6.8.1: Optimize Function Object: Example and Test
 
from pycppad import *
import time
# Example using a_float -----------------------------------------------------
def pycppad_test_optimize():
  # create function with many variables that are get removed by optimize
  n_sum = 10000
  x     = numpy.array( [ 0. ] )
  a_x   = independent(x)
  a_sum = 0.
  for i in range(n_sum) :
    a_sum = a_sum + a_x[0];
  a_y = numpy.array( [ a_sum ] )
  f   = adfun(a_x, a_y)
  # time for a forward operations before optimize
  x          = numpy.array( [ 1. ] )
  t0         = time.time()
  sum_before = f.forward(0, x)
  sec_before = time.time() - t0
  # time for a forward operations after optimize
  f.optimize()
  t0         = time.time()
  sum_after  = f.forward(0, x)
  sec_after  = time.time() - t0
  assert sum_before == float(n_sum)
  assert sum_after  == float(n_sum)
  # expect sec_before to be less than 2 times sec_after
  assert( sec_after * 1.5 <= sec_before )

Input File: example/optimize.py
7: Using Two Levels of AD: Example and Test

7.a: Purpose
This example is intended to demonstrate how a_float and a2float can be used together to compute derivatives of functions that are defined in terms of derivatives of other functions.

7.b: F(u)
For this example, the function  F : \B{R}^2 \rightarrow  \B{R} is defined by  \[
     F(u) = u_0^2 + u_1^2
\] 
It follows that  \[
\begin{array}{rcl}
     \partial_{u(0)} F(u) = 2 * u_0 \\
     \partial_{u(1)} F(u) = 2 * u_1
\end{array}
\] 


7.c: G(x)
For this example, the function  G : \B{R}^2 \rightarrow \B{R} is defined by  \[
G(x) = x_1 * \partial_{u(0)} F(x_0 , 1) + x_0 * \partial_{u(1)} F(x_0, 1)
\] 
where  \partial{u(j)} F(a, b) denotes the partial of  F with respect to  u_j and evaluated at  u = (a, b) . It follows that  \[
\begin{array}{rcl}
     G (x)                 & = & 2 * x_1 * x_0  + 2 * x_0 \\
     \partial_{x(0)} G (x) & = & 2 * x_1 + 2 \\
     \partial_{x(1)} G (x) & = & 2 * x_0
\end{array}
\] 
 

from pycppad import *
def pycppad_test_two_levels():
  # start recording a_float operations
  x   = numpy.array( [ 2. , 3. ] )
  a_x = independent(x)

  # start recording a2float operations
  a_u = numpy.array( [a_x[0]  , ad(1) ] )
  a2u = independent(a_u)

  # stop a2float recording and store operations if f
  a2v = numpy.array( [ a2u[0] * a2u[0] + a2u[1] * a2u[1] ] )
  a_f = adfun(a2u, a2v)              # F(u0, u1) = u0 * u0 + u1 * u1

  # evaluate the gradient of F
  a_J = a_f.jacobian(a_u)

  # stop a_float recording and store operations in g
  a_y = numpy.array( [ a_x[1] * a_J[0,0] + a_x[0] * a_J[0,1] ] )
  g   = adfun(a_x, a_y)  # G(x0, x1) = x1 * F_u0(x0, 1) + x0 * F_u1(x0, 1)
  
  # evaluate the gradient of G
  J   = g.jacobian(x)

  assert J[0,0] == 2. * x[1] + 2
  assert J[0,1] == 2. * x[0]

Input File: example/two_levels.py
8: Fourth Order Runge Kutta

8.a: Syntax
 yf = runge_kutta_4(ftiyidt)

8.b: Purpose
See 8.h: Motivation below as well as the purpose described here. We are given a function  f : \B{R}^n \rightarrow \B{R}^n , and a point  yi \in \B{R}^n such that an unknown function  y : \B{R} \rightarrow \B{R}^n  satisfies the equations  \[
     \begin{array}{rcl}
          y( ti ) & = & yi \\
          y'(t)    & = & f[t, y(t) ] \\
     \end{array}
\] 
We use the Fourth order Runge-Kutta formula (see equation 16.1.2 of Numerical Recipes in Fortran, 2nd ed.) wish to approximate the value of  \[
     yf = y( ti + \Delta t )
\] 


8.c: f
If t is a scalar and y is a vector with size  n ,
     k = f(t, y)
returns a vector of size  n that is the value of  f(t, y) at the specified values.

8.d: ti
is a scalar that specifies the value of  ti in the problem above.

8.e: yi
is a vector of size  n that specifies the value of  yi in the problem above.

8.f: dt
is a scalar that specifies the value of  \Delta t in the problem above.

8.g: yf
is a vector of size  n that is the approximation for  y( t + \Delta t ) .

8.h: Motivation
This routine makes very few assumptions about the objects used to do these calculations. Thus, smart objects can be used for all sorts of purposes; for example, computing derivatives of the solution of an ODE. The table below lists the assumptions on the objects passed into runge_kutta_4. In this table, s and t are scalars, d is a decimal number (i.e., a float) and u and v are vectors with size  n .
operation result
d * s scalar
s + t scalar
s * u vector with size  n
d * u vector with size  n
s * u vector with size  n
u + v vector with size  n

8.i: Source Code
 
def runge_kutta_4(f, ti, yi, dt) :
	k1 = dt * f(ti         , yi)
	k2 = dt * f(ti + .5*dt , yi + .5*k1) 
	k3 = dt * f(ti + .5*dt , yi + .5*k2) 
	k4 = dt * f(ti + dt    , yi + k3)
	yf = yi + (1./6.) * ( k1 + 2.*k2 + 2.*k3 + k4 )
	return yf 


8.j: Example
  1. The file 8.1: runge_kutta_4_correct.py contains an example and test of using runge_kutta_4 to solve an ODE.
  2. The file 8.2: runge_kutta_4_ad.py contains an example and test of differentiating the numerical solution of an ODE.
  3. The file 8.3: runge_kutta_4_cpp.py contains an example and test of using pycppad 6.2: adfun object to evaluate python functions with C++ speed of execution.

Input File: pycppad/runge_kutta_4.py
8.1: runge_kutta_4 A Correctness Example and Test

8.1.a: Discussion
Define  y : \B{R} \rightarrow \B{R}^n by  \[
     y_j (t) =  t^{j+1}
\] 
It follows that the derivative of  y(t) satisfies the 8: runge_kutta_4 ODE equation where  y(0) = 0  and  f(t, y) is given by  \[
f(t , y)_j = y_j '(t) = \left\{ \begin{array}{ll}
 1                 & {\; \rm if \;} j = 0      \\
 (j+1) y_{j-1} (t) & {\; \rm otherwise }
\end{array} \right.
\] 


8.1.b: Source Code
 
from pycppad import *
def pycppad_test_runge_kutta_4_correct() :
	def fun(t , y) :
		n        = y.size
		f        = numpy.zeros(n)
		f[0]     = 1.
		index    = numpy.array( range(n-1) ) + 1
		f[index] = (index + 1) * y[index-1] 
		return f
	n  = 5              # size of y(t) (order of method plus 1)
	ti = 0.             # initial time
	dt = 2.             # a very large time step size to test correctness
	yi = numpy.zeros(n) # initial value for y(t); i.e., y(0)

	# take one 4-th order Runge-Kutta integration step of size dt 
	yf = runge_kutta_4(fun, ti, yi, dt)

	# check the results
	t_jp = 1.                                # t^0 at t = dt
	for j in range(n-1) :
		t_jp = t_jp * dt                    # t^(j+1) at t = dt
		assert abs( yf[j] - t_jp ) < 1e-10  # check yf[j] = t^(j+1)

Input File: example/runge_kutta_4_correct.py
8.2: runge_kutta_4 An AD Example and Test

8.2.a: Discussion
Define  y : \B{R} \times \B{R} \rightarrow \B{R}^n by  \[
     y_j (x, t) =  x t^{j+1}
\] 
It follows that the derivative of  y(t) satisfies the 8: runge_kutta_4 ODE equation where  y(0) = 0  and  f(t, y) is given by  \[
f(t , y)_j = \partial_t y_j (x, t) = \left\{ \begin{array}{ll}
 x                    & {\; \rm if \;} j = 0      \\
 (j+1) y_{j-1} (x, t) & {\; \rm otherwise }
\end{array} \right.
\] 
It also follows that  \[
     \partial_x y_j (x, t) = t^{j+1}
\] 


8.2.b: Source Code
 
from pycppad import *
def pycppad_test_runge_kutta_4_ad() :
	def fun(t , y) :
		n        = y.size
		f        = ad( numpy.zeros(n) )
		f[0]     = a_x[0]
		index    = numpy.array( range(n-1) ) + 1
		f[index] = (index + 1) * y[index-1] 
		return f
	n  = 5         # size of y(t) (order of method plus 1)
	ti = 0.        # initial time
	dt = 2.        # a very large time step size (method is exact)

	# initial value for y(t); i.e., y(0)
	a_yi = ad( numpy.zeros(n) ) 
	
	# declare a_x to be the independent variable vector
	x    = numpy.array( [.5] )
	a_x  = independent( numpy.array( x ) )

	# take one 4-th order Runge-Kutta integration step of size dt 
	a_yf = runge_kutta_4(fun, ti, a_yi, dt)

	# define the AD function g : x -> yf
	g = adfun(a_x, a_yf)

	# compute the derivative of g w.r.t x at x equals .5
	dg = g.jacobian(x)
	
	# check the result is as expected
	t_jp = 1                                        # t^0
	for j in range(n-1) :
		t_jp  = t_jp * dt                          # t^(j+1) at t = dt
		assert abs( a_yf[j] - x[0]*t_jp ) < 1e-10  # check yf[j] = x*t^(j+1_
		assert abs( dg[j,0] - t_jp    ) < 1e-10    # check dg[j] = t^(j+1)

Input File: example/runge_kutta_4_ad.py
8.3: runge_kutta_4 With C++ Speed: Example and Test

8.3.a: Purpose
In this example we demonstrate how a Python function can be recorded in a pycppad function object and then evaluated at much higher speeds than the Python evaluation.

8.3.b: Discussion
Define  y : \B{R}^2 \times \B{R} \rightarrow \B{R}^n by  \[
     \begin{array}{rcl}
     y(x, 0)            & = & x_0 
     \\
     \partial_t y(x, t) & = & x_1 y(x, t)
     \end{array}
\] 
It follows that  \[
     y(x, t) = x_0 \exp ( x_1 t )
\] 
Suppose we want to compute values for the function  g : \B{R}^2 \rightarrow \B{R}  defined by  \[
     g(x) = y(x, 1)
\] 
In this example we compare the execution time for doing this in pure Python with using a pycppad function object to compute  g(x) in C++.

8.3.c: Source Code
 
from pycppad import *
import time
def pycppad_test_runge_kutta_4_cpp() :
	x_1 = 0;   # use this variable to switch x_1 between float and ad(float)
	def fun(t , y) :
		f     = x_1 * y
		return f
	# Number of Runge-Kutta times steps to include in the function object
	M = 100 

	# Start time for recording the pycppad function object
	s0  = time.time()
	# Declare three independent variables. The operation sequence does not
	# depend on x, so we could use any value here.
	x    = numpy.array( [.1, .1, .1] )
	a_x  = independent( numpy.array( x ) )
	# First independent variables, x[0], is the value of y(0)
	a_y = numpy.array( [ a_x[0] ] )
	# Make x_1 a variable so can use rk4 with various coefficients.
	x_1 = a_x[1]
	# Make dt a variable so can use rk4 with various step sizes.
	dt  = a_x[2]
	# f(t, y) does not depend on t, so no need to make t a variable.
	t   = ad(0.)
	# Record the operations for 10 time step
	for k in range(M) :
		a_y = runge_kutta_4(fun, t, a_y, dt)
		t   = t + dt
	# define the AD function rk4 : x -> y
	rk4 = adfun(a_x, a_y)
	# amount of time it took to tape this function object
	tape_sec = time.time() - s0
	# make the fucntion object more efficient
	s0       = time.time()
	rk4.optimize()
	opt_sec  = time.time() - s0

	ti  = 0.              # initial time
	tf  = 1.              # final time
	N   = M * 100         # number of time steps 
	dt  = (tf - ti) / N   # size of time step
	x_0 = 2.              # use this for initial value of y(t)
	x_1 = .5              # use this for coefficient in ODE

	# python version of integrator with float values
	s0  = time.time()
	t   = ti
	y   = numpy.array( [ x_0 ] ); 
	for k in range(N) :
		y = runge_kutta_4(fun, t, y, dt)
		t = t + dt
	# number of seconds to solve the ODE using python float
	python_sec =  time.time() - s0
	# check solution is correct
	assert( abs( y[0] - x_0 * exp( x_1 * tf ) ) < 1e-10 ) 

	# pycppad function object version of integrator 
	s0  = time.time()
	t   = ti
	x   = numpy.array( [ x_0 , x_1 , dt ] ) 
	for k in range(N/M) :
		y    = rk4.forward(0, x);
		x[0] = y[0];
	# number of seconds to solve the ODE using pycppad function object
	cpp_sec =  time.time() - s0
	# check solution is correct
	assert( abs( y[0] - x_0 * exp( x_1 * tf ) ) < 1e-10 ) 

	# Uncomment the print statement below to see actual times on your machine
	format =          'cpp_sec            = %8f,\n'
	format = format + 'python_sec/cpp_sec = %5.1f\n'
	format = format + 'tape_sec/cpp_sec   = %5.1f\n'
	format = format + 'opt_sec/cpp_sec    = %5.1f'
	s      = cpp_sec
	# print format % (s, python_sec/s, tape_sec/s, opt_sec/s )
	
	# check that C++ is always more than 30 times faster (on all systems)
	assert( 30. * cpp_sec <= python_sec )

Input File: example/runge_kutta_4_cpp.py
9: Extensions, Bug Fixes, and Changes

9.a: 2014-07-10
Suppress warnings about using deprecated features of the numpy C-API; see numpy c-api deprecation (http://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html)

9.b: 2014-05-29
Add support for the 5.4.2: future divide operator .

9.c: Long Ago
9.1: 2010 , 9.2: 2010 , 9.3: 2010 , 9.4: 2008
Input File: omh/whats_new.omh
9.1: Extensions, Bug Fixes, and Changes During 2012

9.1.a: 10-19
The package pycppad.cppad_ was missing form the installation (after testing). This is due to be a bug in the python distutils (http://docs.python.org/library/distutils.html) package when the --inplace option is specified. The 2.h: install instructions have been changed to avoid this problem.

9.1.b: 10-01
Change the 8.3: runge_kutta_4_cpp.py test from asserting that C++ is 100 times faster than Python to 75 times faster.

9.1.c: 04-17
  1. Add missing --inplace flag in 2.f: build instructions.
  2. Add with_debugging option to 2.g: testing so that test_more.py does not fail when you build without debugging.


9.1.d: 04-16
  1. Simplify the setup.py script used during the 2: install process. This included having the user to download and install CppAD, instead of having setup.py do this task.
  2. Document the change to the derivative of the 5.8: abs that occurred in CppAD on 2011-12-30 (http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-30) .

Input File: omh/whats_new_12.omh
9.2: Extensions, Bug Fixes, and Changes During 2011

9.2.a: 10-18
  1. Fix problem with import cppad_ in 20111017 version.
  2. Change 8.3: runge_kutta_4_cpp.py to use 6.8: optimize and to check that using pycppad is always more than 100 times faster than straight python for this test case.


9.2.b: 10-17
Add tape optimization; see 6.8: optimize .

9.2.c: 10-16
  1. Add the conditional expressions; see 5.9: condexp .
  2. Use newer version of CppAD that fixes warnings found by g++ version 4.6.1.


9.2.d: 03-23
The --undef option is not available to ./setup.py build so ./setup.py build was changed to ./setup.py build_ext in the 2.f: build instructions .

9.2.e: 03-20
  1. There was a problem with the pycppad 2: install instructions. To be specific, the debug build was not being installed because of the presence of --inplace in the example setup.py command. This has been fixed in the new 2.f.a: debug build instructions .
  2. Change from using cppad-20100101.5 to using cppad-20110101.2.gpl.tgz; see CppAD whats_new (http://www.coin-or.org/CppAD/Doc/whats_new.htm) .
  3. Use external sub directory of current working directory, instead of $HOME/install, for default directory that holds a local copy of CppAD; see cppad_parent_dir under 2.e: Required Setup Information .
  4. Add boost_python_include_dir to list of 2.e: Required Setup Information .

Input File: omh/whats_new_11.omh
9.3: Extensions, Bug Fixes, and Changes During 2010

9.3.a: 12-12
The 6.1: independent(x) and 6.2: adfun(x,y) functions require the elements of x and y to all have the same type. If this was not the case, pycppad might crash with out a useful error message. This has been fixed.

9.3.b: 12-05
First version that works (and installs properly) under MS windows using the cygwin (www.cygwin.com) system; see special pycppad instructions for installing under 2.f.d: cygwin .

9.3.c: 11-30
Upgraded from using cppad-20100101.2 to cppad-20100101.5. This fixes a problem when installing pycppad with version 1.44.0 of boost-python (http://www.boost.org/doc/libs/1_44_0/libs/python/doc/index.html) .

9.3.d: 03-18
The upstream tarball for CppAD was missing from The 2: install process was failing because the upstream tarball was missing from http://www.coin-or.org/download/source/CppAD/ . This has been fixed, and a newer upstream release has been used.

9.3.e: 02-10
  1. Added --inplace to the 2.f.b: optimized build instructions. Improved the 2.h: install instructions and the discussion of the 2.i: python path .
  2. Changes exit to sys.exit in 2.g: test_example.py and test_more.py (required on some systems).
  3. Fixed the 5.3.1: ad_unary.py example so it displays the correct source code and changed all to numpy.all in that example (required on some systems).


9.3.f: 01-31
Add the 6.3: abort_recording function.

9.3.g: 01-30
Change setup.py do that it patches the CppAD distribution (this fixes a problem with the 2.f.b: optimized build).

9.3.h: 01-29
Added the python ODE solver 8: runge_kutta_4 which can be used with `smart' floating point types (e.g., AD types).

9.3.i: 01-28
Upgraded from using cppad-20090909.0 to cppad-20100101.0.
Input File: omh/whats_new_10.omh
9.4: Extensions, Bug Fixes, and Changes During 2009

9.4.a: 06-21
The Boost Python library name was not being used correctly and hence one would get the error message: "Cannot find the Boost Python library" ... This has been fixed.

9.4.b: 06-13
The setup.py file (used during the 2.f: build step of the install) had /home/bradbell/install as the value of cppad_parent_dir. This has been changed so that the default is $HOME/install.

9.4.c: 05-13
Changed to a new version of CppAD, (20090303.0).

9.4.d: 05-12
The Boost Python library libboost_python-py26.so had a problem with ad(x) where x was of type int. This has been fixed.

9.4.e: 02-13
Changed to new version of CppAD, (20090131.0) so that tanh is now included in 5.7: std_math (it was documented but missing from the actual pycppad implementation before this date).

9.4.f: 02-09
Added cross reference links from the examples to the corresponding section. For example, when independent appears in example code, it is linked in the following way: 6.1: independent .

9.4.g: 01-31
  1. Change distribution to automatically download and link in proper version of CppAD.
  2. Remove tanh from standard math functions because previous release of CppAD did not include it (a new release of CppAD is being built so that tanh can be included in pycppad).
  3. The two tests in example/ad_unary.py had the same name and hence only one was being run. This has been fixed.


9.4.h: 01-30
  1. Organize the documentation by grouping the 5: ad_variable and 6: ad_function operations.
  2. Add a 3: get_started.py section.
  3. Add the 5.3: unary plus and minus operators .
  4. Sort the installation 2.g: testing results in alphabetical order.
  5. Use the ** exponentiation operator in 3: get_started.py .
  6. Put this whats new section in inverse order by date.
  7. Improve the error reporting done by the 5.2: value function.


9.4.i: 01-29
A list of 4: examples has been added to the documentation. The code for the 7: two_levels.py example was missing, this has been fixed.

The pycppad tests no longer require any package to run; i.e., it is no longer necessary to install --       py.test
-
(- http://codespeak.net/py/dist/) for testing pycppad.

9.4.j: 01-28
Change from an general AD package to just one package; i.e. from pyad-yyyymmdd.tar.gz to pycppad-yyyymmdd.tar.gz . This is a change in API requiring user code to change import pyad.cppad to import pycppad.

9.4.k: 01-26
Released first version, pyad-20090126.

Add build inplace, and omitting the setup.py install step, to 2: install documentation.
Input File: omh/whats_new_09.omh
10: License
-------------------------------------------------------------------------------
pycppad: Python Algorithmic Differentiation Using CppAD
Authors: Sebastian F. Walter and Bradley M. Bell.

BSD style using http://www.opensource.org/licenses/bsd-license.php template 
as it was on 2009-01-24 with the following substutions:
	<YEAR>         = 2008-2009
	<OWNER>        = Bradley M. Bell and Sebastian F. Walter
	<ORGANIZATION> = contributors' organizations
In addition,   "Neither the name of the contributors' organizations" 
was changed to "Neither the names of the contributors' organizations" 
-------------------------------------------------------------------------------

Copyright (c) 2008-2009, Bradley M. Bell and Seastian F. Walter
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, 
are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice, 
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright notice, 
      this list of conditions and the following disclaimer in the documentation 
      and/or other materials provided with the distribution.
    * Neither the names of the contributors' organizations nor the names of 
      its contributors may be used to endorse or promote products derived from 
      this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE 
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


Input File: omh/license.omh
11: Alphabetic Listing of Cross Reference Tags
A
6.3: abort_recording
Abort a Recording of AD Operations
6.3.1: abort_recording.py
abort_recording: Example and Test
5.8: abs
Absolute Value Functions
5.8.1: abs.py
abs: Example and Test
5.1: ad
Create an Object With One Higher Level of AD
5.1.1: ad.py
ad: Example and Test
6: ad_function
AD Function Methods
5.4: ad_numeric
Binary Numeric Operators With an AD Result
5.4.1: ad_numeric.py
Binary Numeric Operators With an AD Result: Example and Test
5.3: ad_unary
Unary Plus and Minus Operators
5.3.1: ad_unary.py
Unary Plus and Minus Operators: Example and Test
5: ad_variable
AD Variable Methods
6.2: adfun
Create an AD Function Object
6.2.1: adfun.py
adfun: Example and Test
5.5: assign_op
Computed Assignment Operators
5.5.1: assign_op.py
Computed Assignment Operators: Example and Test
C
5.6: compare_op
Binary Comparison Operators
5.6.1: compare_op.py
a_float Comparison Operators: Example and Test
5.9: condexp
Conditional Expressions
5.9.1: condexp.py
condexp: Example and Test
E
4: example
List of All the pycppad Examples
F
6.4: forward
Forward Mode: Derivative in One Domain Direction
6.4.1: forward_0.py
Forward Order Zero: Example and Test
6.4.2: forward_1.py
Forward Order One: Example and Test
5.4.2: future_div_op.py
Future Division Operator: Example and Test
G
3: get_started.py
get_started: Example and Test
H
6.7: hessian
Driver for Computing Hessian in a Range Direction
6.7.1: hessian.py
Hessian Driver: Example and Test
I
6.1: independent
Create an Independent Variable Vector
6.1.1: independent.py
independent: Example and Test
2: install
Installing pycppad
J
6.6: jacobian
Driver for Computing Entire Derivative
6.6.1: jacobian.py
Entire Derivative: Example and Test
L
10: license
License
O
6.8: optimize
Optimize an AD Function Object Tape
6.8.1: optimize.py
Optimize Function Object: Example and Test
P
: pycppad
pycppad-20140710: A Python Algorithm Derivative Package
R
6.5: reverse
Reverse Mode: Derivative in One Range Direction
6.5.1: reverse_1.py
Reverse Order One: Example and Test
6.5.2: reverse_2.py
Reverse Order Two: Example and Test
8: runge_kutta_4
Fourth Order Runge Kutta
8.2: runge_kutta_4_ad.py
runge_kutta_4 An AD Example and Test
8.1: runge_kutta_4_correct.py
runge_kutta_4 A Correctness Example and Test
8.3: runge_kutta_4_cpp.py
runge_kutta_4 With C++ Speed: Example and Test
S
5.7: std_math
Standard Math Unary Functions
5.7.1: std_math.py
Standard Math Unary Functions: Example and Test
T
7: two_levels.py
Using Two Levels of AD: Example and Test
V
5.2: value
Create an Object With One Lower Level of AD
5.2.1: value.py
value: Example and Test
W
9: whats_new
Extensions, Bug Fixes, and Changes
9.4: whats_new_09
Extensions, Bug Fixes, and Changes During 2009
9.3: whats_new_10
Extensions, Bug Fixes, and Changes During 2010
9.2: whats_new_11
Extensions, Bug Fixes, and Changes During 2011
9.1: whats_new_12
Extensions, Bug Fixes, and Changes During 2012

12: Keyword Index
!= 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
* 5.4: Binary Numeric Operators With an AD Result
     example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
** 5.4: Binary Numeric Operators With an AD Result
     example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
*= 5.5: Computed Assignment Operators
     example 5.5.1: Computed Assignment Operators: Example and Test
+
     binary 5.4: Binary Numeric Operators With an AD Result
     example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     unary 5.3.1: Unary Plus and Minus Operators: Example and Test
     unary 5.3: Unary Plus and Minus Operators
+= 5.5: Computed Assignment Operators
     example 5.5.1: Computed Assignment Operators: Example and Test
-
     binary 5.4: Binary Numeric Operators With an AD Result
     example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     unary 5.3.1: Unary Plus and Minus Operators: Example and Test
     unary 5.3: Unary Plus and Minus Operators
-= 5.5: Computed Assignment Operators
     example 5.5.1: Computed Assignment Operators: Example and Test
/ 5.4: Binary Numeric Operators With an AD Result
     example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     future example 5.4.2: Future Division Operator: Example and Test
/= 5.5: Computed Assignment Operators
     example 5.5.1: Computed Assignment Operators: Example and Test
2009
     feature 9.4: Extensions, Bug Fixes, and Changes During 2009
< 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
<= 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
== 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
> 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
>= 5.6: Binary Comparison Operators
     example 5.6.1: a_float Comparison Operators: Example and Test
A
AD
     abort recording 6.3: Abort a Recording of AD Operations
     binary numeric operator 5.4: Binary Numeric Operators With an AD Result
     decrease level 5.2: Create an Object With One Lower Level of AD
     increase level 5.1: Create an Object With One Higher Level of AD
     python : pycppad-20140710: A Python Algorithm Derivative Package
     two levels 7: Using Two Levels of AD: Example and Test
     unary operator 5.3: Unary Plus and Minus Operators
a2float 7: Using Two Levels of AD: Example and Test
     create 5.1.d: Create an Object With One Higher Level of AD: a_x
a_float
     comparison operator 5.6.1: a_float Comparison Operators: Example and Test
     create 5.1.d: Create an Object With One Higher Level of AD: a_x
abort
     AD recording 6.3: Abort a Recording of AD Operations
abort_recording
     example 6.3.1: abort_recording: Example and Test
abs 5.9: Conditional Expressions
    5.8: Absolute Value Functions
     example 5.8.1: abs: Example and Test
ad 5.1: Create an Object With One Higher Level of AD
     example 5.1.1: ad: Example and Test
     variable 5: AD Variable Methods
adfun
     example 6.2.1: adfun: Example and Test
algorithmic differentiation
     python : pycppad-20140710: A Python Algorithm Derivative Package
arccos 5.7: Standard Math Unary Functions
arcsin 5.7: Standard Math Unary Functions
arctan 5.7: Standard Math Unary Functions
assignment
     assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
     computed operator 5.5: Computed Assignment Operators
automatic differentiation
     python : pycppad-20140710: A Python Algorithm Derivative Package
B
binary
     AD numeric operator 5.4: Binary Numeric Operators With an AD Result
     bool a_float operator example 5.6.1: a_float Comparison Operators: Example and Test
     bool operator 5.6: Binary Comparison Operators
     comparison operator 5.6: Binary Comparison Operators
     numeric operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
bool
     a_float comparison operator example 5.6.1: a_float Comparison Operators: Example and Test
     binary operator 5.6: Binary Comparison Operators
bug
     fix 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
     python distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
build
     debugging 2.f.a: Installing pycppad: Building.With Debugging
     optimize 2.f.b: Installing pycppad: Building.Optimized
     pycppad 2.f: Installing pycppad: Building
C
C++ speed
     python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
comparison
     a_float operator 5.6.1: a_float Comparison Operators: Example and Test
     binary operator 5.6: Binary Comparison Operators
computed
     assignment operator 5.5: Computed Assignment Operators
     assignment operator example 5.5.1: Computed Assignment Operators: Example and Test
condexp
     example 5.9.1: condexp: Example and Test
conditional
     expression 5.9.1: condexp: Example and Test
cos 5.7: Standard Math Unary Functions
cosh 5.7: Standard Math Unary Functions
create
     a2float 5.1.d: Create an Object With One Higher Level of AD: a_x
     a_float 5.1.d: Create an Object With One Higher Level of AD: a_x
D
debugging
     build 2.f.a: Installing pycppad: Building.With Debugging
dependent
     variables 6.2: Create an AD Function Object
derivative
     domain direction 6.4: Forward Mode: Derivative in One Domain Direction
     entire driver 6.6: Driver for Computing Entire Derivative
     entire example 6.6.1: Entire Derivative: Example and Test
     ODE solution 8.2: runge_kutta_4 An AD Example and Test
     range direction 6.5: Reverse Mode: Derivative in One Range Direction
direction
     domain derivative 6.4: Forward Mode: Derivative in One Domain Direction
     range derivative 6.5: Reverse Mode: Derivative in One Range Direction
distutils
     python bug 2.f.c: Installing pycppad: Building.Bug in Python Distutils
divide 5.4: Binary Numeric Operators With an AD Result
domain
     direction derivative 6.4: Forward Mode: Derivative in One Domain Direction
download
     pycppad 2.c: Installing pycppad: Downloading
driver
     entire derivative 6.6: Driver for Computing Entire Derivative
     hessian 6.7: Driver for Computing Hessian in a Range Direction
E
entire
     derivative driver 6.6: Driver for Computing Entire Derivative
     derivative example 6.6.1: Entire Derivative: Example and Test
example
     != 5.6.1: a_float Comparison Operators: Example and Test
     * 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     ** 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     *= 5.5.1: Computed Assignment Operators: Example and Test
     + 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     + 5.3.1: Unary Plus and Minus Operators: Example and Test
     += 5.5.1: Computed Assignment Operators: Example and Test
     - 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     - 5.3.1: Unary Plus and Minus Operators: Example and Test
     -= 5.5.1: Computed Assignment Operators: Example and Test
     / 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     /= 5.5.1: Computed Assignment Operators: Example and Test
     < 5.6.1: a_float Comparison Operators: Example and Test
     <= 5.6.1: a_float Comparison Operators: Example and Test
     == 5.6.1: a_float Comparison Operators: Example and Test
     > 5.6.1: a_float Comparison Operators: Example and Test
     >= 5.6.1: a_float Comparison Operators: Example and Test
     abort_recording 6.3.1: abort_recording: Example and Test
     abs 5.8.1: abs: Example and Test
     ad 5.1.1: ad: Example and Test
     adfun 6.2.1: adfun: Example and Test
     bool a_float binary operator 5.6.1: a_float Comparison Operators: Example and Test
     computed assignment operator 5.5.1: Computed Assignment Operators: Example and Test
     condexp 5.9.1: condexp: Example and Test
     entire derivative 6.6.1: Entire Derivative: Example and Test
     future / 5.4.2: Future Division Operator: Example and Test
     get_started 3: get_started: Example and Test
     hessian 6.7.1: Hessian Driver: Example and Test
     independent 6.1.1: independent: Example and Test
     list 4: List of All the pycppad Examples
     numeric binary operator 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
     optimize 6.8.1: Optimize Function Object: Example and Test
     order one forward 6.4.2: Forward Order One: Example and Test
     order one reverse 6.5.1: Reverse Order One: Example and Test
     order two reverse 6.5.2: Reverse Order Two: Example and Test
     order zero forward 6.4.1: Forward Order Zero: Example and Test
     runge_kutta_8.3: runge_kutta_4 With C++ Speed: Example and Test
     runge_kutta_8.2: runge_kutta_4 An AD Example and Test
     runge_kutta_8.1: runge_kutta_4 A Correctness Example and Test
     standard math 5.7.1: Standard Math Unary Functions: Example and Test
     value 5.2.1: value: Example and Test
exp 5.7: Standard Math Unary Functions
exponentiation 5.4: Binary Numeric Operators With an AD Result
extract
     pycppad 2.d: Installing pycppad: Unpacking
F
feature
     version 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
fix
     bug 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
forward 6.4: Forward Mode: Derivative in One Domain Direction
     order one example 6.4.2: Forward Order One: Example and Test
     order zero example 6.4.1: Forward Order Zero: Example and Test
future /
     example 5.4.2: Future Division Operator: Example and Test
G
get_started
     example 3: get_started: Example and Test
H
hessian
     driver 6.7: Driver for Computing Hessian in a Range Direction
     example 6.7.1: Hessian Driver: Example and Test
     Lagrangian 6.7: Driver for Computing Hessian in a Range Direction
I
independent
     abort recording 6.3: Abort a Recording of AD Operations
     example 6.1.1: independent: Example and Test
     variables 6.1.a: Create an Independent Variable Vector: Syntax
install
     pycppad 2.h: Installing pycppad: Installing
     pycppad 2: Installing pycppad
J
jacobian 6.6: Driver for Computing Entire Derivative
     example 6.6.1: Entire Derivative: Example and Test
L
Lagrangian
     hessian 6.7: Driver for Computing Hessian in a Range Direction
level
     decrease AD 5.2: Create an Object With One Lower Level of AD
     increase AD 5.1: Create an Object With One Higher Level of AD
levels
     two AD 7: Using Two Levels of AD: Example and Test
license
     pycppad 10: License
list
     example 4: List of All the pycppad Examples
log 5.7: Standard Math Unary Functions
log10 5.7: Standard Math Unary Functions
M
math
     example standard 5.7.1: Standard Math Unary Functions: Example and Test
memory
     optimize 6.8: Optimize an AD Function Object Tape
minus
     binary 5.4: Binary Numeric Operators With an AD Result
     unary 5.3: Unary Plus and Minus Operators
N
numeric
     AD binary operator 5.4: Binary Numeric Operators With an AD Result
     binary operator example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
O
ODE solution
     correct 8.1: runge_kutta_4 A Correctness Example and Test
     derivative 8.2: runge_kutta_4 An AD Example and Test
ODE solver
     Runge-Kutta 8: Fourth Order Runge Kutta
one
     order forward example 6.4.2: Forward Order One: Example and Test
     order reverse example 6.5.1: Reverse Order One: Example and Test
operation
     optimize sequence 6.8.1: Optimize Function Object: Example and Test
operations
     optimize sequence 6.8: Optimize an AD Function Object Tape
operator
     AD numeric binary 5.4: Binary Numeric Operators With an AD Result
     AD unary 5.3: Unary Plus and Minus Operators
     bool a_float binary example 5.6.1: a_float Comparison Operators: Example and Test
     bool binary 5.6: Binary Comparison Operators
     computed assignment 5.5: Computed Assignment Operators
     computed assignment example 5.5.1: Computed Assignment Operators: Example and Test
     numeric binary example 5.4.1: Binary Numeric Operators With an AD Result: Example and Test
optimize 6.8: Optimize an AD Function Object Tape
     build 2.f.b: Installing pycppad: Building.Optimized
     example 6.8.1: Optimize Function Object: Example and Test
order
     forward one example 6.4.2: Forward Order One: Example and Test
     forward zero example 6.4.1: Forward Order Zero: Example and Test
     reverse one example 6.5.1: Reverse Order One: Example and Test
     reverse two example 6.5.2: Reverse Order Two: Example and Test
P
path
     python 2.i: Installing pycppad: Python Path
plus
     binary 5.4: Binary Numeric Operators With an AD Result
     unary 5.3: Unary Plus and Minus Operators
pycppad : pycppad-20140710: A Python Algorithm Derivative Package
     build 2.f: Installing pycppad: Building
     download 2.c: Installing pycppad: Downloading
     extract 2.d: Installing pycppad: Unpacking
     install 2.h: Installing pycppad: Installing
     install 2: Installing pycppad
     license 10: License
     test 2.g: Installing pycppad: Testing
     unpack 2.d: Installing pycppad: Unpacking
python
     AD : pycppad-20140710: A Python Algorithm Derivative Package
     bug distutils 2.f.c: Installing pycppad: Building.Bug in Python Distutils
     path 2.i: Installing pycppad: Python Path
R
Runge-Kutta
     ODE solver 8: Fourth Order Runge Kutta
range
     direction derivative 6.5: Reverse Mode: Derivative in One Range Direction
recording
     abort AD 6.3: Abort a Recording of AD Operations
     start 6.1.a: Create an Independent Variable Vector: Syntax
     stop 6.2: Create an AD Function Object
reverse 6.5: Reverse Mode: Derivative in One Range Direction
     order one example 6.5.1: Reverse Order One: Example and Test
     order two example 6.5.2: Reverse Order Two: Example and Test
runge_kutta_4
     correct 8.1: runge_kutta_4 A Correctness Example and Test
     evaluate solution 8.3: runge_kutta_4 With C++ Speed: Example and Test
     example 8.2: runge_kutta_4 An AD Example and Test
S
sequence
     optimize operation 6.8.1: Optimize Function Object: Example and Test
     optimize operations 6.8: Optimize an AD Function Object Tape
sin 5.7: Standard Math Unary Functions
sinh 5.7: Standard Math Unary Functions
speed
     optimize 6.8: Optimize an AD Function Object Tape
     python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
sqrt 5.7: Standard Math Unary Functions
standard
     example math 5.7.1: Standard Math Unary Functions: Example and Test
start
     recording 6.1.a: Create an Independent Variable Vector: Syntax
stop
     recording 6.2: Create an AD Function Object
T
tan 5.7: Standard Math Unary Functions
tanh 5.7: Standard Math Unary Functions
tape
     optimize 6.8: Optimize an AD Function Object Tape
     python function 8.3: runge_kutta_4 With C++ Speed: Example and Test
test
     pycppad 2.g: Installing pycppad: Testing
times 5.4: Binary Numeric Operators With an AD Result
two
     AD levels 7: Using Two Levels of AD: Example and Test
     order reverse example 6.5.2: Reverse Order Two: Example and Test
U
unary
     + 5.3.1: Unary Plus and Minus Operators: Example and Test
     - 5.3.1: Unary Plus and Minus Operators: Example and Test
     AD operator 5.3: Unary Plus and Minus Operators
unpack
     pycppad 2.d: Installing pycppad: Unpacking
V
value 5.2: Create an Object With One Lower Level of AD
     example 5.2.1: value: Example and Test
variable
     ad 5: AD Variable Methods
variables
     dependent 6.2: Create an AD Function Object
     independent 6.1.a: Create an Independent Variable Vector: Syntax
version
     feature 2009 9.4: Extensions, Bug Fixes, and Changes During 2009
Z
zero
     order forward example 6.4.1: Forward Order Zero: Example and Test

13: External Internet References
Reference Location
- http://codespeak.net/py/dist/9.4.i: whats_new_09#01-29
_printable.htm: pycppad
_printable.xml: pycppad
http://code.google.com/p/winpython/2.a: install#Other Instructions
http://docs.python.org/distutils/2.f.c: install#Building.Bug in Python Distutils
http://docs.python.org/library/distutils.html9.1.a: whats_new_12#10-19
http://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html9.a: whats_new#2014-07-10
http://en.wikipedia.org/wiki/Automatic_differentiationc: pycppad#Purpose
http://legacy.python.org/dev/peps/pep-0238/5.4.f: ad_numeric#Example
http://list.coin-or.org/pipermail/cppad/2013q2/000309.html2.a: install#Other Instructions
http://www.boost.orgc: pycppad#Purpose
http://www.boost.org/doc/libs/1_44_0/libs/python/doc/index.html9.3.c: whats_new_10#11-30
http://www.coin-or.org/CppAD/CppAD/c: pycppad#Purpose
http://www.coin-or.org/CppAD/Doc/whats_new.htm9.2.e: whats_new_11#03-20
http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-305.8.f: abs#Directional Derivative
http://www.coin-or.org/CppAD/Doc/whats_new_11.htm#12-309.1.d: whats_new_12#04-16
http://www.coin-or.org/download/source/CppAD/9.3.d: whats_new_10#03-18
http://www.seanet.com/~bradbell/pycppad-20140710.tar.gz2.c: install#Downloading
pycppad.htm: pycppad
pycppad.xml: pycppad
www.cygwin.com9.3.b: whats_new_10#12-05