more work on Extending lecture...

Author: ChrisBarker-NOAA

week-08/extensions/code/ctypes/test_add.py

@@ -40,14 +40,25 @@
 ## prototype for pow():
 ## double pow ( double x, double y )   
 
-print "This should be 81:",
+print "This should be 81.0:",
 print libm.pow(ctypes.c_double(3), ctypes.c_double(4))
 
 ## need to set the return type!
 libm.pow.restype = ctypes.c_double
-print "This should be 81:",
+print "This should be 81.0:",
 print libm.pow(ctypes.c_double(3), ctypes.c_double(4))
 
+## if you are going to call the same function a lot,
+## you can specify the arument types:
+
+libm.pow.restype = ctypes.c_double
+libm.pow.argtypes = [ctypes.c_double, ctypes.c_double]
+print "This should be 81.0:",
+print libm.pow(3, 4.0)
+
+## much easier!
+
+
 
 
 

week-08/extensions/source/extensions.rst

@@ -207,9 +207,11 @@ Building your extension
       description='simple c extension for an example',
       ext_modules=[Extension('add', sources=['add.c'])],
   )
+
 Run the setup.py::
 
   python setup.py build_ext --inplace
+
 (you can also just do ``install`` or ``develop`` if you want to properly installed)
 
 Run the tests
@@ -319,3 +321,301 @@ Example::
 Further reading:
 
 http://docs.python.org/2/library/ctypes.html
+
+
+Calling functions with ctypes
+==============================
+
+The shared lib must be loaded::
+
+    add = ctypes.cdll.LoadLibrary("add.so")
+
+An already loaded lib can be loaded with:
+
+    libc = ctypes.CDLL("/usr/lib/libc.dylib")
+
+ctypes comes with a utility to help find libs::
+
+    ctypes.util.find_library(name)
+
+(good for system libs)
+
+.. nextslide::
+
+Once loaded, a ctypes wrapper around a c function can be called directly::
+
+    print add.add(3,4)
+
+But....
+
+
+C is statically typed -- once compiled, the function must be called with the correct types.
+
+ctypes Data Types
+=================
+
+ctypes will auto-translate these native types:
+
+  - ``None``
+  - int
+  - byte strings (``bytes()``, ``str()``)
+  - ``unicode`` (careful! unicode is ugly in C!)
+
+These can be directly used as parameters when calling C functions.
+
+.. nextslide::
+
+Most types must be wrapped in a ctypes data type::
+
+    printf("An int %d, a double %f\n", 1234, c_double(3.14))
+
+There are ctypes wrappers for all the "standard" C types
+
+http://docs.python.org/2/library/ctypes.html#fundamental-data-types
+
+
+You can also do pointers to types::
+
+    a_lib.a_function( ctypes.byref(c_float(x)))
+
+http://docs.python.org/2/library/ctypes.html#passing-pointers-or-passing-parameters-by-reference
+
+.. nextslide:: C structs
+
+You can define C structs::
+
+  >>> class POINT(ctypes.Structure):
+  ...     _fields_ = [("x", ctypes.c_int),
+  ...                 ("y", ctypes.c_int)]
+  ...
+  >>> point = POINT(10, 20)
+  >>> print point.x, point.y
+  10 20
+  >>> point = POINT(y=5)
+  >>> print point.x, point.y
+  0 5
+
+.. nextslide:: Custom Python Classes 
+
+You can define how to pass data from your custom classes to ctypes:
+
+Define an ``_as_parameter_`` attribute (or property)::
+
+  class MyObject(object):
+      def __init__(self, number):
+          self._as_parameter_ = number
+
+  obj = MyObject(32)
+  printf("object value: %d\n", obj)
+
+http://docs.python.org/2/library/ctypes.html#fundamental-data-types
+
+(careful with types here!)
+
+.. nextslide:: Return Types
+
+To defining the return type, define the ``restype`` attribute.
+
+Pre-defining the entire function signature::
+
+  libm.pow.restype = ctypes.c_double
+  libm.pow.argtypes = [ctypes.c_double, ctypes.c_double]
+
+And you can just call it like a regular python function -- ctypes will type check/convert at run time::
+
+  In [10]: libm.pow('a string', 4)
+  ---------------------------------------------------------------------------
+  ArgumentError                             Traceback (most recent call last)
+  <ipython-input-10-01be690a307b> in <module>()
+  ----> 1 libm.pow('a string', 4)
+
+  ArgumentError: argument 1: <type 'exceptions.TypeError'>: wrong type
+
+Some more features
+===================
+
+Defining callbacks into Python code from C::
+
+    ctypes.CFUNCTYPE(restype, *argtypes, use_errno=False, use_last_error=False)
+
+http://docs.python.org/2/library/ctypes.html#ctypes.CFUNCTYPE
+
+Numpy provides utilities for numpy arrays:
+
+http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.ctypes.html
+
+(works well for C code that takes "classic" C arrays)
+
+
+Summary:
+========
+
+* ``ctypes`` allows you to call shared libraries:
+
+  - Your own custom libs
+  - System libs
+  - Proprietary libs
+
+* Supports almost all of C:
+ 
+ - Custom data types
+
+   - structs 
+   - unions
+   - pointers
+
+ - callbacks
+
+.. nextslide::
+
+* Upside:
+
+  - You can call system libs with little code
+  - You don't need to compile anything
+
+    - at least for system and pre-compiled libs
+
+* Downsides:
+
+  - You need to specify the interface
+
+    - and it is NOT checked for you!
+
+  - Translation is done on the fly at run time
+
+    - performance considerations
+
+LAB
+====
+
+In ``code/ctypes`` you'll find ``add.c``
+
+You can build a shared lib with it with ``make``
+(``make.bat``) on Windows.
+
+``test_ctypes.py`` will call that dll, and a few system dlls.
+
+* Take a look at what's there, and how it works.
+* add another function to add.c, that takes different types (maybe divide?)
+* rebuild, and figure out how to call it with ctypes.
+
+* Try calling other system functions with ctypes.
+
+
+*******
+Cython
+*******
+
+A Python like language with static types which compiles down to C code for Python extensions.
+
+Cython
+=======
+
+* Can write pure python
+  - Fully understands the python types
+
+* With careful typing -- you get pure C (and pure C speed)
+
+* Can also call other C code: libraries or compiled in.
+
+* Used for custom Python extensions and/or call C and C++ code.
+
+.. nextslide::
+
+Further reading:
+
+Web site:
+
+  http://www.cython.org/
+
+Documentation:
+
+  http://docs.cython.org/
+
+Wiki:
+
+  https://github.com/cython/cython/wiki
+
+
+
+Developing with Cython
+========================
+
+First, install cython with::
+
+ ``pip install cython``
+
+Cython files end in the .pyx extension. An example add.pyx::
+
+  def add(int x, int y):
+      cdef int result=0
+      result = x + y
+      return result
+
+
+Basic Cython
+=============
+
+Cython functions can be declared three ways::
+
+  def foo # callable from Python
+
+  cdef foo # only callable from Cython/C
+  
+  cpdef foo # callable from both Cython and Python
+
+Once your .pyx file is created, it is converted to C via
+
+
+cython cy_add.pyx
+Generate "annoted" C code in HTML
+
+
+cython -a cy_add.pyx
+To build your Python extension:
+
+
+python cy_setup.py build_ext --inplace # note Cython defines its' own build_ext in Cython.Distutils.build_ext
+
+Cython can compile pure Python code to C to provide a performance improvement
+
+
+
+::
+
+
+
+
+Consider a more expensive function
+
+
+def f(x):
+    return x**2-x
+
+def integrate_f(a, b, N):
+    s = 0
+    dx = (b-a)/N
+    for i in range(N):
+        s += f(a+i*dx)
+    return s * dx
+
+
+Impovements with static typing
+
+Convert the dynamically typed variables to static types and measure performance improvement before and after
+Can static types and dynamic types be mixed?
+Check the performance in converting the function type to static (see here)
+Use cython -a to compare the generated C code in all cases
+
+def f(x):
+    return x**2-x
+
+def integrate_f(a, b, N):
+    s = 0
+    dx = (b-a)/N
+    for i in range(N):
+        s += f(a+i*dx)
+    return s * dx
+
+
+