======================
Writing Gamera Plugins
======================

Introduction
============

The functionality of Gamera can be extended by writing plugins in
either C++ or Python.  A plugin is simply a set of methods (which are
automagically added to the ``Image`` class) or free-standing
functions.  Plugins are technically just Python modules, but with more
information that allows for easier wrapping and compilation of C++
methods and to support all kinds of automatic things in the graphical
user interface.

Plugins can also be grouped together, with other tools, into toolkits.
Toolkits provide higher-level workflow framework for end-to-end
document recognition by joining together a number of steps from
various plugins.  Toolkits are a whole other discussion, so see the
`writing Gamera toolkits`__ for more information.

.. __: writing_toolkits.html

Before writing any plugin, you should make sure there isn't already a
plugin included that does what you want.  Look at the `list of plugins
included with Gamera`__.

.. __: plugins.html

The files involved
------------------

Each plugin is made up of two files: 

1. A Python file that describes each method in the plugin and the
   plugin itself.  If any methods are "pure Python", they can also be
   defined here, or they can just delegate to functions in other
   Python modules.

2. Optionally, a C++ header file containing implementations of any C++
   methods of the plugin.  This is a header file (``.hpp``) and not an
   implementation file (``.cpp``) because the code you write will be templatized
   and the concrete methods and the glue code connecting to Python to
   C++ will be generated automatically at compile time by the Gamera
   build system.  (What templates are is beyond the scope of this
   document, but it's covered very well in [Stroustrup1997]_.)

Plugins in the Gamera source tree
`````````````````````````````````
The Python metadata files are stored in ``./gamera/plugins/`` and the
C++ source files are stored in ``./include/plugins``.

If you keep these files in the proper directories, they will be
automatically picked up by the build system and compiled.  When Gamera
is started up, it will search the ``./gamera/plugins`` directory and load
all plugins.

Plugins in a toolkit
````````````````````

In a toolkit, the Python metadata files are stored in
``./gamera/toolkits/my_toolkit/plugins/`` and the C++ source files are
stored in ``./include/plugins``, both rooted at the top of your
toolkit directory.

Plugin modules included in toolkits will have to be explicitly
imported before they are available.

A simple example
================

Plugin metadata
---------------

Let's look at a simple metadata file, ``example.py``.  Each method is
described by creating a class that inherits from
``gamera.plugin.PluginFunction`` and defining a number of special
members.  The whole plugin is described by a class that inherits from
``gamera.plugin.PluginModule``:

.. code:: Python

  from gamera.plugin import *
  import _example

  # C++ method
  class volume(PluginFunction):
      """
      Returns the ratio of black pixels to white pixels within the
      bounding box of the image.
      """
      self_type = ImageType([ONEBIT])
      return_type = Float("volume")
      doc_examples = [(ONEBIT,)]

  class ExampleModule(PluginModule):
      category = "Example"
      cpp_headers=["example.hpp"]
      functions = [volume]
      author = "Michael Droettboom and Karl MacMillan"
      url = "http://gamera.informatik.hsnr.de/"
  module = ExampleModule()

Okay, now let's break it down.

The ``gamera.plugin`` module contains all of the utilities necessary to
create Gamera plugins, so the first thing we do is import it:

.. code:: Python

  from gamera.plugin import *

Next, we import the C++ (compiled object file) side of the plugin,
(described in the next section) which *always* has the same name as
the Python metadata module, except with a leading underscore:

.. code:: Python

  import _example

Let's start by describing a minimal C++ method.  All methods and
functions in a plugin are described using a class that inherits from
``gamera.plugin.PluginFunction``:

.. code:: Python

  # C++ method
  class volume(PluginFunction):

Each plugin can (and should) be documented in the usual `Python
docstring`__ way:

.. code:: Python

      """
      Returns the ratio of black pixels to white pixels within the
      bounding box of the image.
      """

.. __: http://www.python.org/doc/current/tut/node6.html

On a related note, you can also have the documentation system
(``doc/gendoc.py``) generate an example automatically.  See
`documenting and unit-testing Plugin functions`_. 

Next, we define ``self_type``, which is the type of object this method
can be called on.  If ``self_type`` is an ``ImageType``, the method
will automatically be added to all ``Image`` objects in Gamera
whenever the plugin is imported.  Within the ``ImageType`` specifier,
you can choose which types of pixels are supported using a list of
pixel type names. Valid options are ``ONEBIT``, ``GREYSCALE``,
``GREY16``, ``FLOAT`` and ``RGB`` (these are all constants imported
from the ``gamera.plugin module``):

.. code:: Python

      self_type = ImageType([ONEBIT])

You can also optionally define ``return_type``.  This specifier is
used to generate a variable name for the result in the GUI, and so the
C++ wrapping mechanism knows how to return the result to Python.  If
you don't specify a return type, Gamera assumes there is no return
result:

.. code:: Python

      return_type = Float("volume")

Obviously, this is a very simple plugin method with no arguments.
Some more involved examples are given below.  In the meantime, let's
look at how this method is contained in a plugin module.

For each plugin module, you also need a class to describe the entire
plugin.  There may be *only one* of these classes per plugin.  This is
done in a similar manner to how the methods are described.

There is a class that inherits from ``gamera.plugin.PluginModule``:

.. code:: Python

  class ExampleModule(PluginModule):

You can specify a category for the plugin's methods on
the context (right-click) menu in the GUI:

.. code:: Python

    category = "Example"

If you have any C++ methods (which we do in this case), you must
specify the C++ header files to include which contain the
corresponding method's source code:

.. code:: Python

    cpp_headers=["example.hpp"]

You must also list all of the plugins and methods in the file so they
can be generated and loaded:

.. code:: Python

    functions = [volume]

Optionally, the author names and a URL for more information can be
specified:

.. code:: Python

    author = "Michael Droettboom and Karl MacMillan"
    url = "http://gamera.informatik.hsnr.de/"

Lastly, we create an instance of this class so the module loader can
do its work:

.. code:: Python

  module = ExampleModule()

C++ code
--------

Since the ``volume`` method needs to look at individual pixels, it is
likely going to be much faster written in C++ than in Python.  Below
is the corresponding ``example.hpp`` that contains the C++
implementation:

.. code:: CPP

  #include "gamera.hpp"

  using namespace Gamera;

  template<class T>
  float volume(const T &m) {
    unsigned int count = 0;
    typename T::const_vec_iterator i = m.vec_begin();
    for (; i != m.vec_end(); i++)
      if (is_black(*i))
        count++;
    return (feature_t(count) / (m.nrows() * m.ncols()));
  }

Most of the declarations needed for Gamera are in ``gamera.hpp``, and
all of that stuff is in the ``Gamera`` namespace, to prevent name
collisions.  You may find it most convenient to just put ``using
namespace Gamera`` at the top of your plugin file, rather than
specifying ``Gamera::...`` everywhere:

.. code:: CPP

  #include "gamera.hpp"
 
  using namespace Gamera;

Next we get to the function itself.  Note that it is templatized.
Since it is our goal to write a single algorithm that may work on
multiple image types, *all* plugin methods are templatized, and the
instantiations of these templates are generated by the Gamera build
system at compile-time based on the ``self_type`` specifier in the
method metadata class (that we specified in ``example.py``).  See how
the first argument ``self`` is templatized as ``T`` so that any
(image) type can be passed in.  The body of the function used the
`Gamera C++ Image API`__ to access and examine the individual pixels:

.. code:: CPP

    template<class T>
    float volume(const T &m) {
      unsigned int count = 0;
      typename T::const_vec_iterator i = m.vec_begin();
      for (; i != m.vec_end(); i++)
        if (is_black(*i))
          count++;
      return (feature_t(count) / (m.nrows() * m.ncols()));
    }

.. __: image_api.html

Building the plugin
-------------------

Okay, so now we're done with the minimal plugin, but obviously
something more has to happen in order to access the C++ code from Python.
Fortunately, that is all done automatically by the Gamera build
system.  If the ``example.py`` is placed in the ``./gamera/plugins``
directory, the build system will automatically find it, use the
metadata to generate a *wrapper* to access ``example.hpp``, and compile
everything.  The next time Gamera is run, the plugin will
automatically be loaded.  The plugin author does not have to learn
about the intricacies of the `Python/C API`__.

.. __: http://www.python.org/doc/current/api/api.html

But, for the sake of some sick curiosity, the generated code looks
something like:

.. code:: CPP

  #include <string>
  #include <stdexcept>
  #include "Python.h"
  #include <list>
  #include "gameramodule.hpp"

  init_features (void)
  {
    Py_InitModule ("_features", _features_methods);
  }

  #include "features.hpp"
 
  using namespace Gamera;

  extern "C"
  {
    void init_example (void);
    static PyObject *call_volume (PyObject * self, PyObject * args);
    static PyMethodDef _features_methods[] =
      { {"volume", call_volume, METH_VARARGS} };

    static PyObject *call_volume(PyObject * self, PyObject * args)
    {
      PyObject *real_self;
      Image *real_self_image;

      FloatVector *return_value = 0;

      if (PyArg_ParseTuple(args, "O", &real_self) <= 0)
        return 0;
      if (!is_ImageObject(real_self)) {
	PyErr_SetString(PyExc_TypeError, "Object is not an image as expected!");
	return 0;
      }
      real_self_image = ((Image *) ((RectObject *) real_self)->m_x);
      image_get_fv(real_self, &real_self_image->features, &real_self_image->features_len);
      try {
      switch (get_image_combination (real_self)) {
	case ONEBITRLEIMAGEVIEW:
	  return_value = volume(*((OneBitRleImageView *) real_self_image));
	  break;
	case RLECC:
	  return_value = volume(*((RleCc *) real_self_image));
	  break;
	case CC:
	  return_value = volume(*((Cc *) real_self_image));
	  break;
	case ONEBITIMAGEVIEW:
	  return_value = volume (*((OneBitImageView *) real_self_image));
	  break;
	default:
	  PyErr_SetString (PyExc_TypeError,
			   "Image types do not match function signature.");
	  return 0;
	}
      }
      catch (std::exception & e)
      {
        PyErr_SetString (PyExc_RuntimeError, e.what());
        return 0;
      }
      PyObject *array_init = get_ArrayInit();
      if (array_init == 0)
        return 0;
      PyObject *str = PyString_FromStringAndSize((char *) (&((*return_value)[0])),
			     	  return_value->size () * sizeof (double));
      PyObject *array = PyObject_CallFunction(array_init, "sO", "d", str);
      delete return_value;
      return array;
    }
    DL_EXPORT (void) init_example (void)
    {
      Py_InitModule ("_example", _example_methods);
    }
  }

**Aren't you glad you don't have to write something like that every
time!**

Advanced features
=================

Specifying arguments
--------------------

Of course, many plugin methods will need to have arguments.  See this
``resize_copy`` method, for example:

.. code:: Python

  # C++ image method with some arguments
  class resize_copy(PluginFunction):
      """
      Copies and resizes an image. In addition to size the type of
      interpolation can be specified to allow tradeoffs between speed
      and quality.
      """
      category = "Utility/Copy"
      self_type = ImageType([ONEBIT, GREYSCALE, GREY16, FLOAT, RGB])
      args = Args([Int("nrows"), Int("ncols"),
                  Choice("Interpolation Type", ["None", "Linear", "Spline"])])
      return_type = ImageType([ONEBIT, GREYSCALE, GREY16, FLOAT, RGB])

And the corresponding C++ declaration:

.. code:: CPP

  template<class T>
  Image* resize_copy(T& image, int nrows, int ncols, int resize_quality);

The ``args`` member variable specifies a list of the arguments that
are passable to the method.  Note that this does not include the first
"argument" to the C++ function, which always corresponds to
``self_type``.  This specification is used to generate wrapper code,
and also to generate dialog boxes in the GUI.  The format of these
argument lists are documented in `Specifying arguments`__.

.. __: args.html

The *args* parameter in the plugin prototype allows the specification of
default values for arguments. These are however used only in the GUI for
the argument dialog box. If you need an actual default argument for your
plugin function, you must define the ``__call__`` method in your plugin, e.g.

.. code:: Python

      # wrapper for passing a default argument
      def __call__(self, nrows, ncols, interpolation="Linear"):
          return _example.resize_copy(self, nrows, ncols, interpolation)
    __call__ = staticmethod(__call__)

``_example`` must be replaced by the actual name of your source file plus
a leading underscore.

Free functions
--------------

It doesn't always make sense to have everything be a method of images.
For example, you may want to create a function that requires a list of
images as input.  Fortunately, you can still use the plugin system to
automate the wrapping/building process, while foregoing the automatic
inclusion of the method in the Image class and on the right-click
context menu. It's as simple as setting ``self_type`` to ``None`` in
the metadata object:

.. code:: Python

  # C++ free function
  class union_images(PluginFunction):
      """
      Creates a new image by overlaying all the images in the given list.
      """
      self_type = None
      args = Args([ImageList('list_of_images')])
      return_type = ImageType([ONEBIT])

As these functions are not image methods, but standalone *callable* classes,
you additionally must create an instance of this class in the same python
metadata file:

.. code:: Python

  union_images = union_images()

Pure Python methods
-------------------

Sometimes there is not much efficiency to be gained by writing the
plugin method in C++, or you want the flexibility of Python for
experimentation.  In that case you can implement the method in pure
Python.  Everything else is the same, except you add a ``__call__``
method with the Python implementation.  It is important that this
method is a ``staticmethod``, since the first ``self`` argument is
going to be an ``Image`` object and not a ``PluginFunction`` object:

.. code:: Python

  # Python image method
  class area(PluginFunction):
      """
      Returns the aspect ratio of the bounding box of the image.
      """
      self_type = ImageType([ONEBIT, GREYSCALE, GREY16, FLOAT, RGB])
      return_type = Float("area")
      pure_python = True
      def __call__(self):
          return float(self.ncols) / float(self.nrows)
      __call__ = staticmethod(__call__)

Free pure Python functions
--------------------------

Since the plugin modules are also just regular Python modules
underneath, it is of course possible to just add ordinary Python
functions as well:

.. code:: Python

  # Python free function
  def filter_small_images(l):
      return [x for x in l if x.ncols > 2 and x.nrows > 2]

Raising exceptions
------------------

The convention in Gamera is to use exceptions for error conditions
rather than by using error codes.

C++ exceptions are automatically propagated to Python.  (All C++
exception types will be converted to Python ``RuntimeError``.)

.. code:: CPP

   throw std::runtime_error("Input is out of range");

From pure Python functions, the standard Python exception mechanism
can be used:

.. code:: Python

   raise RuntimeError("Input is out of range")

Progress bars
-------------

This section describes how to display a progress bar dialog from a
long-running plugin. When the GUI is running, the progress
will be displayed in a window:

  .. image:: images/progress_bar.png

It is also possible to display a progress bar made from text characters
in the console, when the GUI is not running. To make it appear, set
the config option *progress_bar* to ``True`` as follows:

.. code:: Python

   from gamera.config import config
   config.set("progress_bar",True)

Progress bars will add some overhead when displayed, so they
only make sense for plugins that take a long time to complete.
Supporting progress bars adds very minimal overhead when they are not
displayed.


Progress Bars in Python
```````````````````````

To create a progress bar that is a message box in the GUI and
a text line in a non GUI script, you can use the ``ProgressFactory``:

.. code:: Python

   from gamera.util import ProgressFactory
   progress = ProgressFactory("Title", length, numsteps=0)

where *length* is the total number of which the progress fraction is to
be shown. The optional argument *numsteps* can be useful to reduce
the overhead by only updating the progress bar in *numsteps* discrete steps;
when zero, every update call will result in an update of the progress bar.

To update the progress bar, there are two alternative methods:


 - ``.step()`` increases the progress counter by one. Whether the
   displayed progress bar actually is updated depends on *numsteps*.

 - ``.update(steps, length)`` sets the progress bar to the
   *steps*/*length* fraction.


Progress Bars in C++ Plugins
````````````````````````````

To create a plugin method with a progress bar, simply set the
``progress_bar`` member to the message that will be displayed in the
progress box.

.. code:: Python

  class cc_analysis(Segmenter):
    ...
    progress_bar = "Generating connected components"
    ...

Add an extra argument to the C++ function that takes an object of type
``ProgressBar``.  This can be a default argument, to make it easier to
call the plugin function code without requiring a ``ProgressBar`` instance.
Creating a ``ProgressBar`` with no constructor arguments creates a
dummy ``ProgressBar`` object where all methods are ignored.

.. code:: CPP

  template<class T>
  ImageList* cc_analysis(T& image, ProgressBar progress_bar = ProgressBar()) {
    ...
  }

The progress bar window will automatically disappear when the function returns.

There are essentially two ways to update the progress bar:

 - Call ``.set_length(*length*)`` to set the number of steps that will be
   performed, and then call ``.step()`` to increase the step.

 - Call ``.update(*num*, *den*)`` to say that *num* of *den* steps
   have completed.  ``.update`` is useful when the number of steps can
   not be pre-determined.


Documenting and unit-testing Plugin functions
---------------------------------------------

The docstring of each ``PluginFunction`` class is used like a regular
Python docstring, but also has the following advantages in Gamera:
 
  - It will be included in the automatically generated HTML
    documentation.

  - It is displayed in the Documentation pane in the Gamera shell
    window.

  - It is displayed in the automatically-generated dialog box for the
    plugin.

The docstrings should be formatted in reStructuredText_, which is
becoming a de-facto Python standard for documentation, as well as
being rather easy to read and use.

.. _reStructuredText: http://structuredtext.sourceforge.net

The Gamera documentation can be regenerated by going to the ``doc``
directory (in the source distribution) and running the ``gendoc.py``
script::

  python gendoc.py

In addition to text, image examples can be generated on-the-fly and
included in the documentation using the ``doc_examples`` member.  The
``doc_examples`` mechanism is also used to write rudimentary
unit-tests for Gamera's `unit-testing framework`__.

.. __: unit_testing.html

The ``doc_examples`` member is a list of tuples or functions:

- If the element is a tuple, it is a list of arguments that will be
  passed into the plugin method to create an example.  Where image
  arguments are expected, image type identifiers can be used, which
  will load a standard image from disk and use it.  For example::

    (ONEBIT, 52, 32)

  will use the standard OneBit image and the arguments of ``52, 32``.

- If the element is a function, that function will be called to create
  the example.  The function will be passed one argument, ``images``,
  which is a dictionary of the standard Gamera example images.  Any
  images or values returned will be included in the documentation.
  Any exceptions raised by the function will be logged by the
  unit-testing framework when it is run.  For example the following
  loads the standard RGB and GreyScale images, clips them
  appropriately, adds them together, and then returns all of them for
  inclusion in the documentation.

.. code:: Python

    def __doc_example1__(images):
        rgb = images[RGB]
        greyscale = images[GREYSCALE]
        clipped = rgb.clip_image(greyscale)
        return [clipped, greyscale, 
	        clipped.add_images(greyscale.to_rgb(), False)]
    doc_examples = [__doc_example1__]


Examples
````````

The following ``doc_examples`` specifier (from simple_sharpen_)
produces two examples, one on the standard GREYSCALE image, and
another on the standard RGB image, using different values for the
*sharpening_factor*.

.. code:: Python

  doc_examples = [(GREYSCALE, 1.0), (RGB, 3.0)]

.. _simple_sharpen: convolution.html#simple-sharpen

For draw_bezier_, a custom example function was used, which does
not load a standard image.

.. code:: Python

  def __doc_example1__(images):
      from random import randint
      from gamera.core import Image, Dim
      image = Image((0, 0), Dim(100, 100), RGB, DENSE)
      for i in range(10):
          image.draw_bezier((randint(0, 100), randint(0, 100)),
                            (randint(0, 100), randint(0, 100)),
                            (randint(0, 100), randint(0, 100)),
                            (randint(0, 100), randint(0, 100)),
                            RGBPixel(randint(0, 255), 
              	            randint(0,255), 
 			    randint(0, 255)))
      return image
  doc_examples = [__doc_example1__]

.. _draw_bezier: draw.html#draw-bezier


Feature generators: A special kind of plugin
============================================

Plugin methods that take an image as input and generate some floating
point features from it are called "feature generators".  The resulting
floating point features are used by the classifier to classify images.

For efficiency reasons, feature generator functions are implemented
slightly differently from a regular ``PluginFunction``.  Rather than
returning the features as a return value, which would require a memory
copy into the image's feature vector, feature generators write
directly to a buffer that is passed in as an argument.

As an example, let's look at the ``nholes`` feature generator.  The
Python metadata is the same as you would expect, except the member
``feature_function`` is set to ``True``.  This will tell the build
system to treat this plugin method as a feature generator with the
different return value behavior.  The ``return_type`` must be a
``FloatVector``, where ``length`` indicates the number of feature
values that are generated.

.. code:: Python

   class nholes(PluginFunction):
       """
       Returns the average number of transitions from white to black
       in each row or column.
   
       The elements of the returned ``FloatVector`` are:

       0. vertical
       1. horizontal
       
       These features are scale invariant.
       """
       self_type = ImageType([ONEBIT])
       return_type = FloatVector(length=2)
       feature_function = True
       doc_examples = [(ONEBIT,)]

On the C++ side, the function takes two arguments: the image, and a
pointer to a floating point buffer.  Note that the return type is ``void``.

.. code:: CPP

  template<class T>
  void nholes(T &m, feature_t* buf) {
    int vert, horiz;
    
    vert = nholes_1d(m.col_begin(), m.col_end());
    horiz = nholes_1d(m.row_begin(), m.row_end());
    
    *(buf++) = (feature_t)vert / m.ncols();
    *buf = (feature_t)horiz / m.nrows();
  }

(The C++ function ``nholes_1d`` is where all the real work gets done,
and is not important for this illustration.)  Note how the result of the
function is copied directly into the buffer.  

.. note:: It is extremely important not to write more values to the
  buffer than is defined in the metadata ``return_value``.  Doing so
  could cause Python/Gamera to behave erratically or segfault.

It is also possible to write a feature generator in pure Python.

.. code:: Python

   class nholes(PluginFunction):
       self_type = ImageType([ONEBIT])
       return_type = FloatVector(length=2)
       feature_function = True
       doc_examples = [(ONEBIT,)]

       def __call__(self, index):
           buffer = self.features
       
           # Do some processing to get values...
	                 
	   buffer[index] = result1
           buffer[index + 1] = result2  
       __call__ = staticmethod(__call__)


Further reading
===============

- `Writing C++ plugin methods that processes Gamera images`__

.. __: image_api.html

- `Specifying arguments`__

.. __: args.html

- `Passing/returning custom data types to/from plugins`__

.. __: plugins_custom_types.html

- `Grouping plugins together into toolkits`__

.. __: writing_toolkits.html

- And of course, there's lots of examples in the Gamera code itself!

References
==========

.. [Stroustrup1997] Stroustrup, B. 1997. *The C++ Programming
   Language: Third Edition.*  Reading, MA: Addison-Wesley.

