Extending SWIG to support new languages

# 34 Extending SWIG to support new languages

- Introduction
- Prerequisites
- The Big Picture
- Execution Model
  - Preprocessing
  - Parsing
  - Parse Trees
  - Attribute namespaces
  - Symbol Tables
  - The %feature directive
  - Code Generation
  - SWIG and XML
- Primitive Data Structures
  - Strings
  - Hashes
  - Lists
  - Common operations
  - Iterating over Lists and Hashes
  - I/O
- Navigating and manipulating parse trees
- Working with attributes
- Type system
  - String encoding of types
  - Type construction
  - Type tests
  - Typedef and inheritance
  - Lvalues
  - Output functions
- Parameters
- Writing a Language Module
  - Execution model
  - Starting out
  - Command line options
  - Configuration and preprocessing
  - Entry point to code generation
  - Module I/O and wrapper skeleton
  - Low-level code generators
  - Configuration files
  - Runtime support
  - Standard library files
  - Examples and test cases
  - Documentation
  - Prerequisites for adding a new language module to the SWIG distribution
  - Coding style guidelines
- Typemaps
  - Proxy classes
- Guide to parse tree nodes

## 34.1 Introduction

This chapter describes SWIG's internal organization and the process by which new target languages can be developed. First, a brief word of warning---SWIG is continually evolving. The information in this chapter is mostly up to date, but changes are ongoing. Expect a few inconsistencies.

Also, this chapter is not meant to be a hand-holding tutorial. As a starting point, you should probably look at one of SWIG's existing modules.

## 34.2 Prerequisites

In order to extend SWIG, it is useful to have the following background:

- An understanding of the C API for the target language.
- A good grasp of the C++ type system.
- An understanding of typemaps and some of SWIG's advanced features.
- Some familiarity with writing C++ (language modules are currently written in C++).

Since SWIG is essentially a specialized C++ compiler, it may be useful to have some prior experience with compiler design (perhaps even a compilers course) to better understand certain parts of the system. A number of books will also be useful. For example, "The C Programming Language" by Kernighan and Ritchie (a.k.a, "K&R") and the C++ standard, "ISO/IEC 14882 Programming Languages - C++" will be of great use.

Also, it is useful to keep in mind that SWIG primarily operates as an extension of the C++ type system. At first glance, this might not be obvious, but almost all SWIG directives as well as the low-level generation of wrapper code are driven by C++ datatypes.

## 34.3 The Big Picture

SWIG is a special purpose compiler that parses C++ declarations to generate wrapper code. To make this conversion possible, SWIG makes three fundamental extensions to the C++ language:

- Typemaps. Typemaps are used to define the conversion/marshalling behavior of specific C++ datatypes. All type conversion in SWIG is based on typemaps. Furthermore, the association of typemaps to datatypes utilizes an advanced pattern matching mechanism that is fully integrated with the C++ type system.
- Declaration Annotation. To customize wrapper code generation, most declarations can be annotated with special features. For example, you can make a variable read-only, you can ignore a declaration, you can rename a member function, you can add exception handling, and so forth. Virtually all of these customizations are built on top of a low-level declaration annotator that can attach arbitrary attributes to any declaration. Code generation modules can look for these attributes to guide the wrapping process.
- Class extension. SWIG allows classes and structures to be extended with new methods and attributes (the `%extend` directive). This has the effect of altering the API in the target language and can be used to generate OO interfaces to C libraries.

It is important to emphasize that virtually all SWIG features reduce to one of these three fundamental concepts. The type system and pattern matching rules also play a critical role in making the system work. For example, both typemaps and declaration annotation are based on pattern matching and interact heavily with the underlying type system.

## 34.4 Execution Model

When you run SWIG on an interface, processing is handled in stages by a series of system components:

- An integrated C preprocessor reads a collection of configuration files and the specified interface file into memory. The preprocessor performs the usual functions including macro expansion and file inclusion. However, the preprocessor also performs some transformations of the interface. For instance, `#define` statements are sometimes transformed into `%constant` declarations. In addition, information related to file/line number tracking is inserted.
- A C/C++ parser reads the preprocessed input and generates a full parse tree of all of the SWIG directives and C declarations found. The parser is responsible for many aspects of the system including renaming, declaration annotation, and template expansion. However, the parser does not produce any output nor does it interact with the target language module as it runs. SWIG is not a one-pass compiler.
- A type-checking pass is made. This adjusts all of the C++ typenames to properly handle namespaces, typedefs, nested classes, and other issues related to type scoping.
- A semantic pass is made on the parse tree to collect information related to properties of the C++ interface. For example, this pass would determine whether or not a class allows a default constructor.
- A code generation pass is made using a specific target language module. This phase is responsible for generating the actual wrapper code. All of SWIG's user-defined modules are invoked during this latter stage of compilation.

The next few sections briefly describe some of these stages.

### 34.4.1 Preprocessing

The preprocessor plays a critical role in the SWIG implementation. This is because a lot of SWIG's processing and internal configuration is managed not by code written in C, but by configuration files in the SWIG library. In fact, when you run SWIG, parsing starts with a small interface file like this (note: this explains the cryptic error messages that new users sometimes get when SWIG is misconfigured or installed incorrectly):

    %include "swig.swg"             // Global SWIG configuration
    %include "langconfig.swg"       // Language specific configuration
    %include "yourinterface.i"      // Your interface file

The `swig.swg` file contains global configuration information. In addition, this file defines many of SWIG's standard directives as macros. For instance, part of of `swig.swg` looks like this:

    ...
    /* Code insertion directives such as %wrapper %{ ... %} */

    #define %init        %insert("init")
    #define %wrapper     %insert("wrapper")
    #define %header      %insert("header")
    #define %runtime     %insert("runtime")

    /* Access control directives */

    #define %immutable   %feature("immutable","1")
    #define %mutable     %feature("immutable")

    /* Directives for callback functions */

    #define %callback(x) %feature("callback") `x`;
    #define %nocallback  %feature("callback");

    /* %ignore directive */

    #define %ignore         %rename($ignore)
    #define %ignorewarn(x)  %rename("$ignore:" x)
    ...

The fact that most of the standard SWIG directives are macros is intended to simplify the implementation of the internals. For instance, rather than having to support dozens of special directives, it is easier to have a few basic primitives such as `%feature` or `%insert`.

The `langconfig.swg` file is supplied by the target language. This file contains language-specific configuration information. More often than not, this file provides run-time wrapper support code (e.g., the type-checker) as well as a collection of typemaps that define the default wrapping behavior. Note: the name of this file depends on the target language and is usually something like `python.swg` or `perl5.swg`.

As a debugging aide, the text that SWIG feeds to its C++ parser can be obtained by running `swig -E interface.i`. This output probably isn't too useful in general, but it will show how macros have been expanded as well as everything else that goes into the low-level construction of the wrapper code.

### 34.4.2 Parsing

The current C++ parser handles a subset of C++. Most incompatibilities with C are due to subtle aspects of how SWIG parses declarations. Specifically, SWIG expects all C/C++ declarations to follow this general form:

    storage type declarator initializer;

`storage` is a keyword such as `extern`, `static`, `typedef`, or `virtual`. `type` is a primitive datatype such as `int` or `void`. `type` may be optionally qualified with a qualifier such as `const` or `volatile`. `declarator` is a name with additional type-construction modifiers attached to it (pointers, arrays, references, functions, etc.). Examples of declarators include `*x`, `**x`, `x[20]`, and `(*x)(int,double)`. The `initializer` may be a value assigned using `=` or body of code enclosed in braces `{ ... }`.

This declaration format covers most common C++ declarations. However, the C++ standard is somewhat more flexible in the placement of the parts. For example, it is technically legal, although uncommon to write something like `int typedef const a` in your program. SWIG simply doesn't bother to deal with this case.

The other significant difference between C++ and SWIG is in the treatment of typenames. In C++, if you have a declaration like this,

    int blah(Foo *x, Bar *y);

it won't parse correctly unless `Foo` and `Bar` have been previously defined as types either using a `class` definition or a `typedef`. The reasons for this are subtle, but this treatment of typenames is normally integrated at the level of the C tokenizer---when a typename appears, a different token is returned to the parser instead of an identifier.

SWIG does not operate in this manner--any legal identifier can be used as a type name. The reason for this is primarily motivated by the use of SWIG with partially defined data. Specifically, SWIG is supposed to be easy to use on interfaces with missing type information.

Because of the different treatment of typenames, the most serious limitation of the SWIG parser is that it can't process type declarations where an extra (and unnecessary) grouping operator is used. For example:

    int (x);         /* A variable x */
    int (y)(int);    /* A function y */

The placing of extra parentheses in type declarations like this is already recognized by the C++ community as a potential source of strange programming errors. For example, Scott Meyers "Effective STL" discusses this problem in a section on avoiding C++'s "most vexing parse."

The parser is also unable to handle declarations with no return type or bare argument names. For example, in an old C program, you might see things like this:

    foo(a,b) {
    ...
    }

In this case, the return type as well as the types of the arguments are taken by the C compiler to be an `int`. However, SWIG interprets the above code as an abstract declarator for a function returning a `foo` and taking types `a` and `b` as arguments).

### 34.4.3 Parse Trees

The SWIG parser produces a complete parse tree of the input file before any wrapper code is actually generated. Each item in the tree is known as a "Node". Each node is identified by a symbolic tag. Furthermore, a node may have an arbitrary number of children. The parse tree structure and tag names of an interface can be displayed using `swig -debug-tags`. For example:

    $ swig -c++ -python -debug-tags example.i
     . top (example.i:1)
     . top . include (example.i:1)
     . top . include . typemap (/r0/beazley/Projects/lib/swig1.3/swig.swg:71)
     . top . include . typemap . typemapitem (/r0/beazley/Projects/lib/swig1.3/swig.swg:71)
     . top . include . typemap (/r0/beazley/Projects/lib/swig1.3/swig.swg:83)
     . top . include . typemap . typemapitem (/r0/beazley/Projects/lib/swig1.3/swig.swg:83)
     . top . include (example.i:4)
     . top . include . insert (/r0/beazley/Projects/lib/swig1.3/python/python.swg:7)
     . top . include . insert (/r0/beazley/Projects/lib/swig1.3/python/python.swg:8)
     . top . include . typemap (/r0/beazley/Projects/lib/swig1.3/python/python.swg:19)
    ...
     . top . include (example.i:6)
     . top . include . module (example.i:2)
     . top . include . insert (example.i:6)
     . top . include . include (example.i:9)
     . top . include . include . class (example.h:3)
     . top . include . include . class . access (example.h:4)
     . top . include . include . class . constructor (example.h:7)
     . top . include . include . class . destructor (example.h:10)
     . top . include . include . class . cdecl (example.h:11)
     . top . include . include . class . cdecl (example.h:11)
     . top . include . include . class . cdecl (example.h:12)
     . top . include . include . class . cdecl (example.h:13)
     . top . include . include . class . cdecl (example.h:14)
     . top . include . include . class . cdecl (example.h:15)
     . top . include . include . class (example.h:18)
     . top . include . include . class . access (example.h:19)
     . top . include . include . class . cdecl (example.h:20)
     . top . include . include . class . access (example.h:21)
     . top . include . include . class . constructor (example.h:22)
     . top . include . include . class . cdecl (example.h:23)
     . top . include . include . class . cdecl (example.h:24)
     . top . include . include . class (example.h:27)
     . top . include . include . class . access (example.h:28)
     . top . include . include . class . cdecl (example.h:29)
     . top . include . include . class . access (example.h:30)
     . top . include . include . class . constructor (example.h:31)
     . top . include . include . class . cdecl (example.h:32)
     . top . include . include . class . cdecl (example.h:33)

Even for the most simple interface, the parse tree structure is larger than you might expect. For example, in the above output, a substantial number of nodes are actually generated by the `python.swg` configuration file which defines typemaps and other directives. The contents of the user-supplied input file don't appear until the end of the output.

The contents of each parse tree node consist of a collection of attribute/value pairs. Internally, the nodes are simply represented by hash tables. A display of the entire parse-tree structure can be obtained using `swig -dump_tree`. There are a number of other parse tree display options, for example, `swig -debug-module <n>` will avoid displaying system parse information and only display the parse tree pertaining to the user's module at stage `n` of processing.

    $ swig -c++ -python -debug-module 4 example.i
          +++ include ----------------------------------------
          | name         - "example.i"

                +++ module ----------------------------------------
                | name         - "example"
                |
                +++ insert ----------------------------------------
                | code         - "\n#include \"example.h\"\n"
                |
                +++ include ----------------------------------------
                | name         - "example.h"

                      +++ class ----------------------------------------
                      | abstract     - "1"
                      | sym:name     - "Shape"
                      | name         - "Shape"
                      | kind         - "class"
                      | symtab       - 0x40194140
                      | sym:symtab   - 0x40191078

                            +++ access ----------------------------------------
                            | kind         - "public"
                            |
                            +++ constructor ----------------------------------------
                            | sym:name     - "Shape"
                            | name         - "Shape"
                            | decl         - "f()."
                            | code         - "{\n    nshapes++;\n  }"
                            | sym:symtab   - 0x40194140
                            |
                            +++ destructor ----------------------------------------
                            | sym:name     - "~Shape"
                            | name         - "~Shape"
                            | storage      - "virtual"
                            | code         - "{\n    nshapes--;\n  }"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "x"
                            | name         - "x"
                            | decl         - ""
                            | type         - "double"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "y"
                            | name         - "y"
                            | decl         - ""
                            | type         - "double"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "move"
                            | name         - "move"
                            | decl         - "f(double,double)."
                            | parms        - double ,double
                            | type         - "void"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "area"
                            | name         - "area"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | value        - "0"
                            | type         - "double"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "perimeter"
                            | name         - "perimeter"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | value        - "0"
                            | type         - "double"
                            | sym:symtab   - 0x40194140
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "nshapes"
                            | name         - "nshapes"
                            | decl         - ""
                            | storage      - "static"
                            | type         - "int"
                            | sym:symtab   - 0x40194140
                            |
                      +++ class ----------------------------------------
                      | sym:name     - "Circle"
                      | name         - "Circle"
                      | kind         - "class"
                      | bases        - 0x40194510
                      | symtab       - 0x40194538
                      | sym:symtab   - 0x40191078

                            +++ access ----------------------------------------
                            | kind         - "private"
                            |
                            +++ cdecl ----------------------------------------
                            | name         - "radius"
                            | decl         - ""
                            | type         - "double"
                            |
                            +++ access ----------------------------------------
                            | kind         - "public"
                            |
                            +++ constructor ----------------------------------------
                            | sym:name     - "Circle"
                            | name         - "Circle"
                            | parms        - double
                            | decl         - "f(double)."
                            | code         - "{ }"
                            | sym:symtab   - 0x40194538
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "area"
                            | name         - "area"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | type         - "double"
                            | sym:symtab   - 0x40194538
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "perimeter"
                            | name         - "perimeter"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | type         - "double"
                            | sym:symtab   - 0x40194538
                            |
                      +++ class ----------------------------------------
                      | sym:name     - "Square"
                      | name         - "Square"
                      | kind         - "class"
                      | bases        - 0x40194760
                      | symtab       - 0x40194788
                      | sym:symtab   - 0x40191078

                            +++ access ----------------------------------------
                            | kind         - "private"
                            |
                            +++ cdecl ----------------------------------------
                            | name         - "width"
                            | decl         - ""
                            | type         - "double"
                            |
                            +++ access ----------------------------------------
                            | kind         - "public"
                            |
                            +++ constructor ----------------------------------------
                            | sym:name     - "Square"
                            | name         - "Square"
                            | parms        - double
                            | decl         - "f(double)."
                            | code         - "{ }"
                            | sym:symtab   - 0x40194788
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "area"
                            | name         - "area"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | type         - "double"
                            | sym:symtab   - 0x40194788
                            |
                            +++ cdecl ----------------------------------------
                            | sym:name     - "perimeter"
                            | name         - "perimeter"
                            | decl         - "f(void)."
                            | parms        - void
                            | storage      - "virtual"
                            | type         - "double"
                            | sym:symtab   - 0x40194788

### 34.4.4 Attribute namespaces

Attributes of parse tree nodes are often prepended with a namespace qualifier. For example, the attributes `sym:name` and `sym:symtab` are attributes related to symbol table management and are prefixed with `sym:`. As a general rule, only those attributes which are directly related to the raw declaration appear without a prefix (type, name, declarator, etc.).

Target language modules may add additional attributes to nodes to assist the generation of wrapper code. The convention for doing this is to place these attributes in a namespace that matches the name of the target language. For example, `python:foo` or `perl:foo`.

### 34.4.5 Symbol Tables

During parsing, all symbols are managed in the space of the target language. The `sym:name` attribute of each node contains the symbol name selected by the parser. Normally, `sym:name` and `name` are the same. However, the `%rename` directive can be used to change the value of `sym:name`. You can see the effect of `%rename` by trying it on a simple interface and dumping the parse tree. For example:

    %rename(foo_i) foo(int);
    %rename(foo_d) foo(double);

    void foo(int);
    void foo(double);
    void foo(Bar *b);

There are various `debug-` options that can be useful for debugging and analysing the parse tree. For example, the `debug-top <n>` or `debug-module <n>` options will dump the entire/top of the parse tree or the module subtree at one of the four `n` stages of processing. The parse tree can be viewed after the final stage of processing by running SWIG:

    $ swig -debug-top 4 example.i
    ...
                +++ cdecl ----------------------------------------
                | sym:name     - "foo_i"
                | name         - "foo"
                | decl         - "f(int)."
                | parms        - int
                | type         - "void"
                | sym:symtab   - 0x40165078
                |
                +++ cdecl ----------------------------------------
                | sym:name     - "foo_d"
                | name         - "foo"
                | decl         - "f(double)."
                | parms        - double
                | type         - "void"
                | sym:symtab   - 0x40165078
                |
                +++ cdecl ----------------------------------------
                | sym:name     - "foo"
                | name         - "foo"
                | decl         - "f(p.Bar)."
                | parms        - Bar *
                | type         - "void"
                | sym:symtab   - 0x40165078

All symbol-related conflicts and complaints about overloading are based on `sym:name` values. For instance, the following example uses `%rename` in reverse to generate a name clash.

    %rename(foo) foo_i(int);
    %rename(foo) foo_d(double;

    void foo_i(int);
    void foo_d(double);
    void foo(Bar *b);

When you run SWIG on this you now get:

    $ ./swig example.i
    example.i:6. Overloaded declaration ignored.  foo_d(double )
    example.i:5. Previous declaration is foo_i(int )
    example.i:7. Overloaded declaration ignored.  foo(Bar *)
    example.i:5. Previous declaration is foo_i(int )

### 34.4.6 The %feature directive

A number of SWIG directives such as `%exception` are implemented using the low-level `%feature` directive. For example:

    %feature("except") getitem(int) {
      try {
         $action
      } catch (badindex) {
         ...
      }
    }

    ...
    class Foo {
    public:
        Object *getitem(int index) throws(badindex);
        ...
    };

The behavior of `%feature` is very easy to describe--it simply attaches a new attribute to any parse tree node that matches the given prototype. When a feature is added, it shows up as an attribute in the `feature:` namespace. You can see this when running with the `-debug-top 4` option. For example:

     +++ cdecl ----------------------------------------
     | sym:name     - "getitem"
     | name         - "getitem"
     | decl         - "f(int).p."
     | parms        - int
     | type         - "Object"
     | feature:except - "{\n    try {\n       $action\n    } catc..."
     | sym:symtab   - 0x40168ac8
     |

Feature names are completely arbitrary and a target language module can be programmed to respond to any feature name that it wants to recognize. The data stored in a feature attribute is usually just a raw unparsed string. For example, the exception code above is simply stored without any modifications.

### 34.4.7 Code Generation

Language modules work by defining handler functions that know how to respond to different types of parse-tree nodes. These handlers simply look at the attributes of each node in order to produce low-level code.

In reality, the generation of code is somewhat more subtle than simply invoking handler functions. This is because parse-tree nodes might be transformed. For example, suppose you are wrapping a class like this:

    class Foo {
    public:
        virtual int *bar(int x);
    };

When the parser constructs a node for the member `bar`, it creates a raw "cdecl" node with the following attributes:

    nodeType    : cdecl
    name        : bar
    type        : int
    decl        : f(int).p
    parms       : int x
    storage     : virtual
    sym:name    : bar

To produce wrapper code, this "cdecl" node undergoes a number of transformations. First, the node is recognized as a function declaration. This adjusts some of the type information--specifically, the declarator is joined with the base datatype to produce this:

    nodeType    : cdecl
    name        : bar
    type        : p.int        <-- Notice change in return type
    decl        : f(int).p
    parms       : int x
    storage     : virtual
    sym:name    : bar

Next, the context of the node indicates that the node is really a member function. This produces a transformation to a low-level accessor function like this:

    nodeType    : cdecl
    name        : bar
    type        : int.p
    decl        : f(int).p
    parms       : Foo *self, int x            <-- Added parameter
    storage     : virtual
    wrap:action : result = (arg1)->bar(arg2)  <-- Action code added
    sym:name    : Foo_bar                     <-- Symbol name changed

In this transformation, notice how an additional parameter was added to the parameter list and how the symbol name of the node has suddenly changed into an accessor using the naming scheme described in the "SWIG Basics" chapter. A small fragment of "action" code has also been generated--notice how the `wrap:action` attribute defines the access to the underlying method. The data in this transformed node is then used to generate a wrapper.

Language modules work by registering handler functions for dealing with various types of nodes at different stages of transformation. This is done by inheriting from a special `Language` class and defining a collection of virtual methods. For example, the Python module defines a class as follows:

    class PYTHON : public Language {
    protected:
    public :
      virtual void main(int, char *argv[]);
      virtual int  top(Node *);
      virtual int  functionWrapper(Node *);
      virtual int  constantWrapper(Node *);
      virtual int  variableWrapper(Node *);
      virtual int  nativeWrapper(Node *);
      virtual int  membervariableHandler(Node *);
      virtual int  memberconstantHandler(Node *);
      virtual int  memberfunctionHandler(Node *);
      virtual int  constructorHandler(Node *);
      virtual int  destructorHandler(Node *);
      virtual int  classHandler(Node *);
      virtual int  classforwardDeclaration(Node *);
      virtual int  insertDirective(Node *);
      virtual int  importDirective(Node *);
    };

The role of these functions is described shortly.

### 34.4.8 SWIG and XML

Much of SWIG's current parser design was originally motivated by interest in using XML to represent SWIG parse trees. Although XML is not currently used in any direct manner, the parse tree structure, use of node tags, attributes, and attribute namespaces are all influenced by aspects of XML parsing. Therefore, in trying to understand SWIG's internal data structures, it may be useful to keep XML in the back of your mind as a model.

## 34.5 Primitive Data Structures

Most of SWIG is constructed using three basic data structures: strings, hashes, and lists. These data structures are dynamic in same way as similar structures found in many scripting languages. For instance, you can have containers (lists and hash tables) of mixed types and certain operations are polymorphic.

This section briefly describes the basic structures so that later sections of this chapter make more sense.

When describing the low-level API, the following type name conventions are used:

- `String`. A string object.
- `Hash`. A hash object.
- `List`. A list object.
- `String_or_char`. A string object or a `char *`.
- `Object_or_char`. An object or a `char *`.
- `Object`. Any object (string, hash, list, etc.)

In most cases, other typenames in the source are aliases for one of these primitive types. Specifically:

    typedef String SwigType;
    typedef Hash   Parm;
    typedef Hash   ParmList;
    typedef Hash   Node;
    typedef Hash   Symtab;
    typedef Hash   Typetab;

### 34.5.1 Strings

`String *NewString(const String_or_char *val)`

Creates a new string with initial value `val`. `val` may be a `char *` or another `String` object. If you want to create an empty string, use "" for val.

`String *NewStringf(const char *fmt, ...)`

Creates a new string whose initial value is set according to a C `printf` style format string in `fmt`. Additional arguments follow depending on `fmt`.

`String *Copy(String *s)`

Make a copy of the string `s`.

`void Delete(String *s)`

Deletes `s`.

`int Len(String_or_char *s)`

Returns the length of the string.

`char *Char(String_or_char *s)`

Returns a pointer to the first character in a string.

`void Append(String *s, String_or_char *t)`

Appends `t` to the end of string `s`.

`void Insert(String *s, int pos, String_or_char *t)`

Inserts `t` into `s` at position `pos`. The contents of `s` are shifted accordingly. The special value `DOH_END` can be used for `pos` to indicate insertion at the end of the string (appending).

`int Strcmp(const String_or_char *s, const String_or_char *t)`

Compare strings `s` and `t`. Same as the C `strcmp()` function.

`int Strncmp(const String_or_char *s, const String_or_char *t, int len)`

Compare the first `len` characters of strings `s` and `t`. Same as the C `strncmp()` function.

`char *Strstr(const String_or_char *s, const String_or_char *pat)`

Returns a pointer to the first occurrence of `pat` in `s`. Same as the C `strstr()` function.

`char *Strchr(const String_or_char *s, char ch)`

Returns a pointer to the first occurrence of character `ch` in `s`. Same as the C `strchr()` function.

`void Chop(String *s)`

Chops trailing whitespace off the end of `s`.

`int Replace(String *s, const String_or_char *pat, const String_or_char *rep, int flags)`

Replaces the pattern `pat` with `rep` in string `s`. `flags` is a combination of the following flags:

    DOH_REPLACE_ANY       - Replace all occurrences
    DOH_REPLACE_ID        - Valid C identifiers only
    DOH_REPLACE_NOQUOTE   - Don't replace in quoted strings
    DOH_REPLACE_FIRST     - Replace first occurrence only.

Returns the number of replacements made (if any).

### 34.5.2 Hashes

`Hash *NewHash()`

Creates a new empty hash table.

`Hash *Copy(Hash *h)`

Make a shallow copy of the hash `h`.

`void Delete(Hash *h)`

Deletes `h`.

`int Len(Hash *h)`

Returns the number of items in `h`.

`Object *Getattr(Hash *h, String_or_char *key)`

Gets an object from `h`. `key` may be a string or a simple `char *` string. Returns NULL if not found.

`int Setattr(Hash *h, String_or_char *key, Object_or_char *val)`

Stores `val` in `h`. `key` may be a string or a simple `char *`. If `val` is not a standard object (String, Hash, or List) it is assumed to be a `char *` in which case it is used to construct a `String` that is stored in the hash. If `val` is NULL, the object is deleted. Increases the reference count of `val`. Returns 1 if this operation replaced an existing hash entry, 0 otherwise.

`int Delattr(Hash *h, String_or_char *key)`

Deletes the hash item referenced by `key`. Decreases the reference count on the corresponding object (if any). Returns 1 if an object was removed, 0 otherwise.

`List *Keys(Hash *h)`

Returns the list of hash table keys.

### 34.5.3 Lists

`List *NewList()`

Creates a new empty list.

`List *Copy(List *x)`

Make a shallow copy of the List `x`.

`void Delete(List *x)`

Deletes `x`.

`int Len(List *x)`

Returns the number of items in `x`.

`Object *Getitem(List *x, int n)`

Returns an object from `x` with index `n`. If `n` is beyond the end of the list, the last item is returned. If `n` is negative, the first item is returned.

`int *Setitem(List *x, int n, Object_or_char *val)`

Stores `val` in `x`. If `val` is not a standard object (String, Hash, or List) it is assumed to be a `char *` in which case it is used to construct a `String` that is stored in the list. `n` must be in range. Otherwise, an assertion will be raised.

`int *Delitem(List *x, int n)`

Deletes item `n` from the list, shifting items down if necessary. To delete the last item in the list, use the special value `DOH_END` for `n`.

`void Append(List *x, Object_or_char *t)`

Appends `t` to the end of `x`. If `t` is not a standard object, it is assumed to be a `char *` and is used to create a String object.

`void Insert(String *s, int pos, Object_or_char *t)`

Inserts `t` into `s` at position `pos`. The contents of `s` are shifted accordingly. The special value `DOH_END` can be used for `pos` to indicate insertion at the end of the list (appending). If `t` is not a standard object, it is assumed to be a `char *` and is used to create a String object.

### 34.5.4 Common operations

The following operations are applicable to all datatypes.

`Object *Copy(Object *x)`

Make a copy of the object `x`.

`void Delete(Object *x)`

Deletes `x`.

`void Setfile(Object *x, String_or_char *f)`

Sets the filename associated with `x`. Used to track objects and report errors.

`String *Getfile(Object *x)`

Gets the filename associated with `x`.

`void Setline(Object *x, int n)`

Sets the line number associated with `x`. Used to track objects and report errors.

`int Getline(Object *x)`

Gets the line number associated with `x`.

### 34.5.5 Iterating over Lists and Hashes

To iterate over the elements of a list or a hash table, the following functions are used:

`Iterator First(Object *x)`

Returns an iterator object that points to the first item in a list or hash table. The `item` attribute of the Iterator object is a pointer to the item. For hash tables, the `key` attribute of the Iterator object additionally points to the corresponding Hash table key. The `item` and `key` attributes are NULL if the object contains no items or if there are no more items.

`Iterator Next(Iterator i)`

Returns an iterator that points to the next item in a list or hash table. Here are two examples of iteration:

    List *l = (some list);
    Iterator i;

    for (i = First(l); i.item; i = Next(i)) {
        Printf(stdout,"%s\n", i.item);
    }

    Hash *h = (some hash);
    Iterator j;

    for (j = First(j); j.item; j= Next(j)) {
        Printf(stdout,"%s : %s\n", j.key, j.item);
    }

### 34.5.6 I/O

Special I/O functions are used for all internal I/O. These operations work on C `FILE *` objects, String objects, and special `File` objects (which are merely a wrapper around `FILE *`).

`int Printf(String_or_FILE *f, const char *fmt, ...)`

Formatted I/O. Same as the C `fprintf()` function except that output can also be directed to a string object. Note: the `%s` format specifier works with both strings and `char *`. All other format operators have the same meaning.

`int Printv(String_or_FILE *f, String_or_char *arg1,..., NULL)`

Prints a variable number of strings arguments to the output. The last argument to this function must be NULL. The other arguments can either be `char *` or string objects.

`int Putc(int ch, String_or_FILE *f)`

Same as the C `fputc()` function.

`int Write(String_or_FILE *f, void *buf, int len)`

Same as the C `write()` function.

`int Read(String_or_FILE *f, void *buf, int maxlen)`

Same as the C `read()` function.

`int Getc(String_or_FILE *f)`

Same as the C `fgetc()` function.

`int Ungetc(int ch, String_or_FILE *f)`

Same as the C `ungetc()` function.

`int Seek(String_or_FILE *f, int offset, int whence)`

Same as the C `seek()` function. `offset` is the number of bytes. `whence` is one of `SEEK_SET`,`SEEK_CUR`, or `SEEK_END`..

`long Tell(String_or_FILE *f)`

Same as the C `tell()` function.

`File *NewFile(const char *filename, const char *mode)`

Create a File object using the `fopen()` library call. This file differs from `FILE *` in that it can be placed in the standard SWIG containers (lists, hashes, etc.).

`File *NewFileFromFile(FILE *f)`

Create a File object wrapper around an existing `FILE *` object.

`int Close(String_or_FILE *f)`

Closes a file. Has no effect on strings.

The use of the above I/O functions and strings play a critical role in SWIG. It is common to see small code fragments of code generated using code like this:

    /* Print into a string */
    String *s = NewString("");
    Printf(s,"Hello\n");
    for (i = 0; i < 10; i++) {
        Printf(s,"%d\n", i);
    }
    ...
    /* Print string into a file */
    Printf(f, "%s\n", s);

Similarly, the preprocessor and parser all operate on string-files.

## 34.6 Navigating and manipulating parse trees

Parse trees are built as collections of hash tables. Each node is a hash table in which arbitrary attributes can be stored. Certain attributes in the hash table provide links to other parse tree nodes. The following macros can be used to move around the parse tree.

`String *nodeType(Node *n)`

Returns the node type tag as a string. The returned string indicates the type of parse tree node.

`Node *nextSibling(Node *n)`

Returns the next node in the parse tree. For example, the next C declaration.

`Node *previousSibling(Node *n)`

Returns the previous node in the parse tree. For example, the previous C declaration.

`Node *firstChild(Node *n)`

Returns the first child node. For example, if `n` was a C++ class node, this would return the node for the first class member.

`Node *lastChild(Node *n)`

Returns the last child node. You might use this if you wanted to append a new node to the of a class.

`Node *parentNode(Node *n)`

Returns the parent of node `n`. Use this to move up the pass tree.

The following macros can be used to change all of the above attributes. Normally, these functions are only used by the parser. Changing them without knowing what you are doing is likely to be dangerous.

`void set_nodeType(Node *n, const String_or_char)`

Change the node type. tree node.

`void set_nextSibling(Node *n, Node *s)`

Set the next sibling.

`void set_previousSibling(Node *n, Node *s)`

Set the previous sibling.

`void set_firstChild(Node *n, Node *c)`

Set the first child node.

`void set_lastChild(Node *n, Node *c)`

Set the last child node.

`void set_parentNode(Node *n, Node *p)`

Set the parent node.

The following utility functions are used to alter the parse tree (at your own risk)

`void appendChild(Node *parent, Node *child)`

Append a child to `parent`. The appended node becomes the last child.

`void deleteNode(Node *node)`

Deletes a node from the parse tree. Deletion reconnects siblings and properly updates the parent so that sibling nodes are unaffected.

## 34.7 Working with attributes

Since parse tree nodes are just hash tables, attributes are accessed using the `Getattr()`, `Setattr()`, and `Delattr()` operations. For example:

    int functionHandler(Node *n) {
        String *name    = Getattr(n,"name");
        String *symname = Getattr(n,"sym:name");
        SwigType *type  = Getattr(n,"type");
        ...
    }

New attributes can be freely attached to a node as needed. However, when new attributes are attached during code generation, they should be prepended with a namespace prefix. For example:

    ...
    Setattr(n,"python:docstring", doc);     /* Store docstring */
    ...

A quick way to check the value of an attribute is to use the `checkAttribute()` function like this:

    if (checkAttribute(n,"storage","virtual")) {
       /* n is virtual */
       ...
    }

Changing the values of existing attributes is allowed and is sometimes done to implement node transformations. However, if a function/method modifies a node, it is required to restore modified attributes to their original values. To simplify the task of saving/restoring attributes, the following functions are used:

`int Swig_save(const char *ns, Node *n, const char *name1, const char *name2, ..., NIL)`

Saves a copy of attributes `name1`, `name2`, etc. from node `n`. Copies of the attributes are actually resaved in the node in a different namespace which is set by the `ns` argument. For example, if you call `Swig_save("foo",n,"type",NIL)`, then the "type" attribute will be copied and saved as "foo:type". The namespace name itself is stored in the "view" attribute of the node. If necessary, this can be examined to find out where previous values of attributes might have been saved.

`int Swig_restore(Node *n)`

Restores the attributes saved by the previous call to `Swig_save()`. Those attributes that were supplied to `Swig_save()` will be restored to their original values.

The `Swig_save()` and `Swig_restore()` functions must always be used as a pair. That is, every call to `Swig_save()` must have a matching call to `Swig_restore()`. Calls can be nested if necessary. Here is an example that shows how the functions might be used:

    int variableHandler(Node *n) {
        Swig_save("variableHandler",n,"type","sym:name",NIL);
        String *symname = Getattr(n,"sym:name");
        SwigType *type  = Getattr(n,"type");
        ...
        Append(symname,"_global");         // Change symbol name
        SwigType_add_pointer(type);        // Add pointer
        ...
        generate wrappers
        ...
        Swig_restore(n);                  // Restore original values
        return SWIG_OK;
    }

`int Swig_require(const char *ns, Node *n, const char *name1, const char *name2, ..., NIL)`

This is an enhanced version of `Swig_save()` that adds error checking. If an attribute name is not present in `n`, a failed assertion results and SWIG terminates with a fatal error. Optionally, if an attribute name is specified as "*name", a copy of the attribute is saved as with `Swig_save()`. If an attribute is specified as "?name", the attribute is optional. `Swig_restore()` must always be called after using this function.

## 34.8 Type system

SWIG implements the complete C++ type system including typedef, inheritance, pointers, references, and pointers to members. A detailed discussion of type theory is impossible here. However, let's cover the highlights.

### 34.8.1 String encoding of types

All types in SWIG consist of a base datatype and a collection of type operators that are applied to the base. A base datatype is almost always some kind of primitive type such as `int` or `double`. The operators consist of things like pointers, references, arrays, and so forth. Internally, types are represented as strings that are constructed in a very precise manner. Here are some examples:

    C datatype                     SWIG encoding (strings)
    -----------------------------  --------------------------
    int                            "int"
    int *                          "p.int"
    const int *                    "p.q(const).int"
    int (*x)(int,double)           "p.f(int,double).int"
    int [20][30]                   "a(20).a(30).int"
    int (F::*)(int)                "m(F).f(int).int"
    vector<int> *                  "p.vector<(int)>"

Reading the SWIG encoding is often easier than figuring out the C code---just read it from left to right. For a type of "p.f(int,double).int" is a "pointer to a function(int,double) that returns int".

The following operator encodings are used in type strings:

    Operator              Meaning
    -------------------   -------------------------------
    p.                    Pointer to
    a(n).                 Array of dimension n
    r.                    C++ reference
    m(class).             Member pointer to class
    f(args).              Function.
    q(qlist).             Qualifiers

In addition, type names may be parameterized by templates. This is represented by enclosing the template parameters in `<( ... )>`. Variable length arguments are represented by the special base type of `v(...)`.

If you want to experiment with type encodings, the raw type strings can be inserted into an interface file using backticks `` wherever a type is expected. For instance, here is an extremely perverted example:

    `p.a(10).p.f(int,p.f(int).int)` foo(int, int (*x)(int));

This corresponds to the immediately obvious C declaration:

    (*(*foo(int,int (*)(int)))[10])(int,int (*)(int));

Aside from the potential use of this declaration on a C programming quiz, it motivates the use of the special SWIG encoding of types. The SWIG encoding is much easier to work with because types can be easily examined, modified, and constructed using simple string operations (comparison, substrings, concatenation, etc.). For example, in the parser, a declaration like this

    int *a[30];

is processed in a few pieces. In this case, you have the base type "`int`" and the declarator of type "`a(30).p.`". To make the final type, the two parts are just joined together using string concatenation.

### 34.8.2 Type construction

The following functions are used to construct types. You should use these functions instead of trying to build the type strings yourself.

`void SwigType_add_pointer(SwigType *ty)`

Adds a pointer to `ty`.

`void SwigType_del_pointer(SwigType *ty)`

Removes a single pointer from `ty`.

`void SwigType_add_reference(SwigType *ty)`

Adds a reference to `ty`.

`void SwigType_add_array(SwigType *ty, String_or_char *dim)`

Adds an array with dimension `dim` to `ty`.

`void SwigType_del_array(SwigType *ty)`

Removes a single array dimension from `ty`.

`int SwigType_array_ndim(SwigType *ty)`

Returns number of array dimensions of `ty`.

`String* SwigType_array_getdim(SwigType *ty,int n)`

Returns `n`th array dimension of `ty`.

`void SwigType_array_setdim(SwigType *ty, int n, const String_or_char *rep)`

Sets `n`th array dimensions of `ty` to `rep`.

`void SwigType_add_qualifier(SwigType *ty, String_or_char *q)`

Adds a type qualifier `q` to `ty`. `q` is typically `"const"` or `"volatile"`.

`void SwigType_add_memberpointer(SwigType *ty, String_or_char *cls)`

Adds a pointer to a member of class `cls` to `ty`.

`void SwigType_add_function(SwigType *ty, ParmList *p)`

Adds a function to `ty`. `p` is a linked-list of parameter nodes as generated by the parser. See the section on parameter lists for details about the representation.

`void SwigType_add_template(SwigType *ty, ParmList *p)`

Adds a template to `ty`. `p` is a linked-list of parameter nodes as generated by the parser. See the section on parameter lists for details about the representation.

`SwigType *SwigType_pop(SwigType *ty)`

Removes the last type constructor from `ty` and returns it. `ty` is modified.

`void SwigType_push(SwigType *ty, SwigType *op)`

Pushes the type operators in `op` onto type `ty`. The opposite of `SwigType_pop()`.

`SwigType *SwigType_pop_arrays(SwigType *ty)`

Removes all leading array operators from `ty` and returns them. `ty` is modified. For example, if `ty` is `"a(20).a(10).p.int"`, then this function would return `"a(20).a(10)."` and modify `ty` so that it has the value `"p.int"`.

`SwigType *SwigType_pop_function(SwigType *ty)`

Removes a function operator from `ty` including any qualification. `ty` is modified. For example, if `ty` is `"f(int).int"`, then this function would return `"f(int)."` and modify `ty` so that it has the value `"int"`.

`SwigType *SwigType_base(SwigType *ty)`

Returns the base type of a type. For example, if `ty` is `"p.a(20).int"`, this function would return `"int"`. `ty` is unmodified.

`SwigType *SwigType_prefix(SwigType *ty)`

Returns the prefix of a type. For example, if `ty` is `"p.a(20).int"`, this function would return `"p.a(20)."`. `ty` is unmodified.

### 34.8.3 Type tests

The following functions can be used to test properties of a datatype.

`int SwigType_ispointer(SwigType *ty)`

Checks if `ty` is a standard pointer.

`int SwigType_ismemberpointer(SwigType *ty)`

Checks if `ty` is a member pointer.

`int SwigType_isreference(SwigType *ty)`

Checks if `ty` is a C++ reference.

`int SwigType_isarray(SwigType *ty)`

Checks if `ty` is an array.

`int SwigType_isfunction(SwigType *ty)`

Checks if `ty` is a function.

`int SwigType_isqualifier(SwigType *ty)`

Checks if `ty` is a qualifier.

`int SwigType_issimple(SwigType *ty)`

Checks if `ty` is a simple type. No operators applied.

`int SwigType_isconst(SwigType *ty)`

Checks if `ty` is a const type.

`int SwigType_isvarargs(SwigType *ty)`

Checks if `ty` is a varargs type.

`int SwigType_istemplate(SwigType *ty)`

Checks if `ty` is a templatized type.

### 34.8.4 Typedef and inheritance

The behavior of `typedef` declaration is to introduce a type alias. For instance, `typedef int Integer` makes the identifier `Integer` an alias for `int`. The treatment of typedef in SWIG is somewhat complicated due to the pattern matching rules that get applied in typemaps and the fact that SWIG prefers to generate wrapper code that closely matches the input to simplify debugging (a user will see the typedef names used in their program instead of the low-level primitive C datatypes).

To handle `typedef`, SWIG builds a collection of trees containing typedef relations. For example,

    typedef int Integer;
    typedef Integer *IntegerPtr;
    typedef int Number;
    typedef int Size;

produces two trees like this:

                     int               p.Integer
                   ^  ^  ^                 ^
                  /   |   \                |
                 /    |    \               |
            Integer  Size   Number    IntegerPtr

To resolve a single typedef relationship, the following function is used:

`SwigType *SwigType_typedef_resolve(SwigType *ty)`

Checks if `ty` can be reduced to a new type via typedef. If so, returns the new type. If not, returns NULL.

Typedefs are only resolved in simple typenames that appear in a type. For example, the type base name and in function parameters. When resolving types, the process starts in the leaf nodes and moves up the tree towards the root. Here are a few examples that show how it works:

    Original type            After typedef_resolve()
    ------------------------ -----------------------
    Integer                  int
    a(30).Integer            int
    p.IntegerPtr             p.p.Integer
    p.p.Integer              p.p.int

For complicated types, the process can be quite involved. Here is the reduction of a function pointer:

    p.f(Integer, p.IntegerPtr, Size).Integer          : Start
    p.f(Integer, p.IntegerPtr, Size).int
    p.f(int, p.IntegerPtr, Size).int
    p.f(int, p.p.Integer, Size).int
    p.f(int, p.p.int, Size).int
    p.f(int, p.p.int, int).int                        : End

Two types are equivalent if their full type reductions are the same. The following function will fully reduce a datatype:

`SwigType *SwigType_typedef_resolve_all(SwigType *ty)`

Fully reduces `ty` according to typedef rules. Resulting datatype will consist only of primitive typenames.

### 34.8.5 Lvalues

When generating wrapper code, it is necessary to emit datatypes that can be used on the left-hand side of an assignment operator (an lvalue). However, not all C datatypes can be used in this way---especially arrays and const-qualified types. To generate a type that can be used as an lvalue, use the following function:

`SwigType *SwigType_ltype(SwigType *ty)`

Converts type `ty` to a type that can be used as an lvalue in assignment. The resulting type is stripped of qualifiers and arrays are converted to a pointers.

The creation of lvalues is fully aware of typedef and other aspects of the type system. Therefore, the creation of an lvalue may result in unexpected results. Here are a few examples:

    typedef double Matrix4[4][4];
    Matrix4 x;    // type = 'Matrix4', ltype='p.a(4).double'

    typedef const char * Literal;
    Literal y;    // type = 'Literal', ltype='p.char'

### 34.8.6 Output functions

The following functions produce strings that are suitable for output.

`String *SwigType_str(SwigType *ty, String_or_char *id = 0)`

Generates a C string for a datatype. `id` is an optional declarator. For example, if `ty` is "p.f(int).int" and `id` is "foo", then this function produces "`int (*foo)(int)`". This function is used to convert string-encoded types back into a form that is valid C syntax.

`String *SwigType_lstr(SwigType *ty, String_or_char *id = 0)`

This is the same as `SwigType_str()` except that the result is generated from the type's lvalue (as generated from SwigType_ltype).

`String *SwigType_lcaststr(SwigType *ty, String_or_char *id = 0)`

Generates a casting operation that converts from type `ty` to its lvalue. `id` is an optional name to include in the cast. For example, if `ty` is "`q(const).p.char`" and `id` is "`foo`", this function produces the string "`(char *) foo`".

`String *SwigType_rcaststr(SwigType *ty, String_or_char *id = 0)`

Generates a casting operation that converts from a type's lvalue to a type equivalent to `ty`. `id` is an optional name to include in the cast. For example, if `ty` is "`q(const).p.char`" and `id` is "`foo`", this function produces the string "`(const char *) foo`".

`String *SwigType_manglestr(SwigType *ty)`

Generates a mangled string encoding of type `ty`. The mangled string only contains characters that are part of a valid C identifier. The resulting string is used in various parts of SWIG, but is most commonly associated with type-descriptor objects that appear in wrappers (e.g., `SWIGTYPE_p_double`).

## 34.9 Parameters

Several type-related functions involve parameter lists. These include functions and templates. Parameter list are represented as a list of nodes with the following attributes:

    "type"        -  Parameter type  (required)
    "name"        -  Parameter name  (optional)
    "value"       -  Initializer     (optional)

Typically parameters are denoted in the source by using a typename of `Parm *` or `ParmList *`. To walk a parameter list, simply use code like this:

    Parm *parms;
    Parm *p;
    for (p = parms; p; p = nextSibling(p)) {
        SwigType *type  = Getattr(p,"type");
        String   *name  = Getattr(p,"name");
        String   *value = Getattr(p,"value");
        ...
    }

Note: this code is exactly the same as what you would use to walk parse tree nodes.

An empty list of parameters is denoted by a NULL pointer.

Since parameter lists are fairly common, the following utility functions are provided to manipulate them:

`Parm *CopyParm(Parm *p);`

Copies a single parameter.

`ParmList *CopyParmList(ParmList *p);`

Copies an entire list of parameters.

`int ParmList_len(ParmList *p);`

Returns the number of parameters in a parameter list.

`String *ParmList_str(ParmList *p);`

Converts a parameter list into a C string. For example, produces a string like "`(int *p, int n, double x);`".

`String *ParmList_protostr(ParmList *p);`

The same as `ParmList_str()` except that parameter names are not included. Used to emit prototypes.

`int ParmList_numrequired(ParmList *p);`

Returns the number of required (non-optional) arguments in `p`.

## 34.10 Writing a Language Module

One of the easiest routes to supporting a new language module is to copy an already supported language module implementation and modify it. Be sure to choose a language that is similar in nature to the new language. All language modules follow a similar structure and this section briefly outlines the steps needed to create a bare-bones language module from scratch. Since the code is relatively easy to read, this section describes the creation of a minimal Python module. You should be able to extrapolate this to other languages.

### 34.10.1 Execution model

Code generation modules are defined by inheriting from the `Language` class, currently defined in the `Source/Modules` directory of SWIG. Starting from the parsing of command line options, all aspects of code generation are controlled by different methods of the `Language` that must be defined by your module.

### 34.10.2 Starting out

To define a new language module, first create a minimal implementation using this example as a guide:

    #include "swigmod.h"

    class PYTHON : public Language {
    public:

      virtual void main(int argc, char *argv[]) {
        printf("I'm the Python module.\n");
      }

      virtual int top(Node *n) {
        printf("Generating code.\n");
        return SWIG_OK;
      }

    };

    extern "C" Language *
    swig_python(void) {
      return new PYTHON();
    }

The "swigmod.h" header file contains, among other things, the declaration of the `Language` base class and so you should include it at the top of your language module's source file. Similarly, the "swigconfig.h" header file contains some other useful definitions that you may need. Note that you should not include any header files that are installed with the target language. That is to say, the implementation of the SWIG Python module shouldn't have any dependencies on the Python header files. The wrapper code generated by SWIG will almost always depend on some language-specific C/C++ header files, but SWIG itself does not.

Give your language class a reasonable name, usually the same as the target language. By convention, these class names are all uppercase (e.g. "PYTHON" for the Python language module) but this is not a requirement. This class will ultimately consist of a number of overrides of the virtual functions declared in the `Language` base class, in addition to any language-specific member functions and data you need. For now, just use the dummy implementations shown above.

The language module ends with a factory function, `swig_python()`, that simply returns a new instance of the language class. As shown, it should be declared with the `extern "C"` storage qualifier so that it can be called from C code. It should also return a pointer to the base class (`Language`) so that only the interface (and not the implementation) of your language module is exposed to the rest of SWIG.

Save the code for your language module in a file named "`python.cxx`" and. place this file in the `Source/Modules` directory of the SWIG distribution. To ensure that your module is compiled into SWIG along with the other language modules, modify the file `Source/Modules/Makefile.am` to include the additional source files. In addition, modify the file `Source/Modules/swigmain.cxx` with an additional command line option that activates the module. Read the source---it's straightforward.

Next, at the top level of the SWIG distribution, re-run the `autogen.sh` script to regenerate the various build files:

    $ ./autogen.sh

Next re-run `configure` to regenerate all of the Makefiles:

    $ ./configure

Finally, rebuild SWIG with your module added:

    $ make

Once it finishes compiling, try running SWIG with the command-line option that activates your module. For example, `swig -python foo.i`. The messages from your new module should appear.

### 34.10.3 Command line options

When SWIG starts, the command line options are passed to your language module. This occurs before any other processing occurs (preprocessing, parsing, etc.). To capture the command line options, simply use code similar to this:

    void Language::main(int argc, char *argv[]) {
      for (int i = 1; i < argc; i++) {
          if (argv[i]) {
              if(strcmp(argv[i],"-interface") == 0) {
                if (argv[i+1]) {
                  interface = NewString(argv[i+1]);
                  Swig_mark_arg(i);
                  Swig_mark_arg(i+1);
                  i++;
                } else {
                  Swig_arg_error();
                }
    	  } else if (strcmp(argv[i],"-globals") == 0) {
    	    if (argv[i+1]) {
    	      global_name = NewString(argv[i+1]);
    	      Swig_mark_arg(i);
    	      Swig_mark_arg(i+1);
    	      i++;
    	    } else {
    	      Swig_arg_error();
    	    }
    	  } else if ( (strcmp(argv[i],"-proxy") == 0)) {
    	    proxy_flag = 1;
    	    Swig_mark_arg(i);
    	  } else if (strcmp(argv[i],"-keyword") == 0) {
    	    use_kw = 1;
    	    Swig_mark_arg(i);
    	  } else if (strcmp(argv[i],"-help") == 0) {
    	    fputs(usage,stderr);
    	  }
              ...
          }
      }
    }

The exact set of options depends on what you want to do in your module. Generally, you would use the options to change code generation modes or to print diagnostic information.

If a module recognizes an option, it should always call `Swig_mark_arg()` to mark the option as valid. If you forget to do this, SWIG will terminate with an unrecognized command line option error.

### 34.10.4 Configuration and preprocessing

In addition to looking at command line options, the `main()` method is responsible for some initial configuration of the SWIG library and preprocessor. To do this, insert some code like this:

    void main(int argc, char *argv[]) {
       ... command line options ...

       /* Set language-specific subdirectory in SWIG library */
       SWIG_library_directory("python");

       /* Set language-specific preprocessing symbol */
       Preprocessor_define("SWIGPYTHON 1", 0);

       /* Set language-specific configuration file */
       SWIG_config_file("python.swg");

       /* Set typemap language (historical) */
       SWIG_typemap_lang("python");
    }

The above code does several things--it registers the name of the language module with the core, it supplies some preprocessor macro definitions for use in input files (so that they can determine the target language), and it registers a start-up file. In this case, the file `python.swg` will be parsed before any part of the user-supplied input file.

Before proceeding any further, create a directory for your module in the SWIG library (The `Lib` directory). Now, create a configuration file in the directory. For example, `python.swg`.

Just to review, your language module should now consist of two files-- an implementation file `python.cxx` and a configuration file `python.swg`.

### 34.10.5 Entry point to code generation

SWIG is a multi-pass compiler. Once the `main()` method has been invoked, the language module does not execute again until preprocessing, parsing, and a variety of semantic analysis passes have been performed. When the core is ready to start generating wrappers, it invokes the `top()` method of your language class. The argument to `top` is a single parse tree node that corresponds to the top of the entire parse tree.

To get the code generation process started, the `top()` procedure needs to do several things:

- Initialize the wrapper code output.
- Set the module name.
- Emit common initialization code.
- Emit code for all of the child nodes.
- Finalize the wrapper module and cleanup.

An outline of `top()` might be as follows:

    int Python::top(Node *n) {

       /* Get the module name */
       String *module = Getattr(n,"name");

       /* Get the output file name */
       String *outfile = Getattr(n,"outfile");

       /* Initialize I/O (see next section) */
       ...

       /* Output module initialization code */
       ...

       /* Emit code for children */
       Language::top(n);

       ...
       /* Cleanup files */
       ...

       return SWIG_OK;
    }

### 34.10.6 Module I/O and wrapper skeleton

Within SWIG wrappers, there are four main sections. These are (in order)

- runtime: This section has most of the common SWIG runtime code
- header: This section holds declarations and inclusions from the .i file
- wrapper: This section holds all the wrappering code
- init: This section holds the module initalisation function (the entry point for the interpreter)

Different parts of the SWIG code will fill different sections, then upon completion of the wrappering all the sections will be saved to the wrapper file.

To perform this will require several additions to the code in various places, such as:

    class PYTHON : public Language {
    protected:
       /* General DOH objects used for holding the strings */
       File *f_runtime;
       File *f_header;
       File *f_wrappers;
       File *f_init;

    public:
       ...

    };

    int Python::top(Node *n) {

       ...

       /* Initialize I/O */
       f_runtime = NewFile(outfile, "w");
       if (!f_runtime) {
          FileErrorDisplay(outfile);
          SWIG_exit(EXIT_FAILURE);
       }
       f_init = NewString("");
       f_header = NewString("");
       f_wrappers = NewString("");

       /* Register file targets with the SWIG file handler */
       Swig_register_filebyname("header", f_header);
       Swig_register_filebyname("wrapper", f_wrappers);
       Swig_register_filebyname("runtime", f_runtime);
       Swig_register_filebyname("init", f_init);

       /* Output module initialization code */
       ...

       /* Emit code for children */
       Language::top(n);

       ...
       /* Write all to the file */
       Dump(f_header, f_runtime);
       Dump(f_wrappers, f_runtime);
       Wrapper_pretty_print(f_init, f_runtime);

       /* Cleanup files */
       Delete(f_header);
       Delete(f_wrappers);
       Delete(f_init);
       Close(f_runtime);
       Delete(f_runtime);

       return SWIG_OK;
    }

Using this to process a file will generate a wrapper file, however the wrapper will only consist of the common SWIG code as well as any inline code which was written in the .i file. It does not contain any wrappers for any of the functions or classes.

The code to generate the wrappers are the various member functions, which currently have not been touched. We will look at `functionWrapper()` as this is the most commonly used function. In fact many of the other wrapper routines will call this to do their work.

A simple modification to write some basic details to the wrapper looks like this:

    int Python::functionWrapper(Node *n) {
      /* Get some useful attributes of this function */
      String   *name   = Getattr(n,"sym:name");
      SwigType *type   = Getattr(n,"type");
      ParmList *parms  = Getattr(n,"parms");
      String   *parmstr= ParmList_str_defaultargs(parms); // to string
      String   *func   = SwigType_str(type, NewStringf("%s(%s)", name, parmstr));
      String   *action = Getattr(n,"wrap:action");

      Printf(f_wrappers,"functionWrapper   : %s\n", func);
      Printf(f_wrappers,"           action : %s\n", action);
      return SWIG_OK;
    }

This will now produce some useful information within your wrapper file.

    functionWrapper   : void delete_Shape(Shape *self)
               action : delete arg1;

    functionWrapper   : void Shape_x_set(Shape *self,double x)
               action : if (arg1) (arg1)->x = arg2;

    functionWrapper   : double Shape_x_get(Shape *self)
               action : result = (double) ((arg1)->x);

    functionWrapper   : void Shape_y_set(Shape *self,double y)
               action : if (arg1) (arg1)->y = arg2;
    ...

### 34.10.7 Low-level code generators

As ingenious as SWIG is, and despite all its capabilities and the power of its parser, the Low-level code generation takes a lot of work to write properly. Mainly because every language insists on its own manner of interfacing to C/C++. To write the code generators you will need a good understanding of how to manually write an interface to your chosen language, so make sure you have your documentation handy.

At this point it is also probably a good idea to take a very simple file (just one function), and try letting SWIG generate wrappers for many different languages. Take a look at all of the wrappers generated, and decide which one looks closest to the language you are trying to wrap. This may help you to decide which code to look at.

In general most language wrappers look a little like this:

    /* wrapper for TYPE3 some_function(TYPE1,TYPE2); */
    RETURN_TYPE _wrap_some_function(ARGS){
      TYPE1 arg1;
      TYPE2 arg2;
      TYPE3 result;

      if(ARG1 is not of TYPE1) goto fail;
      arg1=(convert ARG1);
      if(ARG2 is not of TYPE2) goto fail;
      arg2=(convert ARG2);

      result=some_function(arg1,arg2);

      convert 'result' to whatever the language wants;

      do any tidy up;

      return ALL_OK;

      fail:
      do any tidy up;
      return ERROR;
    }

Yes, it is rather vague and not very clear. But each language works differently so this will have to do for now.

Tackling this problem will be done in two stages:

- The skeleton: the function wrapper, and call, but without the conversion
- The conversion: converting the arguments to-from what the language wants

The first step will be done in the code, the second will be done in typemaps.

Our first step will be to write the code for `functionWrapper()`. What is shown below is NOT the solution, merely a step in the right direction. There are a lot of issues to address.

- Variable length and default parameters
- Typechecking and number of argument checks
- Overloaded functions
- Inout and Output only arguments

    virtual int functionWrapper(Node *n) {
      /* get useful atributes */
      String   *name   = Getattr(n,"sym:name");
      SwigType *type   = Getattr(n,"type");
      ParmList *parms  = Getattr(n,"parms");
      ...

      /* create the wrapper object */
      Wrapper *wrapper = NewWrapper();

      /* create the functions wrappered name */
      String *wname = Swig_name_wrapper(iname);

      /* deal with overloading */
      ....

      /* write the wrapper function definition */
      Printv(wrapper->def,"RETURN_TYPE ", wname, "(ARGS) {",NIL);

      /* if any additional local variable needed, add them now */
      ...

      /* write the list of locals/arguments required */
      emit_args(type, parms, wrapper);

      /* check arguments */
      ...

      /* write typemaps(in) */
      ....

      /* write constriants */
      ....

      /* Emit the function call */
      emit_action(n,wrapper);

      /* return value if necessary  */
      ....

      /* write typemaps(out) */
      ....

      /* add cleanup code */
      ....

      /* Close the function(ok) */
      Printv(wrapper->code, "return ALL_OK;\n", NIL);

      /* add the failure cleanup code */
      ...

      /* Close the function(error) */
      Printv(wrapper->code, "return ERROR;\n", "}\n", NIL);

      /* final substititions if applicable */
      ...

      /* Dump the function out */
      Wrapper_print(wrapper,f_wrappers);

      /* tidy up */
      Delete(wname);
      DelWrapper(wrapper);

      return SWIG_OK;
    }

Executing this code will produce wrappers which have our basic skeleton but without the typemaps, there is still work to do.

### 34.10.8 Configuration files

At the time of this writing, SWIG supports nearly a dozen languages, which means that for continued sanity in maintaining the configuration files, the language modules need to follow some conventions. These are outlined here along with the admission that, yes it is ok to violate these conventions in minor ways, as long as you know where to apply the proper kludge to keep the overall system regular and running. Engineering is the art of compromise, see...

Much of the maintenance regularity depends on choosing a suitable nickname for your language module (and then using it in a controlled way). Nicknames should be all lower case letters with an optional numeric suffix (no underscores, no dashes, no spaces). Some examples are: `foo`, `bar`, `qux99`.

The numeric suffix variant, as in the last example, is somewhat tricky to work with because sometimes people expect to refer to the language without this number but sometimes that number is extremely relevant (especially when it corresponds to language implementation versions with incompatible interfaces). New language modules that unavoidably require a numeric suffix in their nickname should include that number in all uses, or be prepared to kludge.

The nickname is used in four places:

usage | transform
"skip" tag | (none)
Examples/ subdir name | (none)
Examples/GIFPlot/ subdir name | capitalize (upcase first letter)
Examples/test-suite/ subdir name | (none)

As you can see, most usages are direct.

configure.in
  This file is processed by

  autoconf to generate the `configure` script. This is where you need to add shell script fragments and autoconf macros to detect the presence of whatever development support your language module requires, typically directories where headers and libraries can be found, and/or utility programs useful for integrating the generated wrapper code.

  Use the `AC_ARG_WITH`, `AC_MSG_CHECKING`, `AC_SUBST` macros and so forth (see other languages for examples). Avoid using the `[` and `]` character in shell script fragments. The variable names passed to `AC_SUBST` should begin with the nickname, entirely upcased.

  At the end of the new section is the place to put the aforementioned nickname kludges (should they be needed). See Perl5 and Php4 for examples of what to do. [If this is still unclear after you've read the code, ping me and I'll expand on this further. --ttn]

Makefile.in

  Some of the variables AC_SUBSTituted are essential to the support of your language module. Fashion these into a shell script "test" clause and assign that to a skip tag using "-z" and "-o":

  `skip-qux99 = [ -z "@QUX99INCLUDE@" -o -z "@QUX99LIBS" ]`

  This means if those vars should ever be empty, qux99 support should be considered absent and so it would be a good idea to skip actions that might rely on it.

  Here is where you may also define an alias (but then you'll need to kludge --- don't do this):

  `skip-qux = $(skip-qux99)`

  Lastly, you need to modify each of `check-aliveness`, `check-examples`, `check-test-suite`, `check-gifplot` (all targets) and `lib-languages` (var). Use the nickname for these, not the alias. Note that you can do this even before you have any tests or examples set up; the Makefile rules do some sanity checking and skip around these kinds of problems.

Examples/Makefile.in
  Nothing special here; see comments at top the of this file and look to the existing languages for examples.
Examples/qux99/check.list
  Do `cp ../python/check.list .` and modify to taste. One subdir per line.
Examples/GIFPlot/Qux99/check.list
  Do `cp ../Python/check.list .` and modify to taste. One subdir per line.
Lib/qux99/extra-install.list
  If you add your language to the top-level Makefile.in var `lib-languages`, then `make install` will install all `*.i` and `*.swg` files from the language-specific subdirectory of `Lib`. Use (optional) file `extra-install.list` in that directory to name additional files to install (see ruby for example).
Source/Modules/Makefile.am
  Add appropriate files to this Automake file. That's it!

  When you have modified these files, please make sure that the new language module is completely ignored if it is not installed and detected on a box, that is, `make check-examples` and `make check-test-suite` politely displays the ignoring language message.

### 34.10.9 Runtime support

Discuss the kinds of functions typically needed for SWIG runtime support (e.g. `SWIG_ConvertPtr()` and `SWIG_NewPointerObj()`) and the names of the SWIG files that implement those functions.

### 34.10.10 Standard library files

The standard library files that most languages supply keeps growing as SWIG matures. The following are the minimum that are usually supported:

- typemaps.i
- std_string.i
- std_vector.i
- stl.i

Please copy these and modify for any new language.

### 34.10.11 Examples and test cases

Each of the language modules provides one or more examples. These examples are used to demonstrate different features of the language module to SWIG end-users, but you'll find that they're useful during development and testing of your language module as well. You can use examples from the existing SWIG language modules for inspiration.

Each example is self-contained and consists of (at least) a `Makefile`, a SWIG interface file for the example module, and a 'runme' script that demonstrates the functionality for that module. All of these files are stored in the same subdirectory under the `Examples/[lang]` directory. There are two classic examples which should be the first to convert to a new language module. These are the "simple" C example and the "class" C++ example. These can be found, for example for Python, in `Examples/python/simple` and `Examples/python/class`.

By default, all of the examples are built and run when the user types `make check`. To ensure that your examples are automatically run during this process, see the section on configuration files.

### 34.10.12 Documentation

Don't forget to write end-user documentation for your language module. Currently, each language module has a dedicated chapter You shouldn't rehash things that are already covered in sufficient detail in the SWIG Basics and SWIG and C++ chapters. There is no fixed format for what, exactly, you should document about your language module, but you'll obviously want to cover issues that are unique to your language.

Some topics that you'll want to be sure to address include:

- Command line options unique to your language module.
- Non-obvious mappings between C/C++ and target language concepts. For example, if your target language provides a single floating point type, it should be no big surprise to find that C/C++ `float` and `double` types are mapped to it. On the other hand, if your target language doesn't provide support for "classes" or something similar, you'd want to discuss how C++ classes are handled.
- How to compile the SWIG-generated wrapper code into shared libraries that can actually be used. For some languages, there are well-defined procedures for doing this, but for others it's an ad hoc process. Provide as much detail as appropriate, and links to other resources if available.

### 34.10.13 Prerequisites for adding a new language module to the SWIG distribution

If you wish for a new language module to be distributed with SWIG, which we encourage for all popular languages, there are a few requirements. While we appreciate that getting all aspects of a new language working won't happen at the outset, there are a set of minimum requirements before a module can be committed into the SVN repository for distribution with future versions of SWIG. The following are really a summary of this whole section with details being outlined earlier on.

1. Demonstrate basic C code working by porting the "simple" example including a runtime test, see for example `Examples/python/simple`.
2. Demonstrate basic C++ code working by porting the "class" example including a runtime test, see for example `Examples/python/class`.
3. Modify `configure.in`, `Makefile.in` and `Examples/Makefile.in` to run these examples. Please make sure that if the new language is not installed properly on a box, `make -k check` should still work by skipping the tests and examples for the new language module.
4. Get the test-suite running for the new language (`make check-[lang]-test-suite`). While the test-suite tests many corner cases, we'd expect the majority of it to work by compiling the generated code correctly as most of the corner cases are covered in the SWIG core. Get at least one C and one C++ runtime test running in the test-suite.
5. Provide a chapter in the html documentation on the basics of using the language module.
6. Ensure your source code is formatted according to the coding style guidelines.
7. Finally, email the SWIG developers with a patch and a demonstration of commitment to maintaining the language module, certainly in the short term and ideally long term.

Once accepted into SVN, development efforts should concentrate on getting the entire test-suite to work with plenty of runtime tests. Runtime tests should be for existing testcases and new test cases should be added should there be an area not already covered by the existing tests.

### 34.10.14 Coding style guidelines

The coding guidelines for the C/C++ source code are pretty much K&R C style. The style can be inferred from the existing code base and is largely dictated by the `indent` code beautifier tool set to K&R style. The code can formatted using the make targets in the Source directory. Below is an example of how to format the emit.cxx file:

>     $ cd Source
>     $ make beautify-file INDENTFILE=Modules/emit.cxx

Of particular note is indentation is set to 2 spaces and a tab is used instead of 8 spaces. The generated C/C++ code should also follow this style as close as possible. However, tabs should be avoided as unlike the SWIG developers, users will never have consistent tab settings.

## 34.11 Typemaps

### 34.11.1 Proxy classes

## 34.12 Guide to parse tree nodes

This section describes the different parse tree nodes and their attributes.

cdecl

Describes general C declarations including variables, functions, and typedefs. A declaration is parsed as "storage T D" where storage is a storage class, T is a base type, and D is a declarator.

    "name"          - Declarator name
    "type"          - Base type T
    "decl"          - Declarator type (abstract)
    "storage"       - Storage class (static, extern, typedef, etc.)
    "parms"         - Function parameters (if a function)
    "code"          - Function body code (if supplied)
    "value"         - Default value (if supplied)

constructor

C++ constructor declaration.

    "name"          - Name of constructor
    "parms"         - Parameters
    "decl"          - Declarator (function with parameters)
    "code"          - Function body code (if any)
    "feature:new"   - Set to indicate return of new object.

destructor

C++ destructor declaration.

    "name"          - Name of destructor
    "code"          - Function body code (if any)
    "storage"       - Storage class (set if virtual)
    "value"         - Default value (set if pure virtual).

access

C++ access change.

    "kind"          - public, protected, private

constant

Constant created by %constant or #define.

    "name"          - Name of constant.
    "type"          - Base type.
    "value"         - Value.
    "storage"       - Set to %constant
    "feature:immutable" - Set to indicate read-only

class

C++ class definition or C structure definition.

    "name"          - Name of the class.
    "kind"          - Class kind ("struct", "union", "class")
    "symtab"        - Enclosing symbol table.
    "tdname"        - Typedef name. Use for typedef struct { ... } A.
    "abstract"      - Set if class has pure virtual methods.
    "baselist"      - List of base class names.
    "storage"       - Storage class (if any)
    "unnamed"       - Set if class is unnamed.

enum

Enumeration.

    "name"          - Name of the enum (if supplied).
    "storage"       - Storage class (if any)
    "tdname"        - Typedef name (typedef enum { ... } name).
    "unnamed"       - Set if enum is unnamed.

enumitem

Enumeration value.

    "name"          - Name of the enum value.
    "type"          - Type (integer or char)
    "value"         - Enum value (if given)
    "feature:immutable" - Set to indicate read-only

namespace

C++ namespace.

    "name"          - Name of the namespace.
    "symtab"        - Symbol table for enclosed scope.
    "unnamed"       - Set if unnamed namespace
    "alias"         - Alias name. Set for namespace A = B;

using

C++ using directive.

    "name"          - Name of the object being referred to.
    "uname"         - Qualified name actually given to using.
    "node"          - Node being referenced.
    "namespace"     - Namespace name being reference (using namespace name)

classforward

A forward C++ class declaration.

    "name"          - Name of the class.
    "kind"          - Class kind ("union", "struct", "class")

insert

Code insertion directive. For example, %{ ... %} or %insert(section).

    "code"          - Inserted code
    "section"       - Section name ("header", "wrapper", etc.)

top

Top of the parse tree.

    "module"        - Module name

extend

%extend directive.

    "name"          - Module name
    "symtab"        - Symbol table of enclosed scope.

apply

%apply pattern { patternlist }.

    "pattern"       - Source pattern.
    "symtab"        - Symbol table of enclosed scope.

clear

%clear patternlist;

    "firstChild"    - Patterns to clear

include

%include directive.

    "name"         - Filename
    "firstChild"   - Children

import

%import directive.

    "name"         - Filename
    "firstChild"   - Children

module

%module directive.

    "name"         - Name of the module

typemap

%typemap directive.

    "method"       - Typemap method name.
    "code"         - Typemap code.
    "kwargs"       - Keyword arguments (if any)
    "firstChild"   - Typemap patterns

typemapcopy

%typemap directive with copy.

    "method"       - Typemap method name.
    "pattern"      - Typemap source pattern.
    "firstChild"   - Typemap patterns

typemapitem

%typemap pattern. Used with %apply, %clear, %typemap.

    "pattern"      - Typemap pattern (a parameter list)
    "parms"        - Typemap parameters.

types

%types directive.

    "parms"        - List of parameter types.
    "convcode"     - Code which replaces the default casting / conversion code

extern

extern "X" { ... } declaration.

    "name"       - Name "C", "Fortran", etc.