perlxs

NAME

perlxs - XS language reference manual


DESCRIPTION


Introduction

XS is a language used to create an extension interface between Perl and some C library which one wishes to use with Perl. The XS interface is combined with the library to create a new library which can be linked to Perl. An XSUB is a function in the XS language and is the core component of the Perl application interface.

The XS compiler is called xsubpp. This compiler will embed the constructs necessary to let an XSUB, which is really a C function in disguise, manipulate Perl values and creates the glue necessary to let Perl access the XSUB. The compiler uses typemaps to determine how to map C function parameters and variables to Perl values. The default typemap handles many common C types. A supplement typemap must be created to handle special structures and types for the library being linked.

See the perlxstut manpage for a tutorial on the whole extension creation process.


On The Road

Many of the examples which follow will concentrate on creating an interface between Perl and the ONC+ RPC bind library functions. Specifically, the rpcb_gettime() function will be used to demonstrate many features of the XS language. This function has two parameters; the first is an input parameter and the second is an output parameter. The function also returns a status value.

bool_t rpcb_gettime(const char *host, time_t *timep);

From C this function will be called with the following statements.

#include <rpc/rpc.h> bool_t status; time_t timep; status = rpcb_gettime( "localhost", &timep );

If an XSUB is created to offer a direct translation between this function and Perl, then this XSUB will be used from Perl with the following code. The $status and $timep variables will contain the output of the function.

use RPC; $status = rpcb_gettime( "localhost", $timep );

The following XS file shows an XS subroutine, or XSUB, which demonstrates one possible interface to the rpcb_gettime() function. This XSUB represents a direct translation between C and Perl and so preserves the interface even from Perl. This XSUB will be invoked from Perl with the usage shown above. Note that the first three #include statements, for EXTERN.h, perl.h, and XSUB.h, will always be present at the beginning of an XS file. This approach and others will be expanded later in this document.

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include <rpc/rpc.h> MODULE = RPC PACKAGE = RPC bool_t rpcb_gettime(host,timep) char *host time_t &timep OUTPUT: timep

Any extension to Perl, including those containing XSUBs, should have a Perl module to serve as the bootstrap which pulls the extension into Perl. This module will export the extension's functions and variables to the Perl program and will cause the extension's XSUBs to be linked into Perl. The following module will be used for most of the examples in this document and should be used from Perl with the use command as shown earlier. Perl modules are explained in more detail later in this document.

package RPC; require Exporter; require DynaLoader; @ISA = qw(Exporter DynaLoader); @EXPORT = qw( rpcb_gettime ); bootstrap RPC; 1;

Throughout this document a variety of interfaces to the rpcb_gettime() XSUB will be explored. The XSUBs will take their parameters in different orders or will take different numbers of parameters. In each case the XSUB is an abstraction between Perl and the real C rpcb_gettime() function, and the XSUB must always ensure that the real rpcb_gettime() function is called with the correct parameters. This abstraction will allow the programmer to create a more Perl-like interface to the C function.


The Anatomy of an XSUB

The following XSUB allows a Perl program to access a C library function called sin() . The XSUB will imitate the C function which takes a single argument and returns a single value.

double sin(x) double x

When using C pointers the indirection operator * should be considered part of the type and the address operator & should be considered part of the variable, as is demonstrated in the rpcb_gettime() function above. See the section on typemaps for more about handling qualifiers and unary operators in C types.

The function name and the return type must be placed on separate lines.

INCORRECT CORRECT double sin(x) double double x sin(x) double x


The Argument Stack

The argument stack is used to store the values which are sent as parameters to the XSUB and to store the XSUB's return value. In reality all Perl functions keep their values on this stack at the same time, each limited to its own range of positions on the stack. In this document the first position on that stack which belongs to the active function will be referred to as position 0 for that function.

XSUBs refer to their stack arguments with the macro ST(x), where x refers to a position in this XSUB's part of the stack. Position 0 for that function would be known to the XSUB as ST(0). The XSUB's incoming parameters and outgoing return values always begin at ST(0). For many simple cases the xsubpp compiler will generate the code necessary to handle the argument stack by embedding code fragments found in the typemaps. In more complex cases the programmer must supply the code.


The RETVAL Variable

The RETVAL variable is a magic variable which always matches the return type of the C library function. The xsubpp compiler will supply this variable in each XSUB and by default will use it to hold the return value of the C library function being called. In simple cases the value of RETVAL will be placed in ST(0) of the argument stack where it can be received by Perl as the return value of the XSUB.

If the XSUB has a return type of void then the compiler will not supply a RETVAL variable for that function. When using the PPCODE: directive the RETVAL variable may not be needed.


The MODULE Keyword

The MODULE keyword is used to start the XS code and to specify the package of the functions which are being defined. All text preceding the first MODULE keyword is considered C code and is passed through to the output untouched. Every XS module will have a bootstrap function which is used to hook the XSUBs into Perl. The package name of this bootstrap function will match the value of the last MODULE statement in the XS source files. The value of MODULE should always remain constant within the same XS file, though this is not required.

The following example will start the XS code and will place all functions in a package named RPC.

MODULE = RPC


The PACKAGE Keyword

When functions within an XS source file must be separated into packages the PACKAGE keyword should be used. This keyword is used with the MODULE keyword and must follow immediately after it when used.

MODULE = RPC PACKAGE = RPC [ XS code in package RPC ] MODULE = RPC PACKAGE = RPCB [ XS code in package RPCB ] MODULE = RPC PACKAGE = RPC [ XS code in package RPC ]

Although this keyword is optional and in some cases provides redundant information it should always be used. This keyword will ensure that the XSUBs appear in the desired package.


The PREFIX Keyword

The PREFIX keyword designates prefixes which should be removed from the Perl function names. If the C function is rpcb_gettime() and the PREFIX value is rpcb_ then Perl will see this function as gettime().

This keyword should follow the PACKAGE keyword when used. If PACKAGE is not used then PREFIX should follow the MODULE keyword.

MODULE = RPC PREFIX = rpc_ MODULE = RPC PACKAGE = RPCB PREFIX = rpcb_


The OUTPUT: Keyword

The OUTPUT: keyword indicates that certain function parameters should be updated (new values made visible to Perl) when the XSUB terminates or that certain values should be returned to the calling Perl function. For simple functions, such as the sin() function above, the RETVAL variable is automatically designated as an output value. In more complex functions the xsubpp compiler will need help to determine which variables are output variables.

This keyword will normally be used to complement the CODE: keyword. The RETVAL variable is not recognized as an output variable when the CODE: keyword is present. The OUTPUT: keyword is used in this situation to tell the compiler that RETVAL really is an output variable.

The OUTPUT: keyword can also be used to indicate that function parameters are output variables. This may be necessary when a parameter has been modified within the function and the programmer would like the update to be seen by Perl.

bool_t rpcb_gettime(host,timep) char *host time_t &timep OUTPUT: timep

The OUTPUT: keyword will also allow an output parameter to be mapped to a matching piece of code rather than to a typemap.

bool_t rpcb_gettime(host,timep) char *host time_t &timep OUTPUT: timep sv_setnv(ST(1), (double)timep);


The CODE: Keyword

This keyword is used in more complicated XSUBs which require special handling for the C function. The RETVAL variable is available but will not be returned unless it is specified under the OUTPUT: keyword.

The following XSUB is for a C function which requires special handling of its parameters. The Perl usage is given first.

$status = rpcb_gettime( "localhost", $timep );

The XSUB follows.

bool_t rpcb_gettime(host,timep) char *host time_t timep CODE: RETVAL = rpcb_gettime( host, &timep ); OUTPUT: timep RETVAL

In many of the examples shown here the CODE: block (and other blocks) will often be contained within braces ( { and } ). This protects the CODE: block from complex INPUT typemaps and ensures the resulting C code is legal.


The NO_INIT Keyword

The NO_INIT keyword is used to indicate that a function parameter is being used as only an output value. The xsubpp compiler will normally generate code to read the values of all function parameters from the argument stack and assign them to C variables upon entry to the function. NO_INIT will tell the compiler that some parameters will be used for output rather than for input and that they will be handled before the function terminates.

The following example shows a variation of the rpcb_gettime() function. This function uses the timep variable as only an output variable and does not care about its initial contents.

bool_t rpcb_gettime(host,timep) char *host time_t &timep = NO_INIT OUTPUT: timep


Initializing Function Parameters

Function parameters are normally initialized with their values from the argument stack. The typemaps contain the code segments which are used to transfer the Perl values to the C parameters. The programmer, however, is allowed to override the typemaps and supply alternate initialization code.

The following code demonstrates how to supply initialization code for function parameters. The initialization code is eval'd by the compiler before it is added to the output so anything which should be interpreted literally, such as double quotes, must be protected with backslashes.

bool_t rpcb_gettime(host,timep) char *host = (char *)SvPV(ST(0),na); time_t &timep = 0; OUTPUT: timep

This should not be used to supply default values for parameters. One would normally use this when a function parameter must be processed by another library function before it can be used. Default parameters are covered in the next section.


Default Parameter Values

Default values can be specified for function parameters by placing an assignment statement in the parameter list. The default value may be a number or a string. Defaults should always be used on the right-most parameters only.

To allow the XSUB for rpcb_gettime() to have a default host value the parameters to the XSUB could be rearranged. The XSUB will then call the real rpcb_gettime() function with the parameters in the correct order. Perl will call this XSUB with either of the following statements.

$status = rpcb_gettime( $timep, $host ); $status = rpcb_gettime( $timep );

The XSUB will look like the code which follows. A CODE: block is used to call the real rpcb_gettime() function with the parameters in the correct order for that function.

bool_t rpcb_gettime(timep,host="localhost") char *host time_t timep = NO_INIT CODE: RETVAL = rpcb_gettime( host, &timep ); OUTPUT: timep RETVAL


Variable-length Parameter Lists

XSUBs can have variable-length parameter lists by specifying an ellipsis (...) in the parameter list. This use of the ellipsis is similar to that found in ANSI C. The programmer is able to determine the number of arguments passed to the XSUB by examining the items variable which the xsubpp compiler supplies for all XSUBs. By using this mechanism one can create an XSUB which accepts a list of parameters of unknown length.

The host parameter for the rpcb_gettime() XSUB can be optional so the ellipsis can be used to indicate that the XSUB will take a variable number of parameters. Perl should be able to call this XSUB with either of the following statements.

$status = rpcb_gettime( $timep, $host ); $status = rpcb_gettime( $timep );

The XS code, with ellipsis, follows.

bool_t rpcb_gettime(timep, ...) time_t timep = NO_INIT CODE: { char *host = "localhost"; if( items > 1 ) host = (char *)SvPV(ST(1), na); RETVAL = rpcb_gettime( host, &timep ); } OUTPUT: timep RETVAL


The PPCODE: Keyword

The PPCODE: keyword is an alternate form of the CODE: keyword and is used to tell the xsubpp compiler that the programmer is supplying the code to control the argument stack for the XSUBs return values. Occasionally one will want an XSUB to return a list of values rather than a single value. In these cases one must use PPCODE: and then explicitly push the list of values on the stack. The PPCODE: and CODE: keywords are not used together within the same XSUB.

The following XSUB will call the C rpcb_gettime() function and will return its two output values, timep and status, to Perl as a single list.

void rpcb_gettime(host) char *host PPCODE: { time_t timep; bool_t status; status = rpcb_gettime( host, &timep ); EXTEND(sp, 2); PUSHs(sv_2mortal(newSViv(status))); PUSHs(sv_2mortal(newSViv(timep))); }

Notice that the programmer must supply the C code necessary to have the real rpcb_gettime() function called and to have the return values properly placed on the argument stack.

The void return type for this function tells the xsubpp compiler that the RETVAL variable is not needed or used and that it should not be created. In most scenarios the void return type should be used with the PPCODE: directive.

The EXTEND() macro is used to make room on the argument stack for 2 return values. The PPCODE: directive causes the xsubpp compiler to create a stack pointer called sp, and it is this pointer which is being used in the EXTEND() macro. The values are then pushed onto the stack with the PUSHs() macro.

Now the rpcb_gettime() function can be used from Perl with the following statement.

($status, $timep) = rpcb_gettime("localhost");


Returning Undef And Empty Lists

Occasionally the programmer will want to simply return undef or an empty list if a function fails rather than a separate status value. The rpcb_gettime() function offers just this situation. If the function succeeds we would like to have it return the time and if it fails we would like to have undef returned. In the following Perl code the value of $timep will either be undef or it will be a valid time.

$timep = rpcb_gettime( "localhost" );

The following XSUB uses the void return type to disable the generation of the RETVAL variable and uses a CODE: block to indicate to the compiler that the programmer has supplied all the necessary code. The sv_newmortal() call will initialize the return value to undef, making that the default return value.

void rpcb_gettime(host) char * host CODE: { time_t timep; bool_t x; ST(0) = sv_newmortal(); if( rpcb_gettime( host, &timep ) ) sv_setnv( ST(0), (double)timep); }

The next example demonstrates how one would place an explicit undef in the return value, should the need arise.

void rpcb_gettime(host) char * host CODE: { time_t timep; bool_t x; ST(0) = sv_newmortal(); if( rpcb_gettime( host, &timep ) ){ sv_setnv( ST(0), (double)timep); } else{ ST(0) = &sv_undef; } }

To return an empty list one must use a PPCODE: block and then not push return values on the stack.

void rpcb_gettime(host) char *host PPCODE: { time_t timep; if( rpcb_gettime( host, &timep ) ) PUSHs(sv_2mortal(newSViv(timep))); else{ /* Nothing pushed on stack, so an empty */ /* list is implicitly returned. */ } }


The REQUIRE: Keyword

The REQUIRE: keyword is used to indicate the minimum version of the xsubpp compiler needed to compile the XS module. An XS module which contains the following statement will only compile with xsubpp version 1.922 or greater:

REQUIRE: 1.922


The CLEANUP: Keyword

This keyword can be used when an XSUB requires special cleanup procedures before it terminates. When the CLEANUP: keyword is used it must follow any CODE:, PPCODE:, or OUTPUT: blocks which are present in the XSUB. The code specified for the cleanup block will be added as the last statements in the XSUB.


The BOOT: Keyword

The BOOT: keyword is used to add code to the extension's bootstrap function. The bootstrap function is generated by the xsubpp compiler and normally holds the statements necessary to register any XSUBs with Perl. With the BOOT: keyword the programmer can tell the compiler to add extra statements to the bootstrap function.

This keyword may be used any time after the first MODULE keyword and should appear on a line by itself. The first blank line after the keyword will terminate the code block.

BOOT: # The following message will be printed when the # bootstrap function executes. printf("Hello from the bootstrap!\n");


Inserting Comments and C Preprocessor Directives

Comments and C preprocessor directives are allowed within CODE:, PPCODE:, BOOT:, and CLEANUP: blocks. The compiler will pass the preprocessor directives through untouched and will remove the commented lines. Comments can be added to XSUBs by placing a # at the beginning of the line. Care should be taken to avoid making the comment look like a C preprocessor directive, lest it be interpreted as such.


Using XS With C++

If a function is defined as a C++ method then it will assume its first argument is an object pointer. The object pointer will be stored in a variable called THIS. The object should have been created by C++ with the new() function and should be blessed by Perl with the sv_setref_pv() macro. The blessing of the object by Perl can be handled by a typemap. An example typemap is shown at the end of this section.

If the method is defined as static it will call the C++ function using the class::method() syntax. If the method is not static the function will be called using the THIS->method() syntax.

The next examples will use the following C++ class.

class colors { public: colors(); ~colors(); int blue(); void set_blue( int ); private: int c_blue; };

The XSUBs for the blue() and set_blue() methods are defined with the class name but the parameter for the object (THIS, or ``self'') is implicit and is not listed.

int color::blue() void color::set_blue( val ) int val

Both functions will expect an object as the first parameter. The xsubpp compiler will call that object THIS and will use it to call the specified method. So in the C++ code the blue() and set_blue() methods will be called in the following manner.

RETVAL = THIS->blue(); THIS->set_blue( val );

If the function's name is DESTROY then the C++ delete function will be called and THIS will be given as its parameter.

void color::DESTROY()

The C++ code will call delete .

delete THIS;

If the function's name is new then the C++ new function will be called to create a dynamic C++ object. The XSUB will expect the class name, which will be kept in a variable called CLASS, to be given as the first argument.

color * color::new()

The C++ code will call new.

RETVAL = new color();

The following is an example of a typemap that could be used for this C++ example.

TYPEMAP color * O_OBJECT OUTPUT # The Perl object is blessed into 'CLASS', which should be a # char* having the name of the package for the blessing. O_OBJECT sv_setref_pv( $arg, CLASS, (void*)$var ); INPUT O_OBJECT if( sv_isobject($arg) && (SvTYPE(SvRV($arg)) == SVt_PVMG) ) $var = ($type)SvIV((SV*)SvRV( $arg )); else{ warn( \"${Package}::$func_name() -- $var is not a blessed SV reference\" ); XSRETURN_UNDEF; }


Interface Strategy

When designing an interface between Perl and a C library a straight translation from C to XS is often sufficient. The interface will often be very C-like and occasionally nonintuitive, especially when the C function modifies one of its parameters. In cases where the programmer wishes to create a more Perl-like interface the following strategy may help to identify the more critical parts of the interface.

Identify the C functions which modify their parameters. The XSUBs for these functions may be able to return lists to Perl, or may be candidates to return undef or an empty list in case of failure.

Identify which values are used by only the C and XSUB functions themselves. If Perl does not need to access the contents of the value then it may not be necessary to provide a translation for that value from C to Perl.

Identify the pointers in the C function parameter lists and return values. Some pointers can be handled in XS with the & unary operator on the variable name while others will require the use of the * operator on the type name. In general it is easier to work with the & operator.

Identify the structures used by the C functions. In many cases it may be helpful to use the T_PTROBJ typemap for these structures so they can be manipulated by Perl as blessed objects.


Perl Objects And C Structures

When dealing with C structures one should select either T_PTROBJ or T_PTRREF for the XS type. Both types are designed to handle pointers to complex objects. The T_PTRREF type will allow the Perl object to be unblessed while the T_PTROBJ type requires that the object be blessed. By using T_PTROBJ one can achieve a form of type-checking because the XSUB will attempt to verify that the Perl object is of the expected type.

The following XS code shows the getnetconfigent() function which is used with ONC+ TIRPC. The getnetconfigent() function will return a pointer to a C structure and has the C prototype shown below. The example will demonstrate how the C pointer will become a Perl reference. Perl will consider this reference to be a pointer to a blessed object and will attempt to call a destructor for the object. A destructor will be provided in the XS source to free the memory used by getnetconfigent(). Destructors in XS can be created by specifying an XSUB function whose name ends with the word DESTROY. XS destructors can be used to free memory which may have been malloc'd by another XSUB.

struct netconfig *getnetconfigent(const char *netid);

A typedef will be created for struct netconfig. The Perl object will be blessed in a class matching the name of the C type, with the tag Ptr appended, and the name should not have embedded spaces if it will be a Perl package name. The destructor will be placed in a class corresponding to the class of the object and the PREFIX keyword will be used to trim the name to the word DESTROY as Perl will expect.

typedef struct netconfig Netconfig; MODULE = RPC PACKAGE = RPC Netconfig * getnetconfigent(netid) char *netid MODULE = RPC PACKAGE = NetconfigPtr PREFIX = rpcb_ void rpcb_DESTROY(netconf) Netconfig *netconf CODE: printf("Now in NetconfigPtr::DESTROY\n"); free( netconf );

This example requires the following typemap entry. Consult the typemap section for more information about adding new typemaps for an extension.

TYPEMAP Netconfig * T_PTROBJ

This example will be used with the following Perl statements.

use RPC; $netconf = getnetconfigent("udp");

When Perl destroys the object referenced by $netconf it will send the object to the supplied XSUB DESTROY function. Perl cannot determine, and does not care, that this object is a C struct and not a Perl object. In this sense, there is no difference between the object created by the getnetconfigent() XSUB and an object created by a normal Perl subroutine.


The Typemap

The typemap is a collection of code fragments which are used by the xsubpp compiler to map C function parameters and values to Perl values. The typemap file may consist of three sections labeled TYPEMAP, INPUT, and OUTPUT. The INPUT section tells the compiler how to translate Perl values into variables of certain C types. The OUTPUT section tells the compiler how to translate the values from certain C types into values Perl can understand. The TYPEMAP section tells the compiler which of the INPUT and OUTPUT code fragments should be used to map a given C type to a Perl value. Each of the sections of the typemap must be preceded by one of the TYPEMAP, INPUT, or OUTPUT keywords.

The default typemap in the ext directory of the Perl source contains many useful types which can be used by Perl extensions. Some extensions define additional typemaps which they keep in their own directory. These additional typemaps may reference INPUT and OUTPUT maps in the main typemap. The xsubpp compiler will allow the extension's own typemap to override any mappings which are in the default typemap.

Most extensions which require a custom typemap will need only the TYPEMAP section of the typemap file. The custom typemap used in the getnetconfigent() example shown earlier demonstrates what may be the typical use of extension typemaps. That typemap is used to equate a C structure with the T_PTROBJ typemap. The typemap used by getnetconfigent() is shown here. Note that the C type is separated from the XS type with a tab and that the C unary operator * is considered to be a part of the C type name.

TYPEMAP Netconfig *<tab>T_PTROBJ


EXAMPLES

File RPC.xs: Interface to some ONC+ RPC bind library functions.

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include <rpc/rpc.h> typedef struct netconfig Netconfig; MODULE = RPC PACKAGE = RPC void rpcb_gettime(host="localhost") char *host CODE: { time_t timep; ST(0) = sv_newmortal(); if( rpcb_gettime( host, &timep ) ) sv_setnv( ST(0), (double)timep ); } Netconfig * getnetconfigent(netid="udp") char *netid MODULE = RPC PACKAGE = NetconfigPtr PREFIX = rpcb_ void rpcb_DESTROY(netconf) Netconfig *netconf CODE: printf("NetconfigPtr::DESTROY\n"); free( netconf );

File typemap: Custom typemap for RPC.xs.

TYPEMAP Netconfig * T_PTROBJ

File RPC.pm: Perl module for the RPC extension.

package RPC; require Exporter; require DynaLoader; @ISA = qw(Exporter DynaLoader); @EXPORT = qw(rpcb_gettime getnetconfigent); bootstrap RPC; 1;

File rpctest.pl: Perl test program for the RPC extension.

use RPC; $netconf = getnetconfigent(); $a = rpcb_gettime(); print "time = $a\n"; print "netconf = $netconf\n"; $netconf = getnetconfigent("tcp"); $a = rpcb_gettime("poplar"); print "time = $a\n"; print "netconf = $netconf\n";


AUTHOR

Dean Roehrich <roehrich@cray.com> Dec 10, 1995