EXSLT - str:tokenize - EXSLT Function Implementation

Supported Version: 1
XML Definition: str.tokenize.xml
Source: str.tokenize.function.xsl

This implementation relies on the exsl:node-set() function.

Source

<func:function name="str:tokenize">
   <xsl:param name="string"
              select="''" />
   <xsl:param name="delimiters"
              select="' &#x9;
'" />
   <xsl:choose>
      <xsl:when test="not($string)">
         <func:result select="/.." />
      </xsl:when>
      <xsl:when test="not(function-available('exsl:node-set'))">
         <xsl:message terminate="yes">
        ERROR: EXSLT - Functions implementation of str:tokenize relies on exsl:node-set().
      </xsl:message>
      </xsl:when>
      <xsl:otherwise>
         <xsl:variable name="tokens">
            <xsl:choose>
               <xsl:when test="not($delimiters)">
                  <xsl:call-template name="str:_tokenize-characters">
                     <xsl:with-param name="string"
                                     select="$string" />
                  </xsl:call-template>
               </xsl:when>
               <xsl:otherwise>
                  <xsl:call-template name="str:_tokenize-delimiters">
                     <xsl:with-param name="string"
                                     select="$string" />
                     <xsl:with-param name="delimiters"
                                     select="$delimiters" />
                  </xsl:call-template>
               </xsl:otherwise>
            </xsl:choose>
         </xsl:variable>
         <func:result select="exsl:node-set($tokens)/token" />
      </xsl:otherwise>
   </xsl:choose>
</func:function>
<xsl:template name="str:_tokenize-characters">
   <xsl:param name="string" />
   <xsl:if test="$string">
      <token>
         <xsl:value-of select="substring($string, 1, 1)" />
      </token>
      <xsl:call-template name="str:_tokenize-characters">
         <xsl:with-param name="string"
                         select="substring($string, 2)" />
      </xsl:call-template>
   </xsl:if>
</xsl:template>
<xsl:template name="str:_tokenize-delimiters">
   <xsl:param name="string" />
   <xsl:param name="delimiters" />
   <xsl:variable name="delimiter"
                 select="substring($delimiters, 1, 1)" />
   <xsl:choose>
      <xsl:when test="not($delimiter)">
         <token>
            <xsl:value-of select="$string" />
         </token>
      </xsl:when>
      <xsl:when test="contains($string, $delimiter)">
         <xsl:if test="not(starts-with($string, $delimiter))">
            <xsl:call-template name="str:_tokenize-delimiters">
               <xsl:with-param name="string"
                               select="substring-before($string, $delimiter)" />
               <xsl:with-param name="delimiters"
                               select="substring($delimiters, 2)" />
            </xsl:call-template>
         </xsl:if>
         <xsl:call-template name="str:_tokenize-delimiters">
            <xsl:with-param name="string"
                            select="substring-after($string, $delimiter)" />
            <xsl:with-param name="delimiters"
                            select="$delimiters" />
         </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
         <xsl:call-template name="str:_tokenize-delimiters">
            <xsl:with-param name="string"
                            select="$string" />
            <xsl:with-param name="delimiters"
                            select="substring($delimiters, 2)" />
         </xsl:call-template>
      </xsl:otherwise>
   </xsl:choose>
</xsl:template>

Change History

Submitted: 2001-06-03
Creator: Jeni Tennison(http://www.jenitennison.com/)

The str:tokenize function splits a string at the occurrences of given characters.

Version	Modified	By	Details
1.1	2001-06-21	Jeni Tennison	Added Javascript and MSXML implementations.
1.2	2001-11-18	Jeni Tennison	Indicated behaviour when second argument is an empty string; str:tokenize() should return a list of tokens, each containing a single character.
1.3	2002-08-21	Craig Stewart	Added 4XSLT and libxslt implementation to the list.
1.4	2002-11-12	Craig Stewart	Updated 4XSLT version to 0.12.0a3.