1
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>str.tokenize.keep.delimiters</title><link rel="stylesheet" type="text/css" href="../reference.css"><meta name="generator" content="DocBook XSL Stylesheets V1.76.1"><link rel="home" href="../index.html" title="DocBook XSL Stylesheets: Reference�Documentation"><link rel="up" href="generallibrary.html" title="General Library Templates"><link rel="prev" href="trim.text.html" title="trim.text"><link rel="next" href="apply-string-subst-map.html" title="apply-string-subst-map"><link rel="copyright" href="copyright.html" title="License"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">str.tokenize.keep.delimiters</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="trim.text.html">Prev</a>�</td><th width="60%" align="center">General Library Templates</th><td width="20%" align="right">�<a accesskey="n" href="apply-string-subst-map.html">Next</a></td></tr></table><hr></div><div class="refentry" title="str.tokenize.keep.delimiters"><a name="str.tokenize.keep.delimiters"></a><div class="titlepage"></div>
1
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>str.tokenize.keep.delimiters</title><link rel="stylesheet" type="text/css" href="../reference.css"><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"><link rel="home" href="../index.html" title="DocBook XSL Stylesheets: Reference�Documentation"><link rel="up" href="generallibrary.html" title="General Library Templates"><link rel="prev" href="trim.text.html" title="trim.text"><link rel="next" href="apply-string-subst-map.html" title="apply-string-subst-map"><link rel="copyright" href="copyright.html" title="License"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">str.tokenize.keep.delimiters</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="trim.text.html">Prev</a>�</td><th width="60%" align="center">General Library Templates</th><td width="20%" align="right">�<a accesskey="n" href="apply-string-subst-map.html">Next</a></td></tr></table><hr></div><div class="refentry"><a name="str.tokenize.keep.delimiters"></a><div class="titlepage"></div>
2
2
<div class="refnamediv"><h2>str.tokenize.keep.delimiters</h2><p>
3
3
str.tokenize.keep.delimiters
4
4
— Tokenize a string while preserving any delimiters
7
7
<p>Based on the occurrence of one or more delimiter characters,
8
8
this function breaks a string into a list of tokens and delimiters,
9
9
marking up each of the tokens with a <a href="http://docbook.org/tdg5/en/html/token.html"><code class="sgmltag-element">token</code></a> element
10
10
and preserving the delimiters as text nodes between the tokens.</p>
12
12
<p>This function is a very slightly modified version of a
13
13
function from the <a class="ulink" href="http://www.exslt.org/" target="_top">EXSLT
14
14
site</a>. The original is available at:
added
removed
doc/lib/str.tokenize.keep.delimiters.html
