~zorba-coders/zorba/tokenize

Zorba - The XQuery Processor

version 2.1

New Features:
  * New node-position module. This module allows to obtain a representation of a node position, which
    can be used to assess structural relationships with other nodes.   
  * New node-reference module. References can be obtained for any node, and
	different nodes cannot have the same identifier.
  * Custom Full-text thesaurus using Zorba URI resolver mechanism.
  * Modified C++ API to add isSequential, getExternalVariables, isBoundContextItem and 
  isBoundExternalVariable Functions
  * Collection allows to retrieve information about the type
  and the annotations of a static collection.
  * Implemented the probe-index-range-value for general indexes
  * Added ExternalFunctionParameter class to be registered in the DynamicContext
  * New module for compiling and executing XQueries from XQuery (xqxq)
  * External functions are allowed to be updating (i.e. return a pending update list)

Optimization:
  * Optimization for positioned sequence access (e.g. $x[$x[1]] where $x is a
  sequence of integers)
  * Optimization for count(collection()) expressions
  *	Optimization of the fn:substring function in the case when its $start and $length arguments
  are integers

Bug Fixes/Other Changes:
  * Fixed bug #3403291 (build for Fedora 15)
  * Fixed bug #3397293 (Incorrect XQueryX error message)
  * Fixed bug #3398637 (Nested element constructors cause loss of namespace bindings)
  * Fixed bug #3401592 (Undo of Create Collection doesn't work)
  * Fixed bug #3404689 (assignment problem in flwor)
  * Fix for bug #3404450 (testdriver_mt problem: caching and validation of docs)
  * Fixed installation bug to not install the .so for test modules
  * Fixed bug #3390410 (Adding a node to a collection causes out of bound in vector)
  * Fixed bug #3406200 (node types were not being moved to new tree during node detach)
  * Fixed bug #3406191 (Namespace bindings lost after update due to bug in node detach)
  * Fixed bug #3290122 (ZDST0003 needs collection name)
  * Fixed bug #3406272 (merging of adjacent text nodes must be done after all
    updates have been applied).
  * Don't raise warnings for unknown annotations.
  * Fixed bug #3409344 (during detach, if the node being detached is the root of
    its tree, it should be detached from the tree as well; otherwise memory
    corruption will occur)
  * Fixed bug # (Should not destroy the parent of a node that is being detached
    before the detach is done).
  * Added const qualifier to StaticContext::getAuditEvent() 
  * Fixed bug #3408181 (available-collection() returns undeclared collections)
  * Fixed bug #859465 (Fatal error if a PUL contains two deactivate IC primitives)
  * Fixed bug #859467 (Fatal error if a PUL contains two activate Foreign Key primitives)
  * Fixed bug #859468 (Fatal error if a PUL contains two activate Simple IC primitives)
  * Fixed bug #859522 (Fatal error if a PUL contains two delete document primitives)
  * Fixed bug #859968 (Using put on an attribute produces an invalid XDM instance and may cause segmentation faults)
  * Fixed bug #855314 (should not fold a constant expr that returns an error item)
  * Fixed bug when parsing a document with a base-uri attribute.
  * Fixed bug #863320 (Sentence is incorrectly incremented when token characters end without sentence terminator)
  * Fixed bug #863730 (static delete-node* functions don't raise ZDDY0012)
  * Removed ZSTR0005 and ZSTR0006 error codes
  * Fixed bug #868258 (Assertion failure with two delete collection)
  * Fixed bug #871623 and #871629 (assertion failures with insertions in dynamic collections)
  * Fixed bug #867262 (allow reuse of iterator over ExtFuncArgItemSequence)
  * Fixed bug #869024 (segmentation fault with node-reference)
  * Fixed bug #869025 (segmentation fault with node-reference)
  * New node-reference module. References can be obtained for any node, and
	different nodes cannot have the same identifier.
  * Fixed bug #872697  (segmentation fault with validation of NMTOKENS)
  * Added undo for node revalidation
  * Optimization for count(collection()) expressions
  * Fixed bug #872796  (validate-in-place can interfere with other update primitives)
  * Fixed bug #872799 (validate-in-place can set incorrect types)
  * Fixed bug #855715 (Invalid escaped characters in regex not caught)
  * Fixed bug #868325 (fn:analyze-string fails with some recursive subgroups)
  * Fixed bug #872697 (segmentation fault with validation of NMTOKENS)
  * General index cannot be declared as unique if the type of its key is
  xs:anyAtomicType or xs:untypedAtomic.
  * Added undo for node revalidation
  * Fixed bug #867133 (SWIG PHP build failure on Mac OSX)
  * Fixed bug #872796 (validate-in-place can interfere with other update primitives)
  * Fixed bug #872799 (validate-in-place can set incorrect types)
  * Fixed bug #872850 (Serialization of Tumbling Window For fails)
  * Fixed bug #855715 (Invalid escaped characters in regex not caught)
  * Fixed bug #862089 (Split binary/xq install directories for modules) by
  splitting "module path" into separate URI and Library paths
  * Fixed bug #872502 (validation of the JSON module xqdoc fails)
  * Fixed bug #897619 (testdriver_mt can not run the XQueryX tests)
  * Fixed bug #867107 (xqdoc dependency to zorba is wrong)
  * Fixed bug #897616 (Fix the generation of the XQFTTS report results for
  W3C)

version 2.0.3
  * Fixed bug #867662 ("nullptr" warning)
  * documentation improvements

version 2.0.2
  * Fixed bug #3401971 (node-by-reference on dynamic collections)
  * documentation improvements
  * installer improvements
  * Fixed bug #3403291 (build for Fedora 15)
  * Don't install the .so for test modules
  * Fix for running ctest on Windows

version 2.0.1

  * Custom Full-text stemming.
  * Custom Full-text tokenization.
  * Fixed scripting bug where apply statement was considered a sequential expr even if the source expr was non-updating and non-sequential.
  * Fixed bug #3394400 (Element replacement, really unexpected result)
  * Fixed bug #3394588 (Unexpected error during pul undo)
  * Fixed bugs #3398511 and #3398552 with "make install"
  * Fixed bug #3398521 for compiling with ZORBA_NO_XMLSCHEMA

version 2.0
  * Zorba 2.0 is a major rewrite of previous versions. It doesn't claim to
    preserve backwards compatibility with any 1.X version.
    Host language API changes have been made to the following components:
    - External functions API
    - XmlDataManager
    - Error reporting
    - Debugger API
    XQuery language and API changes have been made to the following components:
    - XQuery Scripting Extension
    - Function APIs
    - XQuery Data Definition Facility
    - XQuery modules and module versioning
    - Changes to conform with the latest XQuery 3.0
    - Namespaces of annotations, errors, collations.
  * In addition to host language API and XQuery language improvements,
    plenty of bug fixes and optimizations have been made.

version 1.4
  * added a datetime library module with nondeterministic functions
  * new nondeterministic read functions in the http-client module
  * the http-client's send-request function accepts a non-validated request parameter
  * fixed an issue with a wrong static context being used in eval queries (see bug item #2979317)
  * added XQuery 1.1 Switch expressions (only available in 1.1 processing mode)
  * added the fn:parse function from XQuery Functions and Operators 1.1
  * improved error reporting for parser errors
  * improved error reporting for errors regarding sequential expressions
  * bug fixes
    - accept relative URIs in xs:anyURI constructors (#2988209)
    - direct element/attribute construction (#2999765)
    - deep-equal (#2999772)
    - schema-attribute(*) is parsed as a function call instead of a node test (#2993286)
    - handling xml:base attributes during doc loading (#3013096)
    - fixed build with CMake 2.6.0 on Debian Lenny
    - try-catch error if XQuery version < 1.1 (#3005344)
    - W3C test suite bugs CastableAs649 and CastableAs650 (#2999777)
    - assertion in StatelessExtFunctionCallIterator (#3012651)
    - http-client return type handling for (*+xml and text/html)
  * optimization of fn:subsequence
  * new xqDoc function (xqdoc-content) taking a documented module as string
  * new xqDoc feature; reporting functions invoked from within a particular function
  * xqDoc schema change; name element for functions got an arity attribute
  * added a tidy library module that provides functions to tidy an HTML string
    and returns a valid XHTML document node (http://www.zorba-xquery.com/modules/tidy)
  * added a schema library module that provides XML schema related functions (e.g. schema-type)
    (http://www.zorba-xquery.com/modules/schema)
  * added a node reference library module that provides functions to retrieve the URI
    of a node and vice versa (http://www.zorba-xquery.com/modules/reference)
  * added a hmac module (http://www.zorba-xquery.com/modules/security/hmac)
    that provides an sha1 function (e.g. used in OAuth)
  * added a random module (http://www.zorba-xquery.com/modules/random)
    that provides fuctions for generating (pseudo-)random integers and UUIDs.

version 1.2
  * more introspection functions (e.g. in-scope-element and in-scope-attribute)
  * first prototype for higher order functions from XQuery 1.1
  * JSON module in (com/zorba-xquery/www/modules/json)
  * huge performance improvements
  * added an Atom schema to the build-in schemas
  * register a callback function in the static context that is called
    if a collection or index declaration is processed (compiled)
  * nondeterministic (instead of sequential) functions in the file module
  * reference counting the store (de)initialization
  * many bug fixes; for example
    - compile with Xerces 3.1
    - loading external functions (prefix problem)
    - plan serializer fixes
    - serialization fixes
    - base64 decoder
    - loading documents with general entities
    - EXPATH http-client fixes
    - not hoisting nondeterministic functions
    - scripting related fixes
    - unicode related fixes (e.g. for parameters to the doc function)
    - StaticContext::loadProlog fix
    - SAX serialization fix

version 1.0
  * Changes in the C++ API
    * Removed ResultIterator class. All public API methods use simply
      Iterator.
    * Added a new class named ExternalModule.
      This class must be used for registering external functions.
      This change makes the external function API incompatible with
      previous releases.
    * Loading pre-compiled queries (using loadExecutionPlan) requires to pass
      a SerializationCallback which is used for
      retrieving registered uri resolvers or trace streams.
      Moreover, the callback is used for retrieving ExternalModules
      which have been used when compiling the query.
    * Changed the append function in zorba::String to const. Now, the
      function does not change the String itself, but returns a new
      String. Furgthermore zorba::String::append can now handle other
      zorba::String objects - not only const char*. Also the operators
      + and += are defined on zorba::String.
  * Other language bindings (Java, PHP, python, ruby) also now use
    simply Iterator; ResultIterator removed.
  * XQC (C language binding) brought up-to-date with latest revision
    of that spec (https://xqc.sourceforge.net).
  * Resolving of modules is now enhanced by the so called "Zorba
    Module Path".  Moreover, multiple ModuleURIResolvers can be
    registered in one StaticContext.  See the documentation for more
    information.
  * Added static and dynamic context introspection modules.
  * Added the excel library to the built-in modules.
    Implemented the most used excel functions.
  * Added the math library to the built-in modules (the namespace has
    changed to http://www.zorba-xquery.com/modules/math)
  * Pass the Static- and DynamicContext as parameter to the evaluate function
    of external functions.
  * The XQuery class got a setTimeout option that allows the user to set a 
    maximum running time of a particular query execution.
  * The StatelessExternalFunction is now non-pure by default. There is
    a new subclass called PureStatelessExternalFunction which should
    be used instead. PureStatelessExternalFunctions don't have access
    to the Static- and DynamicContext.
  * Renamed XQuery::serialize to XQuery::execute. The new method also
    implements the functionality of the XQuery::applyUpdates method
    (which was removed).
  * Added the serialize library to the built-in modules. The function
    util:serialize-to-string has been removed.
  * The DynamicContext can be used to pass arbitrary name-value pairs
    to the evaluate method of external functions.
  * Implemented CSV to XML converters. See csv2xml.xq module.
    You can convert back and forth between CSV files and XML
    or formated text files and XML.
  * Changes in the email functionnality
    * implemented smtp:send functions as external functions
    * added support for MIME messages (Text in character sets other
      than ASCII, Non-text attachments, Message bodies with multiple
      parts, Header information in non-ASCII character sets)
  * most of the runtime iterators and functions are generated
    (see https://sourceforge.net/apps/mediawiki/zorba/index.php?title=Creating_New_Runtime_Iterators_and_Functions)
  * improved exception messages raised by the serializer
  * Json is now able to parse Unicode messages
  * improved functionality for java, php, python and ruby bindings.

version 0.9.8-rev1 (2009.09.22)
  * bug fixes
    * fixed a crash caused by converting long integers (> 1024 digits)
      to string
    * the compile-only check reports the URI of the encapsulating
      entity in it's error report (used for XQDT semantic checking)
    * fixed 'for $x in () return $x/test' to return the empty sequence
      instead of raising a static error
    * fixed build using Xerces-C 3.0.1
    * fixed build for Mac OS X 10.6 (Snow Leopard)
    * improved error reporting in the command line utility
    * improved detection of the installed version of Xerces-C (raise
      an error if a wrong version is installed)

version 0.9.8 (2009.08.26)
  * More XQuery Scripting Features
    * default initialization for external variables
    * block local variables
  * XQuery 1.1
    * Support for the "validation as TypeName" feature
    * Changed the try/catch syntax as specified in XQuery 1.1 (except
      the try- and catch expressions are ExprSingle and not Expr).
  * Examples for the Java API
  * Function library
  * creating PDFs using XSL Formatting Objects (requires Java)
  * util:parse function
  * converter functions to/from JSON or JsonML and XML
  * function for sending simple text emails
  * the util:tidy function returns a node instead of a string.
  * XQDoc module http://www.zorba-xquery.com/zorba/xqdoc-functions
  * Added a loadProlog function to the static context.
  * Added support for XQueryX and XQueryX Update
  * Added compiled query persistence to the C++ API
    (XQuery::loadExecutionPlan() and XQuery::saveExecutionPlan())
  * Added a printPlan function to the XQuery class of the API.
  * Added new option in the Zorba command line utility: compile-only,
    parse-only, lib-module, print-errors-as-xml.
  * Added two plan printing function (printPlanAsXML and
    printPlanAsDOT) to the SWIG generated APIs.
  * Implemented 'declare option': options are set in the static context.
  * fn:trace can be disabled by setting the option
    QName("http://www.zorba-xquery.org/option", "trace") to "disable".
  * Changed the scripting syntax of the exit expression from 'exit
    with' to 'exit returning'.This is due to an ambiguity in the
    grammar (see http://www.w3.org/Bugs/Public/show_bug.cgi?id=6852
    for more information).
  * Added accessor functions for node items to the Item class in the
    API (i.e. getChildren, getAttributes, and getNodeName).
  * Added a UserException to the API which is raised if the user makes
    a call to the fn:error function.
  * UTF8 support for the lexer
  * bug fixes and performance improvements

version 0.9.5 (2008.02.17)
  * Unified the Tidy and Rest API's
  * Added new sinatures to zorba-util:tidy and zorba-util:tdoc that
    allow setting of tidy functions
  * Added Schema imports using target namespace or location hints
  * Added StaticContext::findFunctions and StaticContext::disableFunciton
  * Added Functions for collections 
  * Windows packages include a zorba_simplestore.dll
  * Improved error messages for regular expression processing
  * Added functions to return an item instead of xs:string
  * Added a new signature to zorba-util:random in order to be able to call
    random within a loop and get different numbers each time.
  * Corrected the types of some functions/operators (YM_DURATION or 
    DT_DURATION instead of DURATION).
  * Now possible to pass empty sequence as collection URI.
  * Now cURL is set to follow an infinite number of redirects.
  * Added validate after update
  * Fixed schema files without target namespace.
  * Fixed notation cast.
  * Added list and union item types to type manager cache
  * Added validation tests for schema substitution groups
  * Added message when error during update test driver.
  * Added partial support for scripting (XQSE)
  * Added support "default element namespace" schema imports
  * Added support variable initializers that refer to forward variable
    declarations.
  * Various bug fixes and performance enhancements.

version 0.9.4 (2008.10.06)
  * added General FLWOR (GFLWOR)
    * added Windowing functionality
    * added ability to perform "outer" joins in the FLWOR with outer
      for clauses
    * output numbering added to FLWOR
  * Added HTTP support to module import
  * fn:parse() is implemented as per the XQuery proposal
  * Extended api to register user-defined URI resolvers for documents,
    collections, schemas, and modules
  * boost dependencies removed
  * Added xquery debugging support
  * Added a store implementation based on Windows DOM.
  * Added functions to make calls to REST based web services through XQuery
  * Added PHP bindings to zorba
  * Added tidy support to convert web pages into xhtml.
  * Added XML Schema support for validation
  * Added the ability to revalidate XML data with respect to a Schema
    after applying updates

version 0.9.21-4303 (2008.06.26):
  * fixed installation problem for python and ruby language bindings
  * fixed some valgrind reported memory problems
  * other minor bug fixes and performance improvements

version 0.9.2-4251 (2008.06.19):
  * Module support (as in the XQuery 1.0 W3C recommendation)
  * Removed boost shared pointers from the API
  * Performance improvements and bug fixes
  * C API (zorbac.h)
  * Rudimentary SWIG interface for generating other language bindings
    (i.e. Ruby and Python)
  * Added support for running multiple queries in the command line utility
  * group by (as in the evolving W3C XQuery 1.1 recommendation)
  * try-catch (as in the evolving W3C recommendation)
  * eval facility to dynamically evaluate a string and execute it as a query
  * some math functions (e.g. sqrt, sin, cos, etc)
  * Basic schema validation (without post schema validation and
    substitution groups)
  * Stabilized the Store API & build the simplestore as a separate library

version 0.9.1 (2008.04.30):
  * Initial release supporting 
    * XQuery 1.0 and XPath 2.0 (as in the W3C recommendation) 
      except modules and only limited support for schema validation
    * XQuery Update Facility 1.0 (as in the evolving W3C recommendation)
    * An In-Memory (replaceable) XML store 
      (implements the XQuery Data Model (XDM) W3C recommendation)
  * This release 0.9 passes 97.6% of the W3C tests of XQuery 1.0 test suite
    and 99.4% of the XQuery Update Facility test suite.