# Add a method to the tokenizer module NQXML

    class Tokenizer
     def testOnlySetInternalEntities(h); @internalEntites = h; end
    end

end

Tests simple predefined entity replacement

Tests the entity replacement routine.

Makes sure special character encoding is working

Make sure names are read correctly.

Make sure the tokenizer can handle here documents.

Creates a tokenizer over the xml and sets it to tokenizing. Expect a parser error that matches regex. If line, col, or pos is not nil, expect the error to occur in the specified position(s).

Make sure we detect missing '?' at the end of processing instructions.

Disallow two slashes in one tag.

Well-formedness check: the same attribute name may not appear twice in the same tag.

Make sure the tokenizer rejects or accepts the proper inputs.

Make sure tokenizer rejects illegal public identifier

Make sure entities are tokenized correctly.

Make sure entities are tokenized correctly. The example XML is taken almost verbatim from the XML specification.

Passes a Tempfile to a Tokenizer. Fails if any parser error is raised.

Tokenizes @rsrc.xml and compares the entities returned with a list of expected values.

This XML fragment made the tokenizer barf because the < was being translated and then seen as a tag open character.