Metadata-Version: 1.1
Name: pascal_tokenizer
Version: 1.0.4
Summary: UNKNOWN
Home-page: UNKNOWN
Author: Artem Gavrilov
Author-email: info@teamfnd.ru
License: UNKNOWN
Description: PyPascalTokenizer
        =================
        
        Module for Python 3. It does tokenizing of Pascal code (it's planned to support full Delphi and FreePascal syntax).
        
        API
        ===
        
        Token struct
        ------------
        
        Token is saved as 4-tuple (text, begin, end, final):
        
        * text: string with token text
        * begin: position of token start, tuple (y, x), where y - 0-based line index and x - 0-based character index in line
        * end: position after token end, tuple (y, x)
        * final: bool, True if it was last token
        
        Class PasTokenizer
        ------------------
        
        * __init__(lines): params: list of source code strings
        * get_next(): get next token and change end pos
        * read_next(): get next token, but don't change end pos
        * is_ended(): check if text ended
        
        Class PasTokenizerStack
        -----------------------
        
        * __init__(lines, comments=True): params: list of source code strings; "comments" allows to get also comment-tokens (otherwise tokenizer skips them)
        * push(s): push token to stack
        * pop(): pop (get and delete) token from stack
        * read_last(): read (get but don't delete) top token from stack
        * is_ended(): check if stack ended
        
        Class PasTokenizerParallelStack
        -------------------------------
        
        Descendant of PasTokenizerStack, which uses thread(s) for parsing entire file. Before destroying it you mast call stop().
        
        * __init__(lines, comments=True, qlong=1000): additonal param: size of internal queue buffer
        
        Utils
        -----
        
        Helper functions to analyze token text.
        
        * is_name(s): Check for valid identifier (can be reserved word too).
        * is_comment(s): Check for valid comment.
        * is_string(s): Check for string constant.
        
        
        Author
        =====
        Artem Gavrilov (@Artem3213212 at Github)
        
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Libraries :: Python Modules
