Metadata-Version: 1.1
Name: pyin
Version: 0.4.4
Summary: basic streaming text processing
Home-page: https://github.com/geowurster/pyin
Author: Kevin Wurster
Author-email: wursterk@gmail.com
License: New BSD License

Copyright (c) 2015, Kevin D. Wurster
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

* The names of its contributors may not be used to endorse or promote products
  derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Description: ====
        pyin
        ====
        
        It's like sed, but Python!
        
        This project is actively being developed but don't get too attached to any
        portions of the API or commandline syntax.  It's a pretty useful utility but
        should not be incorporated into any scripts or projects until ``v1.0``.
        
        .. image:: https://travis-ci.org/geowurster/pyin.svg?branch=master
            :target: https://travis-ci.org/geowurster/pyin
        
        .. image:: https://coveralls.io/repos/geowurster/pyin/badge.svg?branch=master
            :target: https://coveralls.io/r/geowurster/pyin?branch=master
        
        
        Examples
        ========
        
        See the `Cookbook <https://github.com/geowurster/pyin/blob/master/Cookbook.rst>`__ for more examples.
        
        Change newline character in a CSV.
        
        .. code-block:: console
        
            $ more sample-data/csv-with-header.csv | pyin "line.replace('\n', '\r\n')" > output.csv
        
        Extract a BigQuery schema from an existing table and pretty print it:
        
        .. code-block:: console
            $ bq show --format=json ${DATASET}.${TABLE} | pyin -m json -m pprint "pprint.pformat(json.loads(line)['schema']['fields'])"
            [{u'mode': u'NULLABLE', u'name': u'mmsi', u'type': u'STRING'},
            {u'mode': u'NULLABLE', u'name': u'longitude', u'type': u'FLOAT'},
            {u'mode': u'NULLABLE', u'name': u'latitude', u'type': u'FLOAT'}
            ...]
        
        Read the first 100K lines of a CSV and write only the lines where column
        'Msg type' is equal to 5.
        
        .. code-block:: console
        
            $ pyin -i ${INFILE} -o ${OUTFILE} \
                --true \
                --lines 100000 \
                --reader csv.DictReader \
                --import csv \
                --import newlinejson \
                --writer newlinejson.Writer
                "line['Msg type'] == '5'"
        
        
        Installing
        ==========
        
        Via pip:
        
            $ pip install git+https://github.com/geowurster/pyin.git
        
        From master branch:
        
            $ git clone https://github.com/geowurster/pyin
            $ python setup.py install
        
        
        Gotchas
        =======
        
        It's easy to completely modify the line content:
        
        .. code-block:: console
        
            $ pyin -i sample-data/csv-with-header.csv "'operation'"
            operationoperationoperationoperationoperationoperation
        
        Forgetting to use ``-t`` to only get lines that evaluate as ``True``:
        
        .. code-block:: console
        
            $ pyin -i LICENSE.txt "'are' in line"
            FalseFalseFalseFalseFalseFalseTrueFalseFalseFalseFalseFalseFalseFalseFalseFalseTrueFalseFalseFalseFalseFalseFalseFalseFalseFalseFalseFalse
            
            $ pyin -i LICENSE.txt "'are' in line" -t
            modification, are permitted provided that the following conditions are met:
              derived from this software without specific prior written permission.
        
        The ``--reader-option key=val`` values are parsed to their Python type but if the user wants to
        specify something like which JSON library to use for a ``newlinejson.Reader()``
        instance then they must do that via the ``--statement`` option:
        
        .. code-block:: console
        
            $ pyin -i ${INFILE} -o ${OUTFILE}
                --true
                --import newlinejson \
                --import ujson
                --reader newlinejson.Reader \
                --writer newlinejson.Writer \
                --statement "newlinejson.JSON = ujson" \
                "'type' in line and line['type'] is 5"
        
        
        Developing
        ==========
        
        Install:
        
        .. code-block:: console
        
            $ git clone https://github.com/geowurster/pyin
            $ cd pyin
            $ virtualenv venv
            $ source venv/bin/activate
            $ pip install -r requirements-dev.txt
            $ pip install -e .
            $ nosetests --with-coverage
            $ pep8 --max-line-length=120 pyin.py
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Topic :: Text Processing :: Filters
Classifier: Topic :: Text Processing :: General
Classifier: Topic :: Utilities
