Description: python sed
Keywords: sed, python, regexp

# sed.py

## A full implementation of sed in Python

### Contents

General Information

Usage as a command line utility

Usage as a Python module

Sed dialect

Testing

### General Information

#### Description

`sed.py` is a full and working Python implementation of sed. Its reference is GNU sed 4.2 of which it implements all commands and features except the most specific ones. It may be used as a command line utility or it can be used as a module to bring sed functionnality to Python scripts.

A complete set of tests is available as well as a testing utility. These tests include scripts from various origins and cover all aspects of sed functionalities.

#### Platform

`sed.py` is a Python script (Python 2.7). As such, it can run on any platform supporting Python. It can be used as a command line utility or as a module.

#### Licence

`sed.py` is licensed under the MIT licence.

#### Links and contact

- Home page
- Download script
- Download script and tests
- Repository
- Mail to: gilles dot arcas at gmail dot com

### Usage as a command line utility

`sed.py` may be used as console program receiving information from the command line. The format of the command line is:

    sed.py [options] -e<script expression> <input text file>
    sed.py [options] -f<script file> <input text file>

`options` may be one or both of:

`-n` | disable automatic printing
`-r` | use extended regular expressions

### Usage as a Python module

An example covering all necessary symbols:

    from sed import Sed, SedException

    sed = Sed()
    try:
        sed.no_autoprint = True
        sed.regexp_extended = False
        sed.load_script('myscript.sed')
        sed.apply('myinput.txt')
    except SedException as e:
        print e.message
    except:
        raise

Note that `sed.py` prints the lines by default and returns the list of lines printed by the script. `sed.py` has an output parameter which enables to inhibit printing the lines (`output=None`) or to redirect the output to some text file (`output=somefile.txt`).

The script may also be read from a string by using `sed.load_string(my_script_string)`.

### sed dialect

`sed.py` implements all standard commands and regular expression features of sed. Its reference is GNU sed 4.2. It implements almost all its features except the most specific ones.

GNU sed is described in the manual page. This page can serve as a reference for `sed.py` given the differences described in the following.

#### Addresses

`number` | standard behavior
`$` | standard behavior
`/regexp/` | standard behavior
`/regexp/I` | implemented
`\%regexp%` | standard behavior
`address,address` | standard behavior
`address!` | standard behavior
`0,/regexp/` | not implemented
`first~step` | not implemented
`addr1,+N` | not implemented
`addr1,~N` | not implemented

#### Regular expressions

`char` | standard behavior
`*` | standard behavior
`\+` | standard behavior
`\?` | standard behavior
`\{i\} \{i,j} \{i,\}` | standard behavior
`\(regexp\)` | standard behavior
`.` | standard behavior
`^` | standard behavior. When not at start of regexp, matches as itself
`$` | standard behavior. When not at end of regexp, matches as itself
`[list] [^list]` | standard behavior. [.ch.], [=a=], [:space:] are not implemented
`regexp1\|regexp2` | standard behavior
`regexp1regexp2` | standard behavior
`\digit` | standard behavior (back reference)
`\n \t` | standard behavior
`\char` | standard behavior (disable special regexp characters)

Note that for any combination of quantifiers (*, +, ?, {}), consecutive quantifiers or a quantifier starting a regexp will launch an error. This is true in basic or extended regular expression modes.

#### Extended regular expressions

Using the -r switch enables to simplify regular expressions by removing the antislah character before the special characters +, ?, (, ), |, { and }. If these characters must appear as regular characters in a regexp, they must be slashed.

#### Commands

`a\ text` | Compliant with GNU sed description including single line and double address extensions
`b label` | Compliant with GNU sed description
`: label` | Compliant with GNU sed description
`c\ text` | Compliant with GNU sed description including single line and double address extensions
`d` | Compliant with GNU sed description
`D` | Compliant with GNU sed description
`=` | Compliant with GNU sed description (including double address extension)
`g` | Compliant with GNU sed description
`G` | Compliant with GNU sed description
`h` | Compliant with GNU sed description
`H` | Compliant with GNU sed description
`i\ text` | Compliant with GNU sed description (including single line and double address extensions)
`l` | Compliant with GNU sed description (length parameter not implemented)
`n` | Compliant with GNU sed description
`N` | Compliant with GNU sed description
`p` | Compliant with GNU sed description
`P` | Compliant with GNU sed description
`q` | Compliant with GNU sed description (except exit code extension)
`r filename` | Compliant with GNU sed description (including double address extension but not reading from stdin)
`s` | Compliant with GNU sed description except escape sequences in replacement (\L, \l, \U, \u, \E) and modifiers e and M/m.
`t label` | Compliant with GNU sed description
`w filename` | Compliant with GNU sed description (including double address extension but not writing to stdout or stderr)
`x` | Compliant with GNU sed description
`y` | Compliant with GNU sed description
`#` | Compliant with GNU sed description. (Comments start anywhere in the line.) It is possible to extend the #n convention with a r character to specify extended regular expressions (#r or #nr)

The other commands specific to GNU sed are not implemented.

### Testing

#### Description

The working of `sed.py` is tested and compared to the behavior of GNU sed with a set of tests and a testing utility.

The tests are either coded in a Python module or may be stored in test directories as standard sed scripts.

The test suites are:

`unit-suite.py` | a Python module containing unitary tests
`chang-suite.py` | a Python module containing scripts from Roger Chang web site
`test-suite1` | a set of scripts from GNU sed test suite
`test-suite2` | a set of scripts from the seder's grab-bag and Rosetta code web site
`test-suite3` | additional unitary tests better stored in a folder with some extra data text files

#### Testing utility

Tests are launched and checked with the `test-suite.py` Python script. This script uses either `sed.py` to run the script, or any sed executable. This enables to compare the working of `sed.py` with the one of GNU sed.

The calling syntax is a follow:

    test-suite.py <testsuite> [number] [-b executable] [-x list of script references]

`testsuite` | either a Python module or a test directory
`number` | an optional reference number of a test, when present only this tests is run
`executable` | an optional name or path of a sed executable to use for testing
`list of script references` | an optional list of tests to exclude for instance when a feature is not implemented. A script reference is either the title of the test for tests stored in modules, or the the name of the script file.

#### Module test suites

When tests are stored in a Python script, they are coded as lists of four elements:

- the title of the test
- the script itself
- the input list of lines
- the expected result

The input and output lines may be replaced by an empty string. In that case, the data from the previous test are used.

The expected result may be None when the test has no result and ends with an error.

#### Directory test suites

When tests are stored in a directory, they are represented by three or four files with same name but different extensions:

- the script itself, with '.sed' extension
- the input of the script, with '.inp' extension
- the expected result of the script, with '.good' extension
- possibly a file, with '.flags' extension, containing the sed switches -n and/or -r.

Some other files may be used when using reading or writing commands in scripts. In that case, the expected written files must be named with extension '.wgoodN' where N is the number of the expected written file.