Author: David Goodger; open to all Docutils developers
Date: 2020-02-06

# Docutils Project Policies

Author: | David Goodger; open to all Docutils developers
Contact: | docutils-develop@lists.sourceforge.net
Date: | 2020-02-06
Revision: | 8489
Copyright: | This document has been placed in the public domain.

Contents

- How to get a new feature into Docutils
- Python Coding Conventions
- Documentation Conventions
- Copyrights and Licensing
  - 2-Clause BSD license
- Repository
  - Branches
    - Review Criteria
  - Check-ins
- Version Identification
  - `docutils.__version_info__`
  - `docutils.__version__`
  - Policy History
- Snapshots
- Setting Up For Docutils Development
- Mailing Lists
- The Wiki
- Extensions and Related Projects
  - The Sandbox
  - Parallel Projects
  - Other related projects

The Docutils project group is a meritocracy based on code contribution and lots of discussion [3]. A few quotes sum up the policies of the Docutils project. The IETF's classic credo (by MIT professor Dave Clark) is an ideal we can aspire to:

> We reject: kings, presidents, and voting. We believe in: rough consensus and running code.

As architect, chief cook and bottle-washer, David Goodger currently functions as BDFN (Benevolent Dictator For Now). (But he would happily abdicate the throne given a suitable candidate. Any takers?)

Eric S. Raymond, anthropologist of the hacker subculture, writes in his essay The Magic Cauldron:

> The number of contributors [to] projects is strongly and inversely correlated with the number of hoops each project makes a user go through to contribute.

We will endeavour to keep the barrier to entry as low as possible. The policies below should not be thought of as barriers, but merely as a codification of experience to date. These are "best practices"; guidelines, not absolutes. Exceptions are expected, tolerated, and used as a source of improvement. Feedback and criticism is welcome.

As for control issues, Emmett Plant (CEO of the Xiph.org Foundation, originators of Ogg Vorbis) put it well when he said:

> Open source dictates that you lose a certain amount of control over your codebase, and that's okay with us.

[3] | Phrase borrowed from Ben Collins-Sussman of the Subversion project.

# How to get a new feature into Docutils

Question:
  I would very much like to have this new feature in the Docutils core. What exactly do I have to do to make this possible?

- Present your idea at the Docutils-develop mailing list,

  `+1` | usually triggers a fast response,
  `-1` | may be forgotten later,

  and/or file a ticket at Docutils' feature request tracker.

  `+1` | there is a permanent record,
  `-1` | it may stay forgotten among all the other feature requests.

- Convince a Docutils developer that this is a valuable addition worth the effort.

- Contribute. The developers will make sure that the contribution fits nicely into Docutils (cf. the review criteria). This might involve discussing (and compromising on) design and implementation details. It might also lead to the conclusion that the addition fits better in the extensions and related projects.

- Be patient, and be persistent. None of us are paid to do this, it's all in our spare time -- which is precious and rare.

How to make code contributions that are easily accepted:

- Have a look at the Python coding conventions and documentation conventions below.

- Prepare test cases. This vastly helps when integrating new code. Test cases are also examples and showcases for new features. See Docutils Testing for a description of the test suite in `docutils/test/`.

  Ensure the addition works with all supported Python versions.

- Look at the Docutils sources to see how similar features are implemented, learn to do it "the Docutils way".

- Prepare the a patch or an addition to the existing documentation. Include links.

- If you are familiar with version control, consider creating a feature branch in a Docutils repository checkout. (Working with branches is much easier with git. You can get a git clone of the repository from http://repo.or.cz/w/docutils.git or with git-svn.)

# Python Coding Conventions

Contributed code will not be refused merely because it does not strictly adhere to these conditions; as long as it's internally consistent, clean, and correct, it probably will be accepted. But don't be surprised if the "offending" code gets fiddled over time to conform to these conventions.

The Docutils project shall follow the generic coding conventions as specified in the Style Guide for Python Code and Docstring Conventions PEPs, summarized, clarified, and extended as follows:

- 4 spaces per indentation level. No hard tabs.
- Use only 7-bit ASCII, no 8-bit strings. See Docutils Internationalization.
- No one-liner compound statements (i.e., no `if x: return`: use two lines & indentation), except for degenerate class or method definitions (i.e., `class X: pass` is OK.).
- Lines should be no more than 78 characters long.
- Use "StudlyCaps" for class names (except for element classes in docutils.nodes).
- Use "lowercase" or "lowercase_with_underscores" for function, method, and variable names. For short names, maximum two words, joined lowercase may be used (e.g. "tagname"). For long names with three or more words, or where it's hard to parse the split between two words, use lowercase_with_underscores (e.g., "note_explicit_target", "explicit_target"). If in doubt, use underscores.
- Avoid lambda expressions, which are inherently difficult to understand. Named functions are preferable and superior: they're faster (no run-time compilation), and well-chosen names serve to document and aid understanding.
- Avoid functional constructs (filter, map, etc.). Use list comprehensions instead.
- Avoid `from __future__ import` constructs. They are inappropriate for production code.
- Use 'single quotes' for string literals, and """triple double quotes""" for docstrings.

# Documentation Conventions

- Docutils documentation is written using reStructuredText, of course.

- Use 7-bit ASCII if at all possible, and Unicode substitutions when necessary.

- Use the following section title adornment styles:

      ================
       Document Title
      ================

      --------------------------------------------
       Document Subtitle, or Major Division Title
      --------------------------------------------

      Section
      =======

      Subsection
      ----------

      Sub-Subsection
      ``````````````

      Sub-Sub-Subsection
      ..................

- Use two blank lines before each section/subsection/etc. title. One blank line is sufficient between immediately adjacent titles.

- Add a bibliographic field list immediately after the document title/subtitle. See the beginning of this document for an example.

- Add an Emacs "local variables" block in a comment at the end of the document. See the end of this document for an example.

# Copyrights and Licensing

The majority of the Docutils project code and documentation has been placed in the public domain. Unless clearly and explicitly indicated otherwise, any patches (modifications to existing files) submitted to the project for inclusion (via Subversion, SourceForge trackers, mailing lists, or private email) are assumed to be in the public domain as well.

Any new files contributed to the project should clearly state their intentions regarding copyright, in one of the following ways:

- Public domain (preferred): include the statement "This module/document has been placed in the public domain."

- Copyright & open source license: include a copyright notice, along with either an embedded license statement, a reference to an accompanying license file, or a license URL.

  The license should be well known, simple and compatible with other open source software licenses. To keep the number of different licenses at a minimum, using the 2-Clause BSD license is recommended.

## 2-Clause BSD license

(also known as "Simplified" or FreeBSD license)

> If you want a simple, permissive non-copyleft free software license, the FreeBSD license is a reasonable choice. However, please don't call it a “BSD” or “BSD-style” license, because that is likely to cause confusion which could lead to use of the flawed original BSD license.

> —Various Licenses and Comments about Them

Pros:
  - clear wording, structured text
  - license used by the closely related Sphinx project
  - "2-Clause BSD license" is a non-ambiguous name, used by both, OSI and GNU.
References:
  - http://www.freebsd.org/copyright/freebsd-license.html
  - http://www.opensource.org/licenses/BSD-2-Clause
  - http://www.spdx.org/licenses/BSD-2-Clause

# Repository

Please see the repository documentation for details on how to access Docutils' Subversion repository. Anyone can access the repository anonymously. Only project developers can make changes. (If you would like to become a project developer, just ask!) Also see Setting Up For Docutils Development below for some useful info.

Unless you really really know what you're doing, please do not use `svn import`. It's quite easy to mess up the repository with an import.

## Branches

(These branch policies go into effect with Docutils 0.4.)

The "docutils" directory of the trunk (a.k.a. the Docutils core) is used for active -- but stable, fully tested, and reviewed -- development.

If we need to cut a bugfix release, we'll create a maintenance branch based on the latest feature release. For example, when Docutils 0.5 is released, this would be `branches/docutils-0.5`, and any existing 0.4.x maintenance branches may be retired. Maintenance branches will receive bug fixes only; no new features will be allowed here.

Obvious and uncontroversial bug fixes with tests can be checked in directly to the core and to the maintenance branches. Don't forget to add test cases! Many (but not all) bug fixes will be applicable both to the core and to the maintenance branches; these should be applied to both. No patches or dedicated branches are required for bug fixes, but they may be used. It is up to the discretion of project developers to decide which mechanism to use for each case.

Feature additions and API changes will be done in feature branches. Feature branches will not be managed in any way. Frequent small checkins are encouraged here. Feature branches must be discussed on the docutils-develop mailing list and reviewed before being merged into the core.

### Review Criteria

Before a new feature, an API change, or a complex, disruptive, or controversial bug fix can be checked in to the core or into a maintenance branch, it must undergo review. These are the criteria:

- The branch must be complete, and include full documentation and tests.
- There should ideally be one branch merge commit per feature or change. In other words, each branch merge should represent a coherent change set.
- The code must be stable and uncontroversial. Moving targets and features under debate are not ready to be merged.
- The code must work. The test suite must complete with no failures. See Docutils Testing.

The review process will ensure that at least one other set of eyeballs & brains sees the code before it enters the core. In addition to the above, the general Check-ins policy (below) also applies.

## Check-ins

Changes or additions to the Docutils core and maintenance branches carry a commitment to the Docutils user community. Developers must be prepared to fix and maintain any code they have committed.

The Docutils core (`trunk/docutils` directory) and maintenance branches should always be kept in a stable state (usable and as problem-free as possible). All changes to the Docutils core or maintenance branches must be in good shape, usable, documented, tested, and reasonably complete.

- Good shape means that the code is clean, readable, and free of junk code (unused legacy code; by analogy to "junk DNA").
- Usable means that the code does what it claims to do. An "XYZ Writer" should produce reasonable XYZ output.
- Documented: The more complete the documentation the better. Modules & files must be at least minimally documented internally. Docutils Front-End Tools should have a new section for any front-end tool that is added. Docutils Configuration Files should be modified with any settings/options defined. For any non-trivial change, the HISTORY.txt file should be updated.
- Tested means that unit and/or functional tests, that catch all bugs fixed and/or cover all new functionality, have been added to the test suite. These tests must be checked by running the test suite under all supported Python versions, and the entire test suite must pass. See Docutils Testing.
- Reasonably complete means that the code must handle all input. Here "handle" means that no input can cause the code to fail (cause an exception, or silently and incorrectly produce nothing). "Reasonably complete" does not mean "finished" (no work left to be done). For example, a writer must handle every standard element from the Docutils document model; for unimplemented elements, it must at the very least warn that "Output for element X is not yet implemented in writer Y".

If you really want to check code directly into the Docutils core, you can, but you must ensure that it fulfills the above criteria first. People will start to use it and they will expect it to work! If there are any issues with your code, or if you only have time for gradual development, you should put it on a branch or in the sandbox first. It's easy to move code over to the Docutils core once it's complete.

It is the responsibility and obligation of all developers to keep the Docutils core and maintenance branches stable. If a commit is made to the core or maintenance branch which breaks any test, the solution is simply to revert the change. This is not vindictive; it's practical. We revert first, and discuss later.

Docutils will pursue an open and trusting policy for as long as possible, and deal with any aberrations if (and hopefully not when) they happen. We'd rather see a torrent of loose contributions than just a trickle of perfect-as-they-stand changes. The occasional mistake is easy to fix. That's what version control is for!

# Version Identification

The state of development of the current Docutils codebase is stored in two forms: a docutils.__version_info__ namedtuple, and a docutils.__version__ text string.

See also the Docutils Release Procedure, and `docutils.__version__` & `docutils.__version_info__` in docutils/__init__.py.

## `docutils.__version_info__`

Detailed version information is available in `docutils.__version_info__`, an instance of the namedtuple `docutils.VersionInfo`. It is modelled on sys.version_info and has the following attributes:

major : non-negative integer
  Major releases (x.0, e.g. 1.0) will be rare, and will represent major changes in API, functionality, or commitment. The major number will be bumped to 1 when the project is feature-complete, and may be incremented later if there is a major change in the design or API. When Docutils reaches version 1.0, the major APIs will be considered frozen and backward compatibility will become of paramount importance.
minor : non-negative integer
  Releases that change the minor number (x.y, e.g. 0.5) will be feature releases; new features from the Docutils core will be included.
micro : non-negative integer
  Releases that change the micro number (x.y.z, e.g. 0.4.1) will be bug-fix releases. No new features will be introduced in these releases; only bug fixes will be included. The micro number is omitted from `docutils.__version__` when micro=0.
releaselevel : text string

  The release level indicates the development status (or phase) of the project's codebase:

  Release Level | Label | Description
  --- | --- | ---
  alpha | `a` | Reserved for use after major experimental changes, to indicate an unstable codebase.
  beta | `b` | Indicates active development, between releases. Used with serial = 0.
  candidate | `rcN` | Release candidate: indicates that the codebase is ready to release unless significant bugs emerge. Serial N starts at 1.
  final |  | Indicates an official project release. There is no pre-release segment for final releases (no label).

  The abbreviations in the "Label" column are used in the docutils.__version__ identifier text.

serial : non-negative integer
  The serial number is incremented whenever a new pre-release is begun.
release : boolean
  True for official releases and pre-releases, False during development.

One of {major, minor, micro} is incremented after each official release, and the lower-order numbers are reset to 0.

The default state of the repository during active development and between releases is: release level "beta", serial = 0, release = False.

`docutils.__version_info__` can be used to test for a minimally required version, e.g.:

    comparison_version = docutils.VersionInfo(
        major=0, minor=13, micro=0,
        releaselevel='candidate', serial=2, release=True)
    if docutils.__version_info__ >= comparison_version:
        ...

For practical purposes it may suffice to test against a truncated tuple, e.g.:

    if docutils.__version_info__ >= (0, 13)

is True for all versions "larger" than `"0.13"`.

Mind, however, that a test like

    if docutils.__version_info__ > (0, 14)

is True also for development versions and pre-releases of the 0.14 series although, according to PEP 440, these must be sorted before 0.14.

## `docutils.__version__`

`docutils.__version__` contains the version identifier as a text string: a concise, PEP 440-conforming representation of `docutils.__version_info__`.

For version comparison operations, use docutils.__version_info__. Do not parse the text of `docutils.__version__`.

`docutils.__version__` takes the following form:

    <major>.<minor>[.<micro>][<releaselevel>[<serial>]][.dev]
    <--- release segment ---><-- pre-release segment -><- development ->

- The abbreviated forms of each release level, found in the "Label" column in the table above ("a" or "b" or "rc"), are used in the pre-release segment.
- When the serial number is 0, it is omitted from the pre-release segment.
- The development segment is present during active development (as "`.dev`", when `docutils.__version_info__.release` is False), and absent for official releases and pre-releases (when `docutils.__version_info__.release` is True).

Examples of `docutils.__version__` identifiers, over the course of normal development (without branches), in ascending order:

Release Level | Version Identifier
--- | ---
final (release) | 0.14
beta (development) | 0.15b.dev [2]
beta (release) | 0.15b [1]
candidate 1 (dev.) | 0.15rc1.dev
candidate 1 (release) | 0.15rc1
candidate 2 (dev.) | 0.15rc2.dev [1]
candidate 2 (release) | 0.15rc2 [1]
... candidate N | 0.15rcN.dev [1], 0.15rcN [1]
final (release) | 0.15
beta (dev.) | 0.16b.dev [2]

[1] | (1, 2, 3, 4, 5) These steps may be skipped.

[2] | (1, 2) Default active development state between releases.

## Policy History

- Prior to version 0.4, Docutils didn't have an official version numbering policy, and micro releases contained both bug fixes and new features.
- An earlier version of this policy was adopted in October 2005, and took effect with Docutils version 0.4.
- This policy was updated in June 2017 for Docutils version 0.14. See Feature Request #50 and the discussion on docutils-devel from May 28 to June 20 2017.

# Snapshots

Snapshot tarballs can be downloaded from the repository (see the "download snapshot" button in the head of the code listing table).

- the Docutils core, representing the current cutting-edge state of development;
- the sandbox directory with contributed projects and extensions from the Sandbox;

- development branches, representing ongoing development efforts to bring new features into Docutils.

# Setting Up For Docutils Development

When making changes to the code, testing is a must. The code should be run to verify that it produces the expected results, and the entire test suite should be run too. The modified Docutils code has to be accessible to Python for the tests to have any meaning. There are several ways to keep the Docutils code accessible during development:

1. Update your `PYTHONPATH` environment variable so that Python picks up your local working copy of the code.

   We'll assume that the Docutils trunk is checked out under your `~/projects/` directory.

   For the bash shell, add this to your `~/.profile`:

       PYTHONPATH=$HOME/projects/docutils/docutils
       PYTHONPATH=$PYTHONPATH:$HOME/projects/docutils/docutils
       export PYTHONPATH

   The first line points to the directory containing the `docutils` package. The second line adds the directory containing the third-party modules Docutils depends on. The third line exports this environment variable.

   You many also want to add the `tools` directory to your binary `PATH`:

       PATH=$PATH:$HOME/projects/docutils/docutils/tools
       export PATH

2. Create a symlink to the docutils package directory somewhere in the module serach path (`sys.path`), e.g.,

       ln -s ~/projects/docutils/docutils \
            /usr/local/lib/python2.7/site-packages/

   Alternatively, use a path configuration file.

   You may also want to copy or link the front-end tools in `docutils/tools` to a suitable place in your binary PATH, e.g.,

       ln -s ~/projects/docutils/docutils/tools/rst2html.py \
             /usr/local/bin/rst2html

3. Do an editable install with pip.

4. Install in development mode with setuptools.

5. Before you run anything, every time you make a change, reinstall Docutils:

       python setup.py install

   Caution!

   This method is not recommended for day-to-day development; it's too easy to forget. Confusion inevitably ensues.

   If you install Docutils this way, Python will always pick up the last-installed copy of the code. If you ever forget to reinstall the "docutils" package, Python won't see your latest changes.

A useful addition to the `docutils` top-level directory in branches and alternate copies of the code is a `set-PATHS` file containing the following lines:

    # source this file
    export PYTHONPATH=$PWD:$PWD
    export PATH=$PWD/tools:$PATH

Open a shell for this branch, `cd` to the `docutils` top-level directory, and "source" this file. For example, using the bash shell:

    $ cd some-branch/docutils
    $ . set-PATHS

# Mailing Lists

Developers are recommended to subscribe to all Docutils mailing lists.

# The Wiki

There is a development wiki at http://docutils.python-hosting.com/ as a scratchpad for transient notes. Please use the repository for permament document storage.

# Extensions and Related Projects

## The Sandbox

The sandbox directory is a place to play around, to try out and share ideas. It's a part of the Subversion repository but it isn't distributed as part of Docutils releases. Feel free to check in code to the sandbox; that way people can try it out but you won't have to worry about it working 100% error-free, as is the goal of the Docutils core. A project-specific subdirectory should be created for each new project. Any developer who wants to play in the sandbox may do so, but project directories are recommended over personal directories, which discourage collaboration. It's OK to make a mess in the sandbox! But please, play nice.

Please update the sandbox README file with links and a brief description of your work.

In order to minimize the work necessary for others to install and try out new, experimental components, the following sandbox directory structure is recommended:

    sandbox/
        project_name/ # For a collaborative project.
            README.txt  # Describe the requirements, purpose/goals, usage,
                        # and list the maintainers.
            docs/
                ...
            component.py    # The component is a single module.
                            # *OR* (but *not* both)
            component/      # The component is a package.
                __init__.py  # Contains the Reader/Writer class.
                other1.py    # Other modules and data files used
                data.txt     # by this component.
                ...
            test/       # Test suite.
                ...
            tools/      # For front ends etc.
                ...
            setup.py    # Install the component code and tools/ files
                        # into the right places.
        userid/       # For *temporary* personal space.

Some sandbox projects are destined to move to the Docutils core once completed. Others, such as add-ons to Docutils or applications of Docutils, may graduate to become parallel projects.

## Parallel Projects

Parallel projects contain useful code that is not central to the functioning of Docutils. Examples are specialized add-ons or plug-ins, and applications of Docutils. They use Docutils, but Docutils does not require their presence to function.

An official parallel project will have its own directory beside (or parallel to) the main `docutils` directory in the Subversion repository. It can have its own web page in the docutils.sourceforge.net domain, its own file releases and downloadable snapshots, and even a mailing list if that proves useful. However, an official parallel project has implications: it is expected to be maintained and continue to work with changes to the core Docutils.

A parallel project requires a project leader, who must commit to coordinate and maintain the implementation:

- Answer questions from users and developers.
- Review suggestions, bug reports, and patches.
- Monitor changes and ensure the quality of the code and documentation.
- Coordinate with Docutils to ensure interoperability.
- Put together official project releases.

Of course, related projects may be created independently of Docutils. The advantage of a parallel project is that the SourceForge environment and the developer and user communities are already established. Core Docutils developers are available for consultation and may contribute to the parallel project. It's easier to keep the projects in sync when there are changes made to the core Docutils code.

## Other related projects

Many related but independent projects are listed in the Docutils link list. If you want your project to appear there, drop a note at the Docutils-develop mailing list.