Visitor and Traversal Utilities — SQLAlchemy 2.0 Documentation

Release: 2.0.0rc1 | Release Date: December 28, 2022

# SQLAlchemy 2.0 Documentation

# Visitor and Traversal Utilities

The `sqlalchemy.sql.visitors` module consists of classes and functions that serve the purpose of generically traversing a Core SQL expression structure. This is not unlike the Python `ast` module in that is presents a system by which a program can operate upon each component of a SQL expression. Common purposes this serves are locating various kinds of elements such as `Table` or `BindParameter` objects, as well as altering the state of the structure such as replacing certain FROM clauses with others.

Note

the `sqlalchemy.sql.visitors` module is an internal API and is not fully public. It is subject to change and may additionally not function as expected for use patterns that aren’t considered within SQLAlchemy’s own internals.

The `sqlalchemy.sql.visitors` module is part of the internals of SQLAlchemy and it is not usually used by calling application code. It is however used in certain edge cases such as when constructing caching routines as well as when building out custom SQL expressions using the Custom SQL Constructs and Compilation Extension.

Visitor/traversal interface and library functions.

Object Name | Description
--- | ---
anon_map | alias of `cache_anon_map`
cloned_traverse(obj, opts, visitors) | Clone the given expression structure, allowing modifications by visitors for mutable objects.
ExternalTraversal | Base class for visitor objects which can traverse externally using the `traverse()` function.
InternalTraversal | Defines visitor symbols used for internal traversal.
iterate(obj[, opts]) | Traverse the given expression structure, returning an iterator.
replacement_traverse(obj, opts, replace) | Clone the given expression structure, allowing element replacement by a given replacement function.
traverse(obj, opts, visitors) | Traverse and visit the given expression structure using the default iterator.
traverse_using(iterator, obj, visitors) | Visit the given expression structure using the given iterator of objects.
Visitable | Base class for visitable objects.

class sqlalchemy.sql.visitors.ExternalTraversal

  Base class for visitor objects which can traverse externally using the `traverse()` function.

  Direct usage of the `traverse()` function is usually preferred.

  Members

  chain(), iterate(), traverse(), visitor_iterator

  Class signature

  class `sqlalchemy.sql.visitors.ExternalTraversal` (`sqlalchemy.util.langhelpers.MemoizedSlots`)

  method `sqlalchemy.sql.visitors.ExternalTraversal.` chain(visitor: ExternalTraversal) → _ExtT

    ‘Chain’ an additional ExternalTraversal onto this ExternalTraversal

    The chained visitor will receive all visit events after this one.

  method `sqlalchemy.sql.visitors.ExternalTraversal.` iterate(obj: Optional[ExternallyTraversible]) → Iterator[ExternallyTraversible]

    Traverse the given expression structure, returning an iterator of all elements.

  method `sqlalchemy.sql.visitors.ExternalTraversal.` traverse(obj: Optional[ExternallyTraversible]) → Optional[ExternallyTraversible]

    Traverse and visit the given expression structure.

  attribute `sqlalchemy.sql.visitors.ExternalTraversal.` visitor_iterator

    Iterate through this visitor and each ‘chained’ visitor.

class sqlalchemy.sql.visitors.InternalTraversal

  Defines visitor symbols used for internal traversal.

  The `InternalTraversal` class is used in two ways. One is that it can serve as the superclass for an object that implements the various visit methods of the class. The other is that the symbols themselves of `InternalTraversal` are used within the `_traverse_internals` collection. Such as, the `Case` object defines `_traverse_internals` as

      _traverse_internals = [
          ("value", InternalTraversal.dp_clauseelement),
          ("whens", InternalTraversal.dp_clauseelement_tuples),
          ("else_", InternalTraversal.dp_clauseelement),
      ]

  Above, the `Case` class indicates its internal state as the attributes named `value`, `whens`, and `else_`. They each link to an `InternalTraversal` method which indicates the type of datastructure referred towards.

  Using the `_traverse_internals` structure, objects of type `InternalTraversible` will have the following methods automatically implemented:

  - `HasTraverseInternals.get_children()`

  - `HasTraverseInternals._copy_internals()`

  - `HasCacheKey._gen_cache_key()`

  Subclasses can also implement these methods directly, particularly for the `HasTraverseInternals._copy_internals()` method, when special steps are needed.

  New in version 1.4.

  Members

  dp_annotations_key, dp_anon_name, dp_boolean, dp_clauseelement, dp_clauseelement_list, dp_clauseelement_tuple, dp_clauseelement_tuples, dp_dialect_options, dp_dml_multi_values, dp_dml_ordered_values, dp_dml_values, dp_fromclause_canonical_column_collection, dp_fromclause_ordered_set, dp_has_cache_key, dp_has_cache_key_list, dp_has_cache_key_tuples, dp_ignore, dp_inspectable, dp_inspectable_list, dp_multi, dp_multi_list, dp_named_ddl_element, dp_operator, dp_plain_dict, dp_plain_obj, dp_prefix_sequence, dp_propagate_attrs, dp_statement_hint_list, dp_string, dp_string_clauseelement_dict, dp_string_list, dp_string_multi_dict, dp_table_hint_list, dp_type, dp_unknown_structure

  Class signature

  class `sqlalchemy.sql.visitors.InternalTraversal` (`enum.Enum`)

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_annotations_key = 'AK'

    Visit the _annotations_cache_key element.

    This is a dictionary of additional information about a ClauseElement that modifies its role. It should be included when comparing or caching objects, however generating this key is relatively expensive. Visitors should check the “_annotations” dict for non-None first before creating this key.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_anon_name = 'AN'

    Visit a potentially “anonymized” string value.

    The string value is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_boolean = 'B'

    Visit a boolean value.

    The boolean value is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_clauseelement = 'CE'

    Visit a `ClauseElement` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_clauseelement_list = 'CL'

    Visit a list of `ClauseElement` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_clauseelement_tuple = 'CT'

    Visit a tuple of `ClauseElement` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_clauseelement_tuples = 'CTS'

    Visit a list of tuples which contain `ClauseElement` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_dialect_options = 'DO'

    Visit a dialect options structure.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_dml_multi_values = 'DML_MV'

    Visit the values() multi-valued list of dictionaries of an `Insert` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_dml_ordered_values = 'DML_OV'

    Visit the values() ordered tuple list of an `Update` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_dml_values = 'DML_V'

    Visit the values() dictionary of a `ValuesBase` (e.g. Insert or Update) object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_fromclause_canonical_column_collection = 'FC'

    Visit a `FromClause` object in the context of the `columns` attribute.

    The column collection is “canonical”, meaning it is the originally defined location of the `ColumnClause` objects. Right now this means that the object being visited is a `TableClause` or `Table` object only.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_fromclause_ordered_set = 'CO'

    Visit an ordered set of `FromClause` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_has_cache_key = 'HC'

    Visit a `HasCacheKey` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_has_cache_key_list = 'HL'

    Visit a list of `HasCacheKey` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_has_cache_key_tuples = 'HT'

    Visit a list of tuples which contain `HasCacheKey` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_ignore = 'IG'

    Specify an object that should be ignored entirely.

    This currently applies function call argument caching where some arguments should not be considered to be part of a cache key.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_inspectable = 'IS'

    Visit an inspectable object where the return value is a `HasCacheKey` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_inspectable_list = 'IL'

    Visit a list of inspectable objects which upon inspection are HasCacheKey objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_multi = 'M'

    Visit an object that may be a `HasCacheKey` or may be a plain hashable object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_multi_list = 'MT'

    Visit a tuple containing elements that may be `HasCacheKey` or may be a plain hashable object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_named_ddl_element = 'DD'

    Visit a simple named DDL element.

    The current object used by this method is the `Sequence`.

    The object is only considered to be important for cache key generation as far as its name, but not any other aspects of it.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_operator = 'O'

    Visit an operator.

    The operator is a function from the `sqlalchemy.sql.operators` module.

    The operator value is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_plain_dict = 'PD'

    Visit a dictionary with string keys.

    The keys of the dictionary should be strings, the values should be immutable and hashable. The dictionary is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_plain_obj = 'PO'

    Visit a plain python object.

    The value should be immutable and hashable, such as an integer. The value is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_prefix_sequence = 'PS'

    Visit the sequence represented by `HasPrefixes` or `HasSuffixes`.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_propagate_attrs = 'PA'

    Visit the propagate attrs dict. This hardcodes to the particular elements we care about right now.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_statement_hint_list = 'SH'

    Visit the `_statement_hints` collection of a `Select` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_string = 'S'

    Visit a plain string value.

    Examples include table and column names, bound parameter keys, special keywords such as “UNION”, “UNION ALL”.

    The string value is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_string_clauseelement_dict = 'CD'

    Visit a dictionary of string keys to `ClauseElement` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_string_list = 'SL'

    Visit a list of strings.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_string_multi_dict = 'MD'

    Visit a dictionary of string keys to values which may either be plain immutable/hashable or `HasCacheKey` objects.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_table_hint_list = 'TH'

    Visit the `_hints` collection of a `Select` object.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_type = 'T'

    Visit a `TypeEngine` object

    The type object is considered to be significant for cache key generation.

  attribute `sqlalchemy.sql.visitors.InternalTraversal.` dp_unknown_structure = 'UK'

    Visit an unknown structure.

class sqlalchemy.sql.visitors.Visitable

  Base class for visitable objects.

  `Visitable` is used to implement the SQL compiler dispatch functions. Other forms of traversal such as for cache key generation are implemented separately using the `HasTraverseInternals` interface.

  Changed in version 2.0: The `Visitable` class was named `Traversible` in the 1.4 series; the name is changed back to `Visitable` in 2.0 which is what it was prior to 1.4.

  Both names remain importable in both 1.4 and 2.0 versions.

attribute `sqlalchemy.sql.visitors..`sqlalchemy.sql.visitors.anon_map

  alias of `cache_anon_map`

function sqlalchemy.sql.visitors.cloned_traverse(obj: Optional[ExternallyTraversible], opts: Mapping[str, Any], visitors: Mapping[str, Callable[[Any], None]]) → Optional[ExternallyTraversible]

  Clone the given expression structure, allowing modifications by visitors for mutable objects.

  Traversal usage is the same as that of `traverse()`. The visitor functions present in the `visitors` dictionary may also modify the internals of the given structure as the traversal proceeds.

  The `cloned_traverse()` function does not provide objects that are part of the `Immutable` interface to the visit methods (this primarily includes `ColumnClause`, `Column`, `TableClause` and `Table` objects). As this traversal is only intended to allow in-place mutation of objects, `Immutable` objects are skipped. The `Immutable._clone()` method is still called on each object to allow for objects to replace themselves with a different object based on a clone of their sub-internals (e.g. a `ColumnClause` that clones its subquery to return a new `ColumnClause`).

  Changed in version 2.0: The `cloned_traverse()` function omits objects that are part of the `Immutable` interface.

  The central API feature used by the `cloned_traverse()` and `replacement_traverse()` functions, in addition to the `ClauseElement.get_children()` function that is used to achieve the iteration, is the `ClauseElement._copy_internals()` method. For a `ClauseElement` structure to support cloning and replacement traversals correctly, it needs to be able to pass a cloning function into its internal members in order to make copies of them.

  See also

  `traverse()`

  `replacement_traverse()`

function sqlalchemy.sql.visitors.iterate(obj: Optional[ExternallyTraversible], opts: Mapping[str, Any] = {}) → Iterator[ExternallyTraversible]

  Traverse the given expression structure, returning an iterator.

  Traversal is configured to be breadth-first.

  The central API feature used by the `iterate()` function is the `ClauseElement.get_children()` method of `ClauseElement` objects. This method should return all the `ClauseElement` objects which are associated with a particular `ClauseElement` object. For example, a `Case` structure will refer to a series of `ColumnElement` objects within its “whens” and “else_” member variables.

  Parameters:

    - obj – `ClauseElement` structure to be traversed

    - opts – dictionary of iteration options. This dictionary is usually empty in modern usage.

function sqlalchemy.sql.visitors.replacement_traverse(obj: Optional[ExternallyTraversible], opts: Mapping[str, Any], replace: _TraverseTransformCallableType[Any]) → Optional[ExternallyTraversible]

  Clone the given expression structure, allowing element replacement by a given replacement function.

  This function is very similar to the `cloned_traverse()` function, except instead of being passed a dictionary of visitors, all elements are unconditionally passed into the given replace function. The replace function then has the option to return an entirely new object which will replace the one given. If it returns `None`, then the object is kept in place.

  The difference in usage between `cloned_traverse()` and `replacement_traverse()` is that in the former case, an already-cloned object is passed to the visitor function, and the visitor function can then manipulate the internal state of the object. In the case of the latter, the visitor function should only return an entirely different object, or do nothing.

  The use case for `replacement_traverse()` is that of replacing a FROM clause inside of a SQL structure with a different one, as is a common use case within the ORM.

function sqlalchemy.sql.visitors.traverse(obj: Optional[ExternallyTraversible], opts: Mapping[str, Any], visitors: Mapping[str, Callable[[Any], None]]) → Optional[ExternallyTraversible]

  Traverse and visit the given expression structure using the default iterator.

  > e.g.:

  >     from sqlalchemy.sql import visitors

  >     stmt = select(some_table).where(some_table.c.foo == 'bar')

  >     def visit_bindparam(bind_param):
  >         print("found bound value: %s" % bind_param.value)

  >     visitors.traverse(stmt, {}, {"bindparam": visit_bindparam})

  The iteration of objects uses the `iterate()` function, which does a breadth-first traversal using a stack.

  Parameters:

    - obj – `ClauseElement` structure to be traversed

    - opts – dictionary of iteration options. This dictionary is usually empty in modern usage.

    - visitors – dictionary of visit functions. The dictionary should have strings as keys, each of which would correspond to the `__visit_name__` of a particular kind of SQL expression object, and callable functions as values, each of which represents a visitor function for that kind of object.

function sqlalchemy.sql.visitors.traverse_using(iterator: Iterable[ExternallyTraversible], obj: Optional[ExternallyTraversible], visitors: Mapping[str, Callable[[Any], None]]) → Optional[ExternallyTraversible]

  Visit the given expression structure using the given iterator of objects.

  `traverse_using()` is usually called internally as the result of the `traverse()` function.

  Parameters:

    - iterator – an iterable or sequence which will yield `ClauseElement` structures; the iterator is assumed to be the product of the `iterate()` function.

    - obj – the `ClauseElement` that was used as the target of the `iterate()` function.

    - visitors – dictionary of visit functions. See `traverse()` for details on this dictionary.

  See also

  `traverse()`