Tests: Refactor SQLAlchemy-related doctests into documentation

docs/by-example/index.rst

@@ -10,6 +10,15 @@ About
 This part of the documentation contains examples how to use the CrateDB Python
 client, exercising different options and scenarios.
 
+
+DBAPI, HTTP, and BLOB interfaces
+================================
+
+The examples in this section are all about CrateDB's `Python DBAPI`_ interface,
+the plain HTTP API interface, and a convenience interface for working with
+`blob tables`_. It also details attributes, methods, and behaviors of the
+``Connection`` and ``Cursor`` objects.
+
 .. toctree::
     :maxdepth: 1
 
@@ -19,3 +28,25 @@ client, exercising different options and scenarios.
     blob
     connection
     cursor
+
+
+.. _sqlalchemy-by-example:
+
+SQLAlchemy interface
+====================
+
+The examples in this section are all about CrateDB's `SQLAlchemy`_ dialect, and
+its corresponding API interfaces, see also :ref:`sqlalchemy-support`.
+
+.. toctree::
+    :maxdepth: 1
+
+    sqlalchemy/getting-started
+    sqlalchemy/cru
+    sqlalchemy/inspection-reflection
+    sqlalchemy/internals
+
+
+.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html
+.. _Python DBAPI: https://peps.python.org/pep-0249/
+.. _SQLAlchemy: https://www.sqlalchemy.org/

src/crate/client/doctests/sqlalchemy.txt → docs/by-example/sqlalchemy/getting-started.rst

@@ -1,6 +1,17 @@
-==================
-SQLAlchemy Support
-==================
+===========================
+SQLAlchemy: Getting started
+===========================
+
+This section of the documentation outlines how to use CrateDB's SQLAlchemy
+integration. It demonstrates both basic database operations (insert, select,
+delete), creating and dropping tables, running queries with aggregations,
+as well as the use of complex and geospatial data types.
+
+.. rubric:: Table of Contents
+
+.. contents::
+   :local:
+
 
 Connection String
 =================
@@ -115,7 +126,7 @@ A regular select query will then fetch the whole documents::
     >>> [(c.name, c.details['gender']) for c in query]
     [('Arthur Dent', 'male'), ('Tricia McMillan', 'female')]
 
-But it is also possible to just select a part of the document, even inside the
+It is also possible to just select a part of the document, even inside the
 ``Object`` type::
 
     >>> sorted(session.query(Character.details['gender']).all())
@@ -242,9 +253,8 @@ Count and Group By
 ==================
 
 SQLAlchemy supports different approaches to issue a query with a count
-aggregate function. Take a look at the `Counting section in the tutorial
-<http://docs.sqlalchemy.org/en/latest/orm/tutorial.html#counting>`_ for a full
-overview.
+aggregate function. Take a look at the `count result rows`_ documentation
+for a full overview.
 
 CrateDB currently doesn't support all variants as it can't handle the
 sub-queries yet.
@@ -368,8 +378,8 @@ In order to create all missing tables the ``create_all`` method can be used::
      "('departments', 'id', 1, 'text')",
      "('departments', 'name', 2, 'text')"]
 
-Delete Tables
--------------
+Drop Tables
+-----------
 
 In order to delete all tables simply use ``Base.metadata.drop_all()``, or to
 delete a single table use ``drop(...)`` as shown below::
@@ -433,3 +443,6 @@ This will result in the following query::
 
     >>> pprint([str(r) for r in session.execute("Select content from archived_tasks")])
     ["('Write Tests',)"]
+
+
+.. _count result rows: http://docs.sqlalchemy.org/en/14/orm/tutorial.html#counting

docs/by-example/sqlalchemy/inspection-reflection.rst

@@ -0,0 +1,81 @@
+=====================================================
+SQLAlchemy: Database schema inspection and reflection
+=====================================================
+
+This section of the documentation, related to CrateDB's SQLAlchemy integration,
+focuses on database schema inspection and reflection features. That is, given
+that you connect to an existing database, how to introspect its metadata
+information to retrieve table- and view-names, and table column metadata.
+
+
+Introduction
+============
+
+The `runtime inspection API`_ provides the ``inspect()`` function, which
+delivers runtime information about a wide variety of SQLAlchemy objects, both
+within SQLAlchemy Core and the SQLAlchemy ORM.
+
+A low level interface which provides a backend-agnostic system of loading lists
+of schema, table, column, and constraint descriptions from a given database is
+available. This is known as the `SQLAlchemy inspector`_.
+
+    >>> inspector = sa.inspect(engine)
+
+
+Inspector usage
+===============
+
+List all schemas:
+
+    >>> inspector.get_schema_names()
+    ['blob', 'doc', 'information_schema', 'pg_catalog', 'sys']
+
+List all tables:
+
+    >>> set(['characters', 'cities', 'locations']).issubset(inspector.get_table_names())
+    True
+
+    >>> set(['checks', 'cluster', 'jobs', 'jobs_log']).issubset(inspector.get_table_names(schema='sys'))
+    True
+
+List all views:
+
+    >>> inspector.get_view_names()
+    ['characters_view']
+
+Get default schema name:
+
+    >>> inspector.default_schema_name
+    'doc'
+
+
+Schema-supported reflection
+===========================
+
+A ``Table`` object can be instructed to load information about itself from the
+corresponding database schema object already existing within the database. This
+process is called *reflection*, see `reflecting database objects`_.
+
+In the most simple case you need only specify the table name, a ``MetaData``
+object, and the ``autoload_with`` argument.
+
+Create a SQLAlchemy table object:
+
+    >>> meta = sa.MetaData()
+    >>> table = sa.Table(
+    ...     "characters", meta,
+    ...     autoload=True,
+    ...     autoload_with=engine)
+
+Reflect column data types from the table metadata:
+
+    >>> table.columns.get('name')
+    Column('name', String(), table=<characters>)
+
+    >>> table.primary_key
+    PrimaryKeyConstraint(Column('id', String(), table=<characters>, primary_key=True...
+
+
+.. _reflecting database objects: https://docs.sqlalchemy.org/en/14/core/reflection.html#reflecting-database-objects
+.. _runtime inspection API: https://docs.sqlalchemy.org/en/14/core/inspection.html
+.. _SQLAlchemy inspector: https://docs.sqlalchemy.org/en/14/core/reflection.html#fine-grained-reflection-with-inspector

src/crate/client/sqlalchemy/doctests/dialect.txt → docs/by-example/sqlalchemy/internals.rst

@@ -1,8 +1,15 @@
-======================
-CrateDialect Internals
-======================
+=====================
+SQLAlchemy: Internals
+=====================
 
-The initialize method sets the default schema name and version info::
+This section of the documentation, related to CrateDB's SQLAlchemy integration,
+focuses on showing specific internals.
+
+
+CrateDialect
+============
+
+The initialize method sets the default schema name and version info:
 
     >>> connection = engine.connect()
     >>> dialect = CrateDialect()
@@ -12,12 +19,12 @@ The initialize method sets the default schema name and version info::
     >>> dialect.server_version_info >= (1, 0, 0)
     True
 
-Check if table exists::
+Check if table exists:
 
     >>> dialect.has_table(connection, 'locations')
     True
 
-Check if schema exists::
+Check if schema exists:
 
     >>> dialect.has_schema(connection, 'doc')
     True

docs/sqlalchemy.rst

@@ -1,10 +1,22 @@
+.. _sqlalchemy-support:
 .. _using-sqlalchemy:
 
-============================
-Using the SQLAlchemy dialect
-============================
+==================
+SQLAlchemy support
+==================
 
-`SQLAlchemy`_ is a popular `Object-Relational Mapping`_ (ORM) tool for Python.
+.. rubric:: Table of contents
+
+.. contents::
+    :local:
+    :depth: 2
+
+
+Introduction
+============
+
+`SQLAlchemy`_ is a popular `Object-Relational Mapping`_ (ORM) library for
+Python.
 
 The CrateDB Python client library provides support for SQLAlchemy. A CrateDB
 `dialect`_ is registered at installation time and can be used without further
@@ -13,25 +25,17 @@ configuration.
 The CrateDB Python client library is validated to work with SQLAlchemy versions
 ``1.3`` and ``1.4``.
 
-.. NOTE::
-
-   This page documents the CrateDB SQLAlchemy dialect.
-
-   For help using the CrateDB `Database API`_ client, consult :ref:`the client
-   documentation <connect>`.
-
 .. SEEALSO::
 
-   Supplementary information about the CrateDB SQLAlchemy dialect can be found
-   in the :ref:`data types appendix <data-types-sqlalchemy>`.
+    For general help using SQLAlchemy, consult the `SQLAlchemy tutorial`_ or the
+    `SQLAlchemy library`_.
 
-   For general help using SQLAlchemy, consult the `SQLAlchemy tutorial`_ or the
-   `SQLAlchemy library`_ .
+    Supplementary information about the CrateDB SQLAlchemy dialect can be found
+    in the :ref:`data types appendix <data-types-sqlalchemy>`.
 
-.. rubric:: Table of contents
+    Code examples for using the CrateDB SQLAlchemy dialect can be found at
+    :ref:`sqlalchemy-by-example`.
 
-.. contents::
-   :local:
 
 .. _connecting:
 
@@ -629,7 +633,7 @@ column on the ``Character`` class.
 .. _operator: https://docs.python.org/2/library/operator.html
 .. _any: http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.ARRAY.Comparator.any
 .. _tuple: https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range
-.. _count result rows: http://docs.sqlalchemy.org/en/latest/orm/tutorial.html#counting
+.. _count result rows: http://docs.sqlalchemy.org/en/14/orm/tutorial.html#counting
 .. _MATCH predicate: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#match-predicate
 .. _arguments reference: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#arguments
 .. _boost values: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#arguments

src/crate/client/tests.py

@@ -345,33 +345,15 @@ def test_suite():
     suite.addTest(doctest.DocTestSuite('crate.client.connection'))
     suite.addTest(doctest.DocTestSuite('crate.client.http'))
 
-    s = doctest.DocFileSuite(
-        'sqlalchemy/doctests/itests.txt',
-        'sqlalchemy/doctests/dialect.txt',
-        'sqlalchemy/doctests/reflection.txt',
-        setUp=setUpCrateLayerAndSqlAlchemy,
-        tearDown=tearDownWithCrateLayer,
-        optionflags=flags,
-        encoding='utf-8'
-    )
-    s.layer = ensure_cratedb_layer()
-    suite.addTest(s)
-
     s = doctest.DocFileSuite(
         'docs/by-example/http.rst',
         'docs/by-example/client.rst',
         'docs/by-example/blob.rst',
+        'docs/by-example/sqlalchemy/getting-started.rst',
+        'docs/by-example/sqlalchemy/cru.rst',
+        'docs/by-example/sqlalchemy/inspection-reflection.rst',
+        'docs/by-example/sqlalchemy/internals.rst',
         module_relative=False,
-        setUp=setUpWithCrateLayer,
-        tearDown=tearDownWithCrateLayer,
-        optionflags=flags,
-        encoding='utf-8'
-    )
-    s.layer = ensure_cratedb_layer()
-    suite.addTest(s)
-
-    s = doctest.DocFileSuite(
-        'doctests/sqlalchemy.txt',
         setUp=setUpCrateLayerAndSqlAlchemy,
         tearDown=tearDownWithCrateLayer,
         optionflags=flags,