Refactor certain doctests towards either reStructuredText documentation or Python tests

.coveragerc

@@ -7,4 +7,5 @@ omit =
     parts/*
     eggs/*
 exclude_lines =
+    # pragma: no cover
     raise NotImplemented

.gitignore

@@ -18,3 +18,4 @@ eggs/
 htmlcov/
 out/
 parts/
+tmp/

DEVELOP.rst

@@ -36,6 +36,25 @@ Ignore specific test directories::
 
     ./bin/test -vvvv --ignore_dir=testing
 
+The ``LayerTest`` test cases have quite some overhead. Omitting them will save
+a few cycles (~70 seconds runtime)::
+
+    ./bin/test -t '!LayerTest'
+
+Invoke all tests without integration tests (~15 seconds runtime)::
+
+    ./bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest'
+
+Yet ~130 test cases, but only ~5 seconds runtime::
+
+    ./bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest' \
+        -t '!test_client_threaded' -t '!test_no_retry_on_read_timeout' \
+        -t '!test_wait_for_http' -t '!test_table_clustered_by'
+
+To inspect the whole list of test cases, run::
+
+    ./bin/test --list-tests
+
 You can run the tests against multiple Python interpreters with `tox`_::
 
     tox

bin/test

@@ -11,6 +11,7 @@ base = os.path.dirname(base)
 sys.argv[0] = os.path.abspath(sys.argv[0])
 
 if __name__ == '__main__':
-    zope.testrunner.run((['--auto-color', '--verbose']) + [
-        '--test-path', join(base, 'src'),
-        ])
+    zope.testrunner.run([
+        '-vvv', '--auto-color',
+        '--test-path', join(base, 'src')],
+    )

codecov.yml

@@ -3,7 +3,12 @@
 
 coverage:
   status:
+
     project:
       default:
-        target: 85%    # the required coverage value
-        threshold: 3%  # the leniency in hitting the target
+        target: auto    # the required coverage value
+        threshold: 3%   # the leniency in hitting the target
+
+    patch:
+      default:
+        informational: true

src/crate/client/doctests/blob.txt → docs/by-example/blob.rst

@@ -1,9 +1,13 @@
-==============
-Blob Container
-==============
+==================
+Blob container API
+==================
+
+The connection object provides a convenience API for easy access to `blob
+tables`_.
 
-The connection provides a blob container implementation for easy use of the
-bloba API.
+
+Get blob container handle
+=========================
 
 Create a connection::
 
@@ -15,14 +19,14 @@ Get a blob container::
     >>> container = client.get_blob_container('myfiles')
 
 
-Store Blobs
+Store blobs
 ===========
 
-The container allows to store a blob without explicitely providing the hash
+The container allows to store a blob without explicitly providing the hash
 for the blob. This feature is possible if the blob is provided as a seekable
 stream like object.
 
-Store a StringIO stream::
+Store a ``StringIO`` stream::
 
     >>> from io import BytesIO
     >>> f = BytesIO(b'StringIO data')
@@ -60,7 +64,7 @@ Check for existence
     False
 
 
-Retrieve Blobs
+Retrieve blobs
 ==============
 
 Blobs can be retrieved using its hash::
@@ -73,7 +77,7 @@ Blobs can be retrieved using its hash::
     True
 
 
-Delete Blobs
+Delete blobs
 ============
 
 Blobs can be deleted using its hash::
@@ -88,10 +92,12 @@ Trying to delete a not existing blob::
     >>> container.delete(string_blob)
     False
 
-Close Connection
+Close connection
 ================
 
 Close the connection to clear the connection pool::
 
     >>> client.close()
 
+
+.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html

src/crate/client/doctests/client.txt → docs/by-example/client.rst

@@ -1,8 +1,19 @@
-============
-Client Usage
-============
+===============
+Database client
+===============
 
-Connect to a Database
+``crate.client.connect`` is the primary method to connect to CrateDB using
+Python. This section of the documentation outlines different methods to connect
+to the database cluster, as well as how to run basic inquiries to the database,
+and closing the connection again.
+
+.. rubric:: Table of Contents
+
+.. contents::
+   :local:
+
+
+Connect to a database
 =====================
 
 Before we can start we have to import the client::

src/crate/client/doctests/connection.txt → docs/by-example/connection.rst

@@ -1,17 +1,35 @@
-==========
-Connection
-==========
+=====================
+The Connection object
+=====================
 
-::
+This documentation section outlines different attributes, methods, and
+behaviors of the ``crate.client.connection.Connection`` object.
+
+The examples use an instance of ``ClientMocked`` instead of a real ``Client``
+instance. This allows us to verify the examples without needing a real database
+connection.
+
+.. rubric:: Table of Contents
+
+.. contents::
+   :local:
 
-    >>> from crate.client import connect
 
-We create a new connection object::
+connect()
+=========
+
+This section sets up a connection object, and inspects some of its attributes.
+
+    >>> from crate.client import connect
+    >>> from crate.client.test_util import ClientMocked
 
-    >>> connection = connect(client=connection_client_mocked)
+    >>> connection = connect(client=ClientMocked())
     >>> connection.lowest_server_version.version
     (2, 0, 0)
 
+cursor()
+========
+
 Calling the ``cursor()`` function on the connection will
 return a cursor object::
 
@@ -23,6 +41,9 @@ cursor object::
     >>> cursor.rowcount
     -1
 
+close()
+=======
+
 Now we close the connection::
 
     >>> connection.close()
@@ -45,9 +66,3 @@ raise a ``ProgrammingError``::
     Traceback (most recent call last):
     ...
     crate.client.exceptions.ProgrammingError: Connection closed
-
-    >>> connection = connect(client=connection_client_mocked)
-    >>> cursor = connection.cursor()
-    >>> cursor.rowcount
-    -1
-

src/crate/client/doctests/cursor.txt → docs/by-example/cursor.rst

@@ -1,35 +1,44 @@
-======
-Cursor
-======
+=================
+The Cursor object
+=================
 
+This documentation section outlines different attributes, methods, and
+behaviors of the ``crate.client.cursor.Cursor`` object.
 
-Setup
-=====
+The example code uses ``ClientMocked`` and ``set_next_response`` for
+demonstration purposes, so they don't need a real database connection.
+
+.. rubric:: Table of Contents
+
+.. contents::
+   :local:
+
+
+Introduction
+============
 
 This section sets up a cursor object, inspects some of its attributes, and sets
 up the response for subsequent cursor operations.
 
-::
-
     >>> from crate.client import connect
     >>> from crate.client.converter import DefaultTypeConverter
     >>> from crate.client.cursor import Cursor
+    >>> from crate.client.test_util import ClientMocked
 
-    >>> connection = connect(client=connection_client_mocked)
+    >>> connection = connect(client=ClientMocked())
     >>> cursor = connection.cursor()
 
-The rowcount and duration attribute is -1 in case no ``execute()`` has been performed on the cursor.
-
-::
+The ``rowcount`` and ``duration`` attributes are ``-1``, in case no ``execute()`` has
+been performed on the cursor yet.
 
     >>> cursor.rowcount
     -1
 
     >>> cursor.duration
     -1
 
-Hardcode the next response of the mocked connection client, so we won't need a sql statement
-to execute::
+Define the response of the mocked connection client. It will be returned on
+request without needing to execute an SQL statement.
 
     >>> connection.client.set_next_response({
     ...     "rows":[ [ "North West Ripple", 1 ], [ "Arkintoofle Minor", 3 ], [ "Alpha Centauri", 3 ] ],
@@ -168,8 +177,6 @@ Iterating over a new cursor without results will immediately raise a Programming
 description
 ===========
 
-::
-
     >>> cursor.description
     (('name', None, None, None, None, None, None), ('position', None, None, None, None, None, None))
 
@@ -238,8 +245,8 @@ The attribute is -1 in case the cursor has been closed::
     ...     "cols":[ "name", "position" ],
     ... })
 
-executemany
-===========
+executemany()
+=============
 
 ``executemany()`` allows to execute a single sql statement against a sequence
 of parameters::
@@ -340,15 +347,15 @@ inspect the ``DataType`` enum, or the documentation about the list of available
 `CrateDB data type identifiers for the HTTP interface`_.
 
 To create a simple converter for converging CrateDB's ``BIT`` type to Python's
-``int`` type::
+``int`` type.
 
     >>> from crate.client.converter import Converter, DataType
 
     >>> converter = Converter({DataType.BIT: lambda value: int(value[2:-1], 2)})
     >>> cursor = connection.cursor(converter=converter)
 
 Proof that the converter works correctly, ``B\'0110\'`` should be converted to
-``6``. CrateDB's ``BIT`` data type has the numeric identifier ``25``::
+``6``. CrateDB's ``BIT`` data type has the numeric identifier ``25``.
 
     >>> connection.client.set_next_response({
     ...     "col_types": [25],
@@ -374,8 +381,6 @@ desired time zone.
 For your reference, in the following examples, epoch 1658167836758 is
 ``Mon, 18 Jul 2022 18:10:36 GMT``.
 
-::
-
     >>> import datetime
     >>> tz_mst = datetime.timezone(datetime.timedelta(hours=7), name="MST")
     >>> cursor = connection.cursor(time_zone=tz_mst)
@@ -402,7 +407,7 @@ The available options are:
 - ``zoneinfo.ZoneInfo("Australia/Sydney")``
 - ``+0530`` (UTC offset in string format)
 
-Let's exercise all of them::
+Let's exercise all of them:
 
     >>> cursor.time_zone = datetime.timezone.utc
     >>> cursor.execute('')