New test test_should_remove_ref_on_disconnect in 0.8.0 fails on Fedora Linux

[musicinmybrain]

I’m testing release 0.8.0 for Fedora Linux and found that one of the tests introduced in this release is failing on all Fedora Linux releases; although an update would probably only target Fedora Rawhide, I checked Fedora 37, 38, and 39 too.

=================================== FAILURES ===================================
_____________________ test_should_remove_ref_on_disconnect _____________________

    @async_adapter
    async def test_should_remove_ref_on_disconnect():
        async with Database("sqlite:///file::memory:?cache=shared", uri=True) as database:
            query = sqlalchemy.schema.CreateTable(notes)
            await database.execute(query)

            query = notes.insert()
            values = {"text": "example1", "completed": True}
            await database.execute(query, values)

        async with Database("sqlite:///file::memory:?cache=shared", uri=True) as database:
            query = notes.select()
>           with pytest.raises(sqlite3.OperationalError):
E           Failed: DID NOT RAISE <class 'sqlite3.OperationalError'>

tests/test_databases.py:1590: Failed

Fedora 37/38 are using Python 3.11.4 and sqlite 3.40.1; Fedora 39/40 are using Python 3.12.0rc1 and sqlite 3.42.0.

Please let me know if there is anything I can do to help investigate this.

[zanieb]

Thanks for raising! This test as added as part of #561 — I believe an operational error should be raised there because a new database should be created on the second connection. I'm not certain of the details of how the shared in-memory database is cleared when its reference is dropped, we'll probably need to investigate that.

[zanieb]

Can you give 1d30e39 a try?

[musicinmybrain]

Can you give 1d30e39 a try?

When applied as a patch to 0.8.0, that commit seemed to fix the failure reported in this bug in local testing. However, I’m now encountering the following on the real builders:

=================================== FAILURES ===================================
_____________________ test_should_remove_ref_on_disconnect _____________________
    @async_adapter
    async def test_should_remove_ref_on_disconnect():
        async with Database("sqlite:///file::memory:?cache=shared", uri=True) as database:
            query = sqlalchemy.schema.CreateTable(notes)
>           await database.execute(query)
tests/test_databases.py:1582: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
databases/core.py:199: in execute
    return await connection.execute(query, values)
databases/core.py:318: in execute
    return await self._connection.execute(built_query)
databases/backends/sqlite.py:145: in execute
    await cursor.execute(query_str, args)
/usr/lib/python3.12/site-packages/aiosqlite/cursor.py:48: in execute
    await self._execute(self._cursor.execute, sql, parameters)
/usr/lib/python3.12/site-packages/aiosqlite/cursor.py:40: in _execute
    return await self._conn._execute(fn, *args, **kwargs)
/usr/lib/python3.12/site-packages/aiosqlite/core.py:133: in _execute
    return await future
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <Connection(Thread-142, started 281473458041216)>
    def run(self) -> None:
        """
        Execute function calls on a separate thread.
    
        :meta private:
        """
        while True:
            # Continues running until all queue items are processed,
            # even after connection is closed (so we can finalize all
            # futures)
            try:
                future, function = self._tx.get(timeout=0.1)
            except Empty:
                if self._running:
                    continue
                break
            try:
                LOG.debug("executing %s", function)
>               result = function()
E               sqlite3.OperationalError: table notes already exists
/usr/lib/python3.12/site-packages/aiosqlite/core.py:106: OperationalError

I don’t see this in every build; I haven’t yet established whether it is flaky (racy) or arch-dependent, and I don’t understand why I haven’t been able to reproduce it locally.

[zanieb]

Thanks! I presume this is also related to garbage collection but between tests now. #569 may resolve.

[musicinmybrain]

I don’t see this in every build; I haven’t yet established whether it is flaky (racy) or arch-dependent, and I don’t understand why I haven’t been able to reproduce it locally.

In five scratch-builds for Fedora Rawhide across all architectures, I saw the above failure on:

• x86_64 in 1/5 builds
• i686 in 5/5 builds
• ppc64le in 0/5 builds
• s390x in 1/5 builds
• aarch64 in 3/5 builds

So it looks like this is present on all architectures but perhaps with different probability.

[musicinmybrain]

Thanks! I presume this is also related to garbage collection but between tests now. #569 may resolve.

I will check this next.

[musicinmybrain]

* `i686` in 5/5 builds

I should not have tested this architecture; it is excluded in the Fedora package, which is why the build failed every time.

[musicinmybrain]

Thanks! I presume this is also related to garbage collection but between tests now. #569 may resolve.

I will check this next.

#569 worked in an initial scratch build. I’ll try it a few more times to make sure there aren’t any other (frequent) flaky failures.

[musicinmybrain]

#569 looks good to me. No failures in five builds on each architecture. Thanks for investigating this.

[zanieb]

Thanks for your help!