Do we need to dump operations to a database and apply them later?

[99alfie]

As part of #10, in test_session.py -> test_deferred_operations, a number of ftrack operations is recorded and dumped to a database, then opened and applied in another session.

This currently has the following issue:

• delete is not supported as an operation atm, so any entities created cannot be deleted and will disrupt the test if run again/leaving garbage in ftrack

Before proceeding and implementing "delete" as a supported operation, I like to know why that feature exists and why anyone would use it. It seems to overcomplicate (for one, the tests). Also, it forces use of a database which is not human readable and is not what I would prefer for a dump file.

If nobody uses it there seems to be no reason to keep it / engineering references: KISS/YAGNI - and "just because you can does not mean you have to".

I would opt to get rid of the functionality in the core - let the user implement this if he wants to

[rkoschmitzky]

Hello @99alfie,

Could you please explain your approach to running the tests? Based on my recollection of your description, it appears that the issue lies within the test setup rather than the core functionalities of the Session. This issue seems to arise when tests are executed in parallel.

This was precisely the rationale behind the inclusion of this change for the actions.

I am curious to understand the purpose of this feature and the scenarios in which it would be advantageous. It appears to introduce unnecessary complexity, particularly in relation to the test suite. Additionally, it enforces the use of a non-human-readable database format, which does not align with my preference for a dump file.

The primary motivation behind introducing this feature was to establish a separation that became necessary when we aimed to divide the distinct extraction and integration phases of a publishing process. This separation deferred the final creation of AssetVersions, allowing the server to dictate the resulting version number.

Recorded operations could be incorporated into Ftrack itself as a published artifact. Subsequently, a separate process could take over and perform further operations on these artifacts. We opted to retain the dbm database, as the Ftrack Python API internally relied on it.

In the absence of users leveraging this feature, it might be worth reconsidering its necessity. Adhering to the principles of KISS (Keep It Simple, Stupid) and YAGNI (You Ain't Gonna Need It), it's important to remember that just because a capability exists doesn't mean it must be utilized.

Given that this API is still in a pre-release stage, it remains uncertain which aspects are actually being utilized. Hence, my suggestion would be to maintain the existing features for now. Once this project gains reasonable adoption, we can then engage in a more in-depth discussion about potentially phasing out certain features.

delete is not supported as an operation atm, so any entities created cannot be deleted and will disrupt the test if run again/leaving garbage in ftrack

Could you provide more detail on this point? Are you indicating that when a deletion is carried out through the Ftrack Python API, it does not get captured in the stack of recorded operations?

[99alfie]

Hi @rkoschmitzky,

thank you, I understand it better now. So let us keep the feature and fix the tests instead.

About the "delete" operation, this is the part of the test in question:

https://github.com/TrixterFilm/trackteroid/blob/5467820e458947c62118bd6ed8cbebe5f1dc5318/tests/test_session.py#L47C1-L59C43

line 49: query_result contains a random production AssetVersion
line 56: it is again queried by its id and assigned to "avs"
line 57: its version number is incremented by 10 - this seems to be an error
line 59: a new asset version is created (avs2), with its version being 10 plus the avs version
avs2 is not deleted

when running SESSION.reconnect_and_commit(), the following happens on its first run:
a random production version (for example, with version number 3) is changed to version number 13. A new version gets created with version number 23. Nothing is reverted. The test succeeds.

when running SESSION.reconnect_and_commit(), a second time, it finds version number 13, tries to create version number 23 and gets a duplicate entry, so the test fails

Logical for the test would be:

• existing asset version is not modified
• latest version is found
• a new version is created
• the new version is deleted

deletion is not supported, only "create" and "update":
https://github.com/TrixterFilm/trackteroid/blob/5467820e458947c62118bd6ed8cbebe5f1dc5318/src/trackteroid/session.py#L204C11-L204C11

so in order to make the test work, we should implement "delete" so as not to leave dangling entities behind?

[rkoschmitzky]

@99alfie I see,

I didn't realize that a delete operation wouldn't be taken care of. It's definitely important to include this functionality.

when running SESSION.reconnect_and_commit(), the following happens on its first run:
a random production version (for example, with version number 3) is changed to version number 13. A new version gets created with version number 23. Nothing is reverted. The test succeeds.

when running SESSION.reconnect_and_commit(), a second time, it finds version number 13, tries to create version number 23 and gets a duplicate entry, so the test fails

The operations created as part of test_deferred_operations aren't committed at all via reconnect_and_commit.
Only the predefined "initial" .dbm file fixture is what is being committed to the server.

Your idea of integrating entity deletion within the same operation stack is quite reasonable though. However, I'm wondering how we can verify that all operations have been accurately transmitted to the server. While it might be tempting to assume successful transmission in the absence of server errors, the truth is that there are no solid guarantees that the data would resemble the recordings, especially if there is no remaining data left to make a comparison with.

If we take the current initial operations as they are (creating a demo user) and would delete the resulting entity, are there any traces we might be able to follow?

[99alfie]

Hi @rkoschmitzky,
ok. Sorry forgot to mention that I am working on the fix/#i10-pytests branch. I just see it has modifications to test_session.py which seem to cause unexpected behaviour. I am reverting it to the original and will start from there now. This might be what is causing the confusion here.

[99alfie]

...that was it, I got the tests working now.