User-Acceptance Testing for Milestone 1

[RussellJimmies]

For Nachet-Interactive to be considered "in production" by the end Milestone 1 (2023Q4), a series of tests will have to be passed. I'm trying to cover all angles of the confusion matrix.

TRUE POSITIVES (TP)

TP TEST 1

Classify an image containing ONE of EACH seed which the model WAS trained to recognize.
Expected result: All seeds are identified correctly with high confidence score.

TP TEST 2

Classify a group photo of seeds, where some seeds are partially covered or overlapped by others.
Expected result: Visible portions of the seeds are correctly identified with high confidence score.

---

TRUE NEGATIVES (TN)

TN Test 1

Classify an image containing seeds which the model was NOT trained to recognize.
Expected result: Model offers guesses with low confidence score.

TN Test 2

Classify an image containing nothing.
Expected result: Model does not return results.

TN Test 3

Classify an image with complex patterns or textures that might mimic the appearance of seeds.
Expected result: Model does not return results.

---

FALSE POSITIVES (FP)

FP Test 1

Classify an image containing similar-looking seeds which the model is NOT trained to recognize.
Expected result: Identifies the seeds as being the seed it CAN recognize but gives it a low confidence score.

[RussellJimmies]

TP TEST 1

[RussellJimmies]

TP TEST 2

[RussellJimmies]

TN TEST 1

[RussellJimmies]

TN TEST 2

[RussellJimmies]

TN TEST 3
Note from Taran: "This is an image I had taken a while ago on the Tagarno so I don't have settings, though it does include the magnification; it's soil peds, which are not seeds but are similar size/shape lumps of soil that are often mixed in with seeds and need to be separated from them. Do you think this applies for this test?"

[RussellJimmies]

FP TEST 1
Note from Taran: "These are Bromus inermis or Bromus riparius. Not the two species of Bromus the model was trained on. "

[MaxenceGui]

Close issue since user-acceptance tests are scheduled for milestone 6.

[rngadam]

Close issue since user-acceptance tests are scheduled for milestone 6.

@MaxenceGui

This looks wrong - we're closing issues because they are scheduled in the future?

Each milestone has to be accepted by users.

And where do we keep these tests documented?

[MaxenceGui]

Close issue since user-acceptance tests are scheduled for milestone 6.

@MaxenceGui

This looks wrong - we're closing issues because they are scheduled in the future?

Each milestone has to be accepted by users.

And where do we keep these tests documented?

Might have been too fast here. I thought it was done and that user acceptance would be reworked in milestone 6.

I can discuss with William to see if we need to reopen the issue and come up with a way to record and keep a trace of the tests that have been done.

[rngadam]

@RussellJimmies