From 6d1f39e0ce28a84e8e6adf9f2177a0f90204f251 Mon Sep 17 00:00:00 2001
From: bhargavbh <2bhargav5@gmail.com>
Date: Tue, 28 Nov 2023 11:57:44 +0100
Subject: [PATCH] draft of the evaluation report
---
evaluations/ConformanceTestSuite_1_Bhargav.md | 25 +++++++++++++++++++
1 file changed, 25 insertions(+)
create mode 100644 evaluations/ConformanceTestSuite_1_Bhargav.md
diff --git a/evaluations/ConformanceTestSuite_1_Bhargav.md b/evaluations/ConformanceTestSuite_1_Bhargav.md
new file mode 100644
index 000000000..909b24982
--- /dev/null
+++ b/evaluations/ConformanceTestSuite_1_Bhargav.md
@@ -0,0 +1,25 @@
+# Evaluation
+
+- **Status:** Accepted
+- **Application Document:** https://github.com/w3f/Grants-Program/blob/master/applications/Polkadot-Protocol-Conformance-Tests.md
+- **Milestone:** 1
+
+| Number | Deliverable | Accepted | Link | Evaluation Notes |
+| ------ | ----------- | -------- | ---- |----------------- |
+| 0.a | License |
|[Included](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/LICENSE)| Apache License 2.0
+| 0.b | Documentation || N/A | Documentation not applicable as it is a feasibility research grant.
+| 0.c | Testing and Testing Guide || [Tests](https://github.com/LimeChain/polkadot-conformance/tree/7c89d26c7221324474f0b7ab8fddeea2ecea440f/tests)| Tests categorised for Kagome and Polkadot along with a script to run them. Snippets included in the research document where necessary.
+| 0.d | Docker File|| Not Provided | Dockerfile not provided but also not a strict requirement.
+| 0.e | Article || [Notion Article](https://limechain.notion.site/Host-Conformance-Testing-Suite-using-Zombienet-Research-05eb42de213a40c8916d3d25735b9a58)| Article with plausible approaches and suggestions.
+| 1. | Host API Research ||[Host API](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md#trie-host-api-research)| Delivered. Explained with test-snippets of state-root computation and hash functions as examples.
+| 2. | SCALE Research ||[SCALE](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md#scale-research)| Delivered. Suggestion to use RPCs as these are mostly unit tests of pure functions.
+| 3. | State Trie Research ||[State Trie](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md#state-trie-research)| Delivered. Leads to the pre-requisite of having a tool that can set a required state before running the tets across multiple host implementations.
+| 4. | BABE Research || [BABE](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md#babe-research)| Delivered. Suggest using log-based approach on Zombienet; Illustrated with simple tests on completeness of secondary slot assignments.
+| 5. | GRANDPA Research ||[GRANDPA](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md#grandpa-research)| Delivered. Similar findings as BABE.
+| 6. | Overall Research Findings ||[Overall Report](https://github.com/LimeChain/polkadot-conformance/blob/7c89d26c7221324474f0b7ab8fddeea2ecea440f/docs/host-conformance-using-zombienet-research.md)| Delivered.
+
+
+## General Remarks
+The feasibility research has been performed in sufficient detail with clear findings and suggested approaches for each of the sub-components. The report is well structured and is a good starting point for teams to design a conformance test-suite. There are no surprising outcomes of the feasibility study. The suggestion of using conformance RPCs for Host API/ SCALE/ State-Trie and Zombienet based assertions and logs with the more involved properties for BABE/ GRANDPA is in line with intuition of Zombienet being useful for integration-tests. The grant also proposes pre-requisite tools for the test-suite, most notably, the tool for setting required state before running the tests. The design of this tool is outside the scope of this grant but a high-level set of requirements have been described.
+The team updated the progress of the project with the Spec Team (W3F) on a bi-weekly basis and has also incorporated suggestions/feedback both in their approach and documentation. Just for records, an earlier [draft](https://limechain.notion.site/Host-Conformance-Testing-Suite-using-Zombienet-Research-05eb42de213a40c8916d3d25735b9a58) of this deliverable was shared with the Spec team and feedback regarding a new section on implications for Host Implementers and more nuanced tests for BABE/GRANDPA have been addressed.
+Overall, milestones outlined in the application have been satisfactorily delivered.