Workflow process instance data unavailable in Indexer enrichment call

[subhashini-egov]

Steps to reproduce:

1. Service calls workflow transition for the first time
2. Service pushes data to Kafka topic
3. Indexer listens to Kafka topic and attempts enrichment of Workflow by calling search API

Issue:

Workflow search returns empty.

Reason:

Workflow persists its data in an async fashion. Before Workflow data is persisted to the DB, Indexer makes the call to the service looking for the data. This is a timing issue.

Fix:

1. Implement Redis caching in Workflow and return data from the cache if data is not present in DB. Cache invalidation should be implemented correctly to avoid stale data.
2. Make the workflow service synchronous and persist directly in DB instead of via persister. This will ensure all data is in the DB as soon as the transition call is made.

Additional Context:

1. This issue has been commonly observed in MUKTA, DRISTI (Pucar) and SELCO usage of the DIGIT platform.
2. This is not always reproducible. Certain services calling WF have this problem while some others work just fine.
3. As a workaround, services do the workflow enrichment in code and avoid doing it in Indexer. This leads to writing a lot of extra code in multiple places.

@talele08 @eGovMan @GhanshyamRawat-eGov