ideas.txt

Possible Ideas

Cognitive Search
- AML skill (still in preview, requires invite)
- Debug sessions (still in preview, requires invite)
- Global scoring statistics (too much of an edge case)
- Okapi BM25 similarity ranking algorithm (will be the default, so not much to show)
- Create a custom skill to invoke Form Recognizer
- https://docs.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-form
- Invoice uploaded to blob storage | Synapse Pipeline Starts Search Indexer (Web Activity) -> Search indexer -> Form Recognizer -> Project to Knowledge Store (JSON in Blob) -> Synapse Pipeline loads latest product costs (Mapping Data Flow with Upsert like `ASA 400 - Write User Profile Data`)
- https://docs.microsoft.com/en-us/azure/search/search-indexer-overview#run-indexers-on-demand
- https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-sql-data-warehouse#sink-transformation
- https://docs.microsoft.com/en-us/azure/search/cognitive-search-concept-intro
- https://docs.microsoft.com/en-us/azure/search/knowledge-store-concept-intro
- https://docs.microsoft.com/en-us/azure/search/knowledge-store-projection-overview
- We have code that is close in https://github.com/solliancenet/tech-immersion-data-ai/tree/master/ai-exp2 


Forms Recognizer
- Process invoices 
- Invoke using Python, returns JSON document of key/value pairs extracted from document
- (Stretch Goal) show use of Sample Labeling Tool
- Get Products table and sample data from CDP deploy, add price data and cost column
- Invoice item in table includes product id so it matches to database
- Add a scenario where WWI wants to validate that their per product profitability is accurate by means of ingesting invoices from their suppliers, and updating the “cost” attribute in the product catalog as appropriate. This would be done in part using Forms Recognizer.
- https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-train-extract


MLOps
- Want to include some aspects of MLOps that are more important to Business owner (Very high level questions like how retrain models process looks like, how ML APIs integrate with Apps etc.). We can only describe these in white board session and not really implement in the Lab.
- (WDS only) Train a product seasonality classifier against the product catalog and transaction history, that can be used to augment the catalog with the product’s selling season. This experiment would be run in AML compute, tracked by AML experiment and model registered there. Model would be deployed as a web service in AKS (for production scenario) and ACI (for the Lab). This can be shown in two ways, using a manual training with XGBoost and by using AutoML.