ReMindful

you used to recall me on my cell phone

From experiencing moments of forgetfulness to facing the challenges of dementia, the loss of cherished memories can bring unimaginable pain. For individuals with Alzheimer's and other conditions that affect memory, losing recollections of loved ones can feel like losing a part of their identity. We want to provide support for those experiencing memory loss, no matter how severe it is. Our goal is to offer a tool that can assist in memory recall and enhance cognitive abilities. Remindful is designed to complement existing dementia treatments (not replace them!!), providing a non-invasive, accessible, and enjoyable supplemental aid.

Features

Active Recall Memory Game:

Remindful engages users in an interactive game using active recall strategies and reminiscence therapy to improve cognitive skills. Users explore memories through pictures, starting with a partially obscured image that gradually unveils key details as they recall and describe elements.

Example-Based Memory Trigger:

Utilizes past events, like participation in UofTHacks, as memory triggers for users. Users mentally reconstruct scenes by recalling objects and activities related to the depicted event.

Voice-Activated Interaction:

Users speak into the microphone, making guesses and describing their memories aloud. The system responds to keywords, such as "laptop," revealing hidden objects and prompting users to continue piecing together the memory.

Scoring and Evaluation:

The game concludes with a comprehensive score based on factors like accuracy, completion time, and the ability to remember all key objects. Provides a structured feedback mechanism, encouraging users to reflect on their memory recall performance and track improvements over time.

Built with

• Flask backend
• Python, YoloV8 & Roboflow to train the model for object detection
• MongoDB to store images
• Web Speech API to transcribe your voice into text in real-time
• Typescript React frontend

How we trained the model

To train our object detection model using Roboflow and YOLOv8, we began by curating a diverse dataset representing various scenarios, including instances from UofTHacks and other comparable events. We fine-tuned YOLOv8's parameters and configurations using Jupyter Notebook for optimal performance in detecting and identifying objects crucial to the memory recall game in Remindful.

Architecture Overview

Getting Started

Prerequisites

1. Before you begin, ensure you have met the following requirements:

• Node.js and npm installed.
• Python and pip installed for the backend.

2. Install required dependencies in root folder and both frontend and backend folders

npm install

Starting the server

(127.0.0.1:5000 by default)

1. cd server
2. python3 -m venv venv
3. source venv/bin/activate (MacOS)
4. venv\Scripts\activate (Windows Powershell)
5. pip install -r requirements.txt
6. python3 app.py

Starting the app

(localhost:3000 by default)

1. cd client
2. npm install
3. npm start

Sneak Peak

Next Steps

• Stable diffusion to erase objects instead of blurring them out (to replicate Adobe's generative fill tool)
• Build a model that determines the "importance" of an item such as frequency, identifiability, user feedback, etc...
• Optimize for faster response times
• More accurate object detection models