School of Medicine | University of St. Gallen

Case Introduction:

In our hackathon project, our objective is to tackle the undeniable influence of nutrition on holistic health. Streamlining the management of individual nutrition stands as a pivotal goal, especially in reaching out to vulnerable communities in a preventive context. Presently, the manual tracking of food consumption imposes a considerable challenge on users. Thus, our endeavor is focused on crafting a solution that mitigates this challenge while efficiently contributing to preventive healthcare efforts.

Expected Final Product:

The project's objective is to create a robust image detection algorithm integrated within a mobile application framework. This algorithm will analyze food images captured via smartphone cameras, supplemented by LIDAR data if available, to generate comprehensive outputs including calorie count, fat content, protein content, carbohydrate content, and other relevant nutritional information. Emphasis will be placed on ensuring the accuracy of these predicted outputs.

In addition to developing the algorithm, participants are tasked with conceptualizing an app, encompassing strategic planning and initial visualizations, as well as outlining an implementation strategy to effectively reach the target demographic. The target population may vary and can be defined by the team, encompassing healthy individuals, those at risk, youth, seniors, or individuals with specific health conditions (excluding diabetes).

Furthermore, elucidate how the app's design will prioritize user engagement, particularly over the long term. This entails detailing strategies to sustain user interest and involvement, potentially incorporating features such as personalized recommendations, interactive elements, gamification, and social connectivity to foster a lasting user-community relationship.

Use Case:

The ideas is to make food tracking attractive for users, motivating them to engage with the app consistently over time. The ultimate goal is to improve health and health literacy, making users aware of how their dietary impacts their overall health.

Deep Dive Slides:

Resources:

• Google food data set (https://github.com/google-research-datasets/Nutrition5k/tree/main?tab=readme-ov-file#download-data)
• Participants are welcome to use any other publicly available datasets that provide food images and data.
• Have a look at our website to get a sense of what we are doing and trying to achieve https://www.c4dhi.org
• use of python recommended
• Existing apps: january.ai, snack snap, Snaq, MyFitnessPal

Presentation:

Format:

• 5-minute pitch followed by 3 min Q&A
• Teams will have access to a TV screen
• Be creative! You can also show app functionality live.

Key Elements:

• Demonstration of functionaly of solution
• Presentation and explanation of outputs
• Description of strategy and concept

Judging Criteria:

• Innovation / Creativity (20%)
  • How does the design creatively increase user engagement, especially in the long term?
• Product design (20%)
  • Is the product design visually appealing?
• Feasibility / Technical assessment (20%)
  • Is the output accurate? How much do the predicted nutritional values vary from the ground truth?
• Potential impact / Reachability (20%)
  • How do the team's strategies maximize ist impact on the target population?
• Presentation (20%)
  • Communication of the developed solution

Point of Contact:

Nicolas Hesse, Marc Gruener or Giuliana Breu will be glad to answer your questions during the Deep Dive. We’ll also be available on Discord.