KENNESAW, Ga. | Nov 25, 2024

A simple “food selfie” might hold critical nutritional data, and a smartphone application created by Kennesaw State University students may be the key to delivering that information as actionable insights to people monitoring their eating habits.

KSU graduate student Afnan Ahmed Crystal has joined assistant professor of information technology Maria Valero and doctoral student El Arbi Belfarsi on The Nutrilyzer, an app capable of extracting nutritional data from a picture taken with a smartphone. Similar to Valero’s revolutionary GlucoCheck device, which is a non-invasive blood glucose monitoring system, the app would allow people to better monitor their day-to-day eating habits through artificial intelligence (AI). 

“We are trying to build an app to complement the GlucoCheck device, allowing users to input relevant information, take a simple picture with their smartphone, and obtain an estimate of the nutritional content using AI. This estimate will be used to analyze the impact on blood glucose in tandem with the GlucoCheck device. We believe the app could empower people to make better decisions about nutrition.” said Crystal, who will graduate next month with her Master of Science in Computer Science.  

A key component of the application is its use of AI. That’s where Belfarsi’s expertise in large language models (LLMs) comes into play. Originally employed as a data analyst at a trading firm, he developed an interest in LLMs and sought to apply his expertise to health informatics systems. He saw the potential for AI to help people make better decisions about nutrition. 

“Large language models are primarily designed for processing textual data,” said Belfarsi, who is pursuing a Ph.D. in Computer Science. “When dealing with images, they rely on additional layers like the Contrastive Language–Image Pretraining (CLIP) model, which is trained to match images with their textual descriptions. Therefore, when prompting for nutritional data, we are effectively leveraging external computer vision models that complement the LLM's capabilities.” 

Valero credits the two students with combining their expertise and addressing potential pitfalls, such as regional cuisines that could potentially generate ethnic bias in the delivery of accurate information. The app has also been made to adjust feedback according to portion sizes. It’s all part of the Internet of Things as a Service Research Group's goal of helping people live healthier lives through applied technology.  

“Next, we're going to give the application to people, ask them to use our continuous glucose monitor, and collect data,” Valero said. “With that data, we will be able to do a model for an individual prediction of how your glucose is going to behave. Beyond that, the app can prescribe other habits like exercise to mitigate lifestyles and other factors.”

The project does not intend to replace the role of a dietician or nutritionist, but to augment the human element with a useful app that anyone can use. Future versions of the app might consider exercise habits chronicled through wearable technology like Fitbits or Apple Watches.

“The main goal is to benchmark whether the LLMs can outperform a dietician's evaluation, or vice versa,” Crystal said. “The second one is to leverage AI for better nutrition decision-making and improve people’s lives. We plan to integrate other wearable devices and push personalized data from these wearables to AI to improve its responses.” 

The Nutrilyzer is in an early testing stage with a possible launch this spring.

Story by Dave Shelles

Photo by Matt Yung

A leader in innovative teaching and learning, Kennesaw State University offers undergraduate, graduate and doctoral degrees to its more than 47,000 students. Kennesaw State is a member of the University System of Georgia with 11 academic colleges. The university’s vibrant campus culture, diverse population, strong global ties and entrepreneurial spirit draw students from throughout the country and the world. Kennesaw State is a Carnegie-designated doctoral research institution (R2), placing it among an elite group of only 7 percent of U.S. colleges and universities with an R1 or R2 status. For more information, visit kennesaw.edu.