SnapCalorie employs artificial intelligence (AI) to estimate the number of calories in food based on photographs.
During his tenure at Google, Wade Norris aimed to develop a project that could have a positive impact on people’s lives. He co-founded Google Lens, an app powered by computer vision that provides information about identified objects. However, he still felt unsatisfied.
A few years ago, Norris joined forces with Scott Baron, a systems engineer in the aerospace industry, to establish SnapCalorie, a health-focused start-up. Powered by AI, SnapCalorie endeavours to accurately determine the caloric count and macronutrient breakdown of a meal by analysing a single photo taken with a smartphone.
SnapCalorie has recently raised a funding of $2 million from notable investors including Accel, Index Ventures, former CrossFit CEO Eric Roza, and Y Combinator. In a pre-seed round, the company had previously raised $125,000 from unidentified investors.
“Human beings struggle to visually estimate the portion size of a plate of food,” noted Norris. “SnapCalorie surpasses the existing methods by combining various new technologies and algorithms.”
To clarify, SnapCalorie is not the first computer vision-based app for counting calories. Applications such as Calorie Mama, Lose It, Food advisor, and Bite.AI have all made efforts to accomplish this goal, although their levels of success have varied. Nevertheless, Norris asserts that what sets SnapCalorie apart is its utilization of depth sensors on supported devices to measure portion size and a team of human reviewers that contribute an additional layer of quality.
“On average, our team is able to reduce caloric errors to less than 20%,” added Norris. “While other apps employ AI for photo-based meal tracking, none of them assist with portion size estimation, which is the most crucial aspect for minimizing errors.”
The health industry holds a considerable amount of scepticism regarding photo-based calorie estimating tools, and rightfully so. In a 2020 study comparing some of the most popular AI-based calorie counters, it was found that the most accurate one, Calorie Mama, was correct only about 63% of the time.
So, what sets SnapCalorie apart in terms of improvement? In addition to depth sensors and human reviewers, Norris highlights the company’s developed algorithm that can ostensibly outperform an individual in estimating a food’s caloric content. By employing this algorithm, SnapCalorie analyzes the photo to identify the different types of food present and gauges the portion size of each item in order to estimate its caloric content.
The results can be recorded in SnapCalorie’s food journal or exported to fitness-tracking platforms such as Apple Health.
The algorithm’s strong performance is attributed to its unique training dataset of 5,000 meals, according to Norris. SnapCalorie created this dataset by capturing thousands of photos of each meal, including soups, burritos, oils, “mystery sauces,” and more, using a robotic rig.
“We ensured that these photos encompassed the diverse and challenging conditions one encounters in the real world, and we weighed every single ingredient on a scale,” explained Norris. “The conventional method of training an AI model involves downloading public web images, having people label them, and then training the model to predict those labels. However, this approach is not feasible for food because people are highly inaccurate at visually estimating portion sizes, thus preventing them from labelling the images afterward.”
Norris acknowledges that SnapCalorie’s algorithm may exhibit bias toward American food due to the team collecting most of the initial training data in the United States. However, the company is currently expanding its training data by incorporating photos from SnapCalorie users and internal data, encompassing various cultural cuisines.
It can be argued that no matter how accurate the algorithm, no app can provide a truly precise account of the calories consumed in a meal. After all, there are numerous variables that these apps fail to consider, such as different cooking methods and the amount of time required for individual foods to break down.
Norris does not claim that SnapCalorie is 100% accurate. Instead, he suggests that the app’s calorie estimating tools should be viewed as part of the broader nutrition puzzle. Norris emphasized another major feature of SnapCalorie—a ChatGPT-powered chatbot that offers meal suggestions based on a user’s goals and preferences—as well as the app’s database of nutritional values.
“We’ve observed an increasing interest among people in understanding what they consume. The detrimental health effects of processed foods are becoming increasingly evident each day,” commented Norris. “Our users particularly appreciate SnapCalorie, especially when dining out, as many restaurants do not provide nutrition values, leaving consumers with no means of logging their meals.”
Regarding its popularity, SnapCalorie appears to be experiencing healthy growth, with nearly 1,000 new users expected to join this month. The company’s current focus is on expansion rather than monetization, although Norris described the burn rate as “very conservative.”
