Image

Nutritionist Generative AI Doctor using Gemini

Want to enhance your nutrition skills? The Nutritionist Generative AI Doctor, which employs the Gemini model, is here for you. With this project, one can understand dietary requirements, monitor calorie intake, and strive for a healthier lifestyle using Artificial Intelligence with ease. This project is appropriate for health-freaks as well as beginners. Now, let's look at the objective of this innovative project - how does it make it easier and smarter for you to achieve your nutrition targets!

Project Overview

The Nutritionist Generative AI Doctor is an advanced tool aimed at detecting meals and providing suitable diet recommendations. With the help of Google Gemini AI, this project is able to take any food image and extract calorie-based information. It also gives details on macronutrients and micronutrients from the meal image.

Be it for weight loss, dieting, understanding the food content of meals, or any other reason, this AI application makes it easier. It helps in computing the calorie values, evaluating the food quality, and offering appropriate diet suggestions. Furthermore, it prepares easy graphic reports in no time for health-conscious people. This project is very useful for health-conscious people, nutritionists, and fitness buffs.

And what’s great is, that you don’t even have to think about it. Just take a picture of your meal and the AI will do its magic. This project is designed to fit every person who wants to be healthier with the innovative technology in the present world.

Prerequisites

Let’s ensure you're all set before we start this exciting process of creating chatbots! Do not worry, it is all easy, but having some prerequisites in preparation will help everything to work efficiently.

  • Basic understanding of Python programming.
  • A Google Colab account to run the project.
  • Knowledge of how to use Google Drive for storing data.
  • Google Gemini API key to access the AI model.
  • Knowledge in PIL to handle food image uploads.
  • Internet connection for accessing online resources and APIs.

Approach

The Nutritionist Generative AI Doctor project follows a simple but effective approach to deliver personalized nutrition insights. First, it integrates the Google Gemini model. Which is set up in Google Colab using Python. Once an image of a meal is uploaded, the AI analyzes it by identifying the food items and breaking them down into calories, macronutrients, and micronutrients. The model calculates the total calorie count and also assesses the healthiness of the meal. This provides a percentage split of carbohydrates, proteins, fats, and more. By using Google’s advanced AI capabilities, the project creates an easy nutrition report within seconds. This makes the entire process seamless and user-friendly. With this step-by-step approach, anyone can track their dietary intake with precision, improving their health decisions effortlessly.

Workflow and Methodology

Workflow

This project is simple and easy to follow. That makes analyzing your meals quick and easy. Below is a breakdown of how this works step by step.

  • Setup Google Colab: First set up Google Colab then install the necessary libraries.
  • Get API Key: Obtain the Google API key to access the Gemini AI model.
  • Upload Meal Image: Upload meal image to the Colab environment.
  • Process Image: Using the Pillow library the image is processed to ensure compatibility.
  • Send to AI Model: Send the processed image to the Google Gemini model.
  • Analyze Food Items: The Gemini model identifies the food items and calculates the total calories and nutrients.
  • Generate Report: The model generates a detailed nutrition report, including macronutrient percentages.
  • View Results: Lastly, the report is displayed.

Methodology

  • Data Input & Image Processing: Users upload a food image. Then it is processed using the Pillow library to ensure compatibility with the AI model.
  • AI Model Integration: Google’s Gemini AI model identifies food items in the image and breaks them down into nutritional components.
  • Analysis: The AI calculates total calories and also a breakdown of macronutrients and micronutrients.
  • Report Generation: The model generates an easy report with suggestions for improvement if needed.

Data Collection and Preparation Workflow

  • Image Collection: The project uses food images collected from different sources
  • Preprocessing: Used the Pillow library to standardize the images in size and format.

Code Explanation

STEP 1:

Mounting Drive

This code shows how to connect your Google Drive account to a Colab workspace. It helps in accessing the files available in the user’s Google Drive by making it present in a particular folder (which is ‘/content/drive’).

# Mount Google Drive
from google.colab import drive
drive.mount('/content/drive/')

STEP 2:

This code installs the Google Generative AI library. The command ensures that the latest version of the library is installed and older versions are upgraded.

!pip install -q -U google-generativeai

This code imports all the essential libraries for handling files, formatting texts, and also interacting with the Google Generative AI tools. The code safely saves the API key in the user data of Google Colab. The to_markdown function is used for formatting text into markdown in order to present it properly in the colab environment.

import pathlib
import textwrap
import google.generativeai as genai
# Used to securely store your API key
from google.colab import userdata
from IPython.display import display
from IPython.display import Markdown
def to_markdown(text):
  text = text.replace('•', '  *')
  return Markdown(textwrap.indent(text, '> ', predicate=lambda _: True))

STEP 3:

This block of code securely gets the API key from Google Colab using userdata.get('api_key'). Next, assign it to the variable GOOGLE_API_KEY. After that, it configures the genai with the API key. This enables the project to make use of Google’s AI models for image analysis.

GOOGLE_API_KEY=userdata.get('api_key')
genai.configure(api_key=GOOGLE_API_KEY)

The following code creates loops to call to the entire models with the use of genai.list_models() These are made accessible by the Google Generative AI library. The code then checks for every model if the generateContent method is supported. If it does, then the name of the model is printed. This helps in identifying the possible AI models available for content generation purposes. This can be utilized in this project for evaluation or even writing text.
for m in genai.list_models():
  if 'generateContent' in m.supported_generation_methods:
    print(m.name)

STEP 4:

Initializing and Formatting Responses with Gemini Pro AI Model

This code initializes the Gemini Pro AI model by creating an instance of GenerativeModel with the name gemini-pro. This instance, assigned to model, represents the "Gemini Pro" model, which will be used for content generation or data analysis. A request is made to the Gemini Pro model to generate a reply to a specified question, and the %%time magic command is included to measure the response time. The output is saved in the response variable for future use. Additionally, the code uses the to_markdown method to format the AI model's response by converting response.text into a well-structured Markdown format, making it easier to read and present.

model = genai.GenerativeModel('gemini-pro')
%%time
response = model.generate_content("What are the essential macronutrients and micronutrients required for a balanced diet?")
to_markdown(response.text)

Again this code makes a request to the Gemini Pro model for a reply to the specified question. The to_markdown method is used to format the text response provided by the AI model. 

%%time
response = model.generate_content("How does the body utilize carbohydrates, fats, and proteins for energy?")
to_markdown(response.text)

STEP 5:

​​Initializing Gemini Pro Vision Model and Displaying Image

This code is for building the Gemini Pro Vision model. An initialization created an instance of GenerativeModel with the name gemini-pro'. This assigns ‘Gemini Pro Vision’, which is the model used for generating content, or analyzing data in later steps to ‘model’. Then imports the PIL.Image module from the Pillow library. This opens an image file located at the specified path. The image is stored in the variable "img". When it is called it displays the image in the output.

model = genai.GenerativeModel('gemini-pro-vision')
import PIL.Image
img = PIL.Image.open('/content/drive/MyDrive/image1.png')
img

Image Opening and Initial Analysis

This code sends the image to the Gemini Pro Vision model for content generation. The model processes the image input and returns a text response. The response text is then formatted using the to_markdown function, which converts it into Markdown format for display.

response = model.generate_content(img)
to_markdown(response.text)

Detailed Calories Breakdown

The prompt given to the AI includes the image analysis task, which asks for the total calorie count for the foods included in the image and the respective number of calories for each food item. In addition to that, the AI indicates whether the food is healthy or not and provides the ratios of nutrients available in the food itself. Using the argument stream=True allows one to get the response as it is being streamed. The response.resolve() serves to ensure that the complete response is accepted and finalized for display

response = model.generate_content([""" You are an expert in nutritionist where you need to see the food items from the image
               and calculate the total calories, also provide the details of every food items with calories intake
               is below format
               1. Item 1 - no of calories
               2. Item 2 - no of calories
               ----
               ----
            Finally you can also mention whether the food is healthy or not and also mention the percentage split of the ratio of carbohydrates, fats, fibers, sugar and other important things required in our diet """, img], stream=True)
response.resolve()
to_markdown(response.text)

This block sends the request to the AI to specifically calculate the number of calories in total in the given image. Then the AI processes it and sends the response. The response is streamed in real-time and finalized with response.resolve(). The result is formatted into Markdown for easy understanding.

response = model.generate_content(["How many total calories are in this image?", img], stream=True)
response.resolve() 
to_markdown(response.text)
response = model.generate_content(["Is the food in this image good for health?", img], stream=True)
response.resolve()
to_markdown(response.text)

Image Opening and Initial Analysis

This block first imports the PIL.Image module from the Pillow library. This opens an image file located at the specified path. The image is stored in the variable "img". When it is called it displays the image in the output. Then it sends the image to the Gemini Pro Vision model for response generation. The model processes the input and replies in texts. The replied message is formatted using the 'to_markdown' function. This function is used to convert the text into the Markdown format for display. 

img = PIL.Image.open('/content/drive/MyDrive/image2.png')
img

Generating and Formatting Content with Gemini Pro Vision Model

response = model.generate_content(img)
to_markdown(response.text)

Detailed Calories Breakdown

The prompt given to the AI includes the image analysis task, which asks for the total calorie count for the foods included in the image and the respective number of calories for each food item. In addition to that, the AI indicates whether the food is healthy or not and provides the ratios of nutrients available in the food itself. Using the argument stream=True allows one to get the response as it is being streamed. The response.resolve() serves to ensure that the complete response is accepted and finalized for display.

response = model.generate_content([""" You are an expert in nutritionist where you need to see the food items from the image
               and calculate the total calories, also provide the details of every food items with calories intake
               is below format
               1. Item 1 - no of calories
               2. Item 2 - no of calories
               ----
               ----
            Finally you can also mention whether the food is healthy or not and also mention the percentage split of the ratio of carbohydrates, fats, fibers, sugar and other important things required in our diet """, img], stream=True)
response.resolve()
to_markdown(response.text)

This block sends a request to the AI asking what food to eat everyday, and the AI processes it and sends the response. The response is streamed in real-time and finalized with response.resolve(). The result is formatted into Markdown for easy understanding.
response = model.generate_content(["Make a list of how many calories you should eat each day by age?", img], stream=True)
response.resolve()
to_markdown(response.text)
response = model.generate_content(["Tell me what food to eat every day?", img], stream=True)
response.resolve()
to_markdown(response.text)

Conclusion

In modern society, it is often difficult to track one's food intake. With the Nutritionist Generative AI Doctor tracking what you eat is no longer a problem. This project utilizes the Google Gemini Pro Vision system and does all the tasks for you. It helps out with all the nutritional research, all the way from caloric intake to complex elements.

You can just share a photo of your plate and the system will write the nutrition facts about the dish within a few seconds. It can help you design your menu, maintain optimal body weight or simply monitor your health. Can you imagine carrying an application, which acts as a smart nutritionist for you all the time?

Do you want to gain control over what and how much you eat? Then let this savvy, AI-integrated project help you shift to a balanced diet. You’ve got the power of AI right in your hands!

Challenges and Solution

Problem: Some errors may occur during installation or importing libraries.
Solution: Try to restart the colab if an error occurs or run pip install again. Besides that, don't forget to check your network connection.

Problem: Obtaining and Managing the Google API Key
Solution: Generating key can be challenging. Watch tutorials about how to register for the API, retrieve the API key, and how store it in Google Colab. Keep in mind that the key must be kept secret otherwise it will be used inappropriately.

Problem: Handling High-Resolution Image
Solution: In case users are dealing with high-resolution photographs, the processing times can be long. Optimize the images by either resizing them or compressing them before uploading. Also, use the Pillow library (PIL.Image) to process and handle images efficiently.

Problem: Understanding Nutrition Data Produced by AI.
Solution: Some may have difficulty decoding the nutrition values included in the reports generated by the AI. To counter this, use visual infographics with examples on what the AI number represents (e.g. how large is calorie count, how is the breakdown of nutrients done).

FAQ

Question 1: Can I upload any kind of images for analysis?
Answer: You can upload clear images of meals, food items, or dishes. With high-quality images that make for clear identification, the AI excels, in identifying and analyzing things like nutrition facts.

Question 2: How do I enhance the performance of the AI in terms of the accuracy of the analysis?
Answer: To enhance, pay attention to the images you are uploading and upload only quality images in good lighting with the food elements being distinctly visible. Do not upload any blurry or too dark images as it will be difficult for the AI to identify the food properly.

Question 3: Does someone with coding experience have to run this project?
Answer: While it helps if you know basic Python and Google Colab, once you set things up by following a clear step-by-step guide, the project can be configured.

Question 4: What API key do I need for this project?
Answer: To access the Gemini Pro Vision model, you’ll need a Google Generative AI API key. This key is what gives the project a leg to stand on in order to connect to the AI model to analyze images and generate content. You can get this key when you sign up for the Google Generative AI API.

Code Editor