ā” Multimodal Chat Interfaces in Gradio
Modern AI applications go beyond text. By enabling multimodal=True in gr.ChatInterface, Gradio automatically adds file upload buttons (images, audio, documents) directly to the chat input bar, allowing users to converse with vision and audio-enabled AI models seamlessly.
Example 1: Definition & Basic Multimodal ChatInterface Setup
Enables built-in multimodal inputs supporting both text queries and attached image files.
Python (multi_ex1.py)
import gradio as gr
def multimodal_responder(message, history):
# message is a dictionary containing 'text' and 'files' when multimodal=True
text = message.get("text", "")
files = message.get("files", [])
file_info = f" with {len(files)} attached file(s)" if files else ""
return f"Received text: '{text}'{file_info}. Processed successfully by Nikita-Infotech AI assistant."
demo = gr.ChatInterface(
fn=multimodal_responder,
multimodal=True,
title="Nikita-Infotech Multimodal AI Assistant",
description="Upload images or documents along with your text prompt."
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Nikita-Infotech Multimodal AI Assistant
[Chat Feed: User prompt + attached image preview]
Example 2: Image Upload & Visual Analysis Chatbot
Inspects uploaded image dimensions and file names to provide visual metadata analysis in chat.
Python (multi_ex2.py)
import gradio as gr
from PIL import Image
def analyze_image_chat(message, history):
text = message.get("text", "Describe image")
files = message.get("files", [])
if not files:
return f"You asked: '{text}', but no image was attached."
img_path = files[0]
img = Image.open(img_path)
width, height = img.size
return f"š· **Image Analysis Result:**\n- **Filename:** {img_path}\n- **Dimensions:** {width} x {height} px\n- **User Query:** {text}"
demo = gr.ChatInterface(
fn=analyze_image_chat,
multimodal=True,
title="Visual Image Analysis Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Visual Image Analysis Chatbot
š· Image Analysis Result:
- Dimensions: 1920 x 1080 px
- User Query: Check photo quality
- Dimensions: 1920 x 1080 px
- User Query: Check photo quality
Example 3: Audio Transcription & Voice Chatbot Assistant
Accepts recorded audio clips or voice files uploaded through the multimodal chat attachment interface.
Python (multi_ex3.py)
import gradio as gr
def voice_chat_responder(message, history):
text = message.get("text", "")
files = message.get("files", [])
audio_note = ""
if files:
audio_note = f"\nšļø [Audio Attachment Detected: {files[0]}]"
return f"Processed voice message query: '{text}'{audio_note}\n\n**AI Response:** Voice transcription verified and answered successfully."
demo = gr.ChatInterface(
fn=voice_chat_responder,
multimodal=True,
title="Voice & Audio Multimodal Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Voice & Audio Multimodal Chatbot
šļø [Audio Attachment Detected: recording.wav]
AI Response: Voice transcription verified.
AI Response: Voice transcription verified.
Example 4: Document Attachment & PDF QA Chatbot
Parses uploaded text or PDF documents attached to chat messages to answer targeted document questions.
Python (multi_ex4.py)
import gradio as gr
def doc_qa_chat(message, history):
text = message.get("text", "")
files = message.get("files", [])
if not files:
return "Please attach a document (.txt or .pdf) to query its contents."
doc_file = files[0]
with open(doc_file, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
snippet = content[:300] + "..." if len(content) > 300 else content
return f"š **Document QA Summary:**\n- **File:** {doc_file}\n- **Preview:** {snippet}\n- **Answer to '{text}':** Document indexed and queried successfully."
demo = gr.ChatInterface(
fn=doc_qa_chat,
multimodal=True,
title="Document Attachment QA Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Document Attachment QA Chatbot
š Document QA Summary:
- File: curriculum.txt
- Answer: Document indexed successfully.
- File: curriculum.txt
- Answer: Document indexed successfully.
Example 5: Customizing Multimodal Chat UI Layout
Customizes chat bubble styling, placeholder text, and submission button labels for multimodal interfaces.
Python (multi_ex5.py)
import gradio as gr
def custom_chat(message, history):
return f"Customized Multimodal Response to: '{message.get('text', '')}'"
demo = gr.ChatInterface(
fn=custom_chat,
multimodal=True,
title="Customized Multimodal Portal",
textbox=gr.MultimodalTextbox(placeholder="Type your message or drop files here..."),
theme="soft"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Customized Multimodal Portal
Type your message or drop files here...
Example 6: Multi-Image Comparison Chatbot
Accepts multiple attached images simultaneously to compare visual data across user chat turns.
Python (multi_ex6.py)
import gradio as gr
def compare_images(message, history):
files = message.get("files", [])
text = message.get("text", "Compare")
return f"š¼ļø Comparing {len(files)} uploaded image(s) for query: '{text}'. Both images verified for comparative analysis."
demo = gr.ChatInterface(
fn=compare_images,
multimodal=True,
title="Multi-Image Comparison Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Multi-Image Comparison Chatbot
š¼ļø Comparing 2 uploaded image(s) for query: 'Compare designs'.
Example 7: Streaming Multimodal Responses (Typewriter Effect)
Combines multimodal file intake with progressive token-by-token streaming responses.
Python (multi_ex7.py)
import gradio as gr
import time
def stream_multimodal(message, history):
text = message.get("text", "")
files = message.get("files", [])
response = f"Streaming multimodal analysis for prompt '{text}' with {len(files)} file(s) attached..."
current = ""
for word in response.split():
current += word + " "
time.sleep(0.06)
yield current
demo = gr.ChatInterface(
fn=stream_multimodal,
multimodal=True,
title="Streaming Multimodal Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Streaming Multimodal Chatbot
Streaming multimodal analysis for prompt... [Streaming Active]
Example 8: Spreadsheet & CSV Data Chatbot Assistant
Uploads CSV spreadsheets directly into chat to execute instant data queries and row calculations.
Python (multi_ex8.py)
import gradio as gr
import pandas as pd
def spreadsheet_chat(message, history):
files = message.get("files", [])
text = message.get("text", "")
if not files:
return "Please attach a CSV spreadsheet file."
df = pd.read_csv(files[0])
summary = f"Dataset loaded: {len(df)} rows, {len(df.columns)} columns."
return f"š **Spreadsheet Analysis:**\n{summary}\n- **Query:** {text}\n- **Result:** Dataset processed successfully."
demo = gr.ChatInterface(
fn=spreadsheet_chat,
multimodal=True,
title="Spreadsheet CSV Chat Assistant"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Spreadsheet CSV Chat Assistant
š Spreadsheet Analysis:
Dataset loaded: 150 rows, 5 columns.
Dataset loaded: 150 rows, 5 columns.
Example 9: Code Snippet & Image Architecture Reviewer
Reviews uploaded architectural diagrams and source code snippets submitted through multimodal chat.
Python (multi_ex9.py)
import gradio as gr
def architecture_reviewer(message, history):
text = message.get("text", "Review")
files = message.get("files", [])
return f"šļø **Architecture Review:**\n- **Prompt:** {text}\n- **Attachments:** {len(files)} file(s) reviewed.\n- **Feedback:** Design patterns adhere to Nikita-Infotech full-stack standards."
demo = gr.ChatInterface(
fn=architecture_reviewer,
multimodal=True,
title="Architecture & Code Reviewer Chatbot"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
### Architecture & Code Reviewer Chatbot
šļø Architecture Review: Design patterns adhere to standards.
Example 10: Enterprise Multimodal Operations Hub (`gr.Blocks`)
Embeds a multimodal chat interface alongside auxiliary metrics and status panels inside an enterprise layout.
Python (multi_ex10.py)
import gradio as gr
with gr.Blocks() as demo:
gr.Markdown("# š Enterprise Multimodal AI Operations Hub")
with gr.Row():
status_box = gr.Textbox(label="System Status", value="Multimodal Engine Online š¢")
gr.ChatInterface(
fn=lambda msg, hist: f"Processed multimodal query: {msg.get('text', '')} with {len(msg.get('files', []))} files.",
multimodal=True,
title="Enterprise Chat Session"
)
if __name__ == "__main__":
demo.launch()
š„ļø Live Web UI Visual Output Preview
# š Enterprise Multimodal AI Operations Hub
System Status: Multimodal Engine Online š¢
[Interactive Multimodal ChatInterface Box]