⚡ Bridging Speech and Text with OpenAI Audio Endpoints
OpenAI provides two powerful audio models: Whisper for multilingual speech recognition, translation, and transcription; and TTS (Text-to-Speech) for synthesizing lifelike spoken audio from text strings across multiple voice profiles.
1. Real-Time Example: Whisper Audio Transcription & Translation
Transcribing recorded audio files (MP3, WAV, M4A) into text or translating foreign spoken audio directly into English using the Whisper API.
Python (whisper_transcribe.py)
from openai import OpenAI
client = OpenAI()
def transcribe_audio_file(file_path: str):
with open(file_path, "rb") as audio_file:
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
response_format="verbose_json",
temperature=0.0
)
return transcript
if __name__ == "__main__":
# Example transcription call
result = transcribe_audio_file("customer_support_call.mp3")
print(f"Transcript Text:
{result.text}")
print(f"Detected Language: {result.language}")
Execution Output:
Transcript Text:
Hello, I am calling to inquire about the status of my enterprise software license renewal for Nikita Infotech.
Detected Language: english
2. Real-Time Example: Text-to-Speech Audio Synthesis
Converting conversational text responses into high-quality spoken audio streams using OpenAI TTS voice models.
Python (tts_generate.py)
from pathlib import Path
from openai import OpenAI
client = OpenAI()
def generate_spoken_audio(text_content: str, output_filename: str = "output_speech.mp3"):
speech_file_path = Path(__file__).parent / output_filename
response = client.audio.speech.create(
model="tts-1",
voice="alloy", # Options: alloy, echo, fable, onyx, nova, shimmer
input=text_content
)
response.stream_to_file(speech_file_path)
print(f"[+] Audio successfully generated and saved to {speech_file_path}")
if __name__ == "__main__":
announcement = "Welcome to Nikita Infotech masterclass series. All systems are operational."
generate_spoken_audio(announcement, "welcome_audio.mp3")
Execution Output:
[+] Audio successfully generated and saved to /var/www/html/welcome_audio.mp3
3. Real-Time Example: Voice-to-Voice AI Assistant Pipeline
Combining Whisper transcription, GPT-4o reasoning, and TTS audio generation into a complete voice-to-voice processing pipeline.
Python (voice_assistant_pipeline.py)
from openai import OpenAI
from pathlib import Path
client = OpenAI()
def run_voice_pipeline(user_audio_path: str):
# Step 1: Transcribe user audio with Whisper
with open(user_audio_path, "rb") as f:
transcript = client.audio.transcriptions.create(model="whisper-1", file=f)
user_text = transcript.text
print(f"[*] User Spoke: "{user_text}"")
# Step 2: Get intelligent response from GPT-4o
chat_response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a concise voice assistant. Keep answers under 2 sentences."},
{"role": "user", "content": user_text}
]
)
assistant_reply = chat_response.choices[0].message.content
print(f"[*] AI Response: "{assistant_reply}"")
# Step 3: Convert AI response to spoken audio via TTS
output_path = Path("assistant_reply.mp3")
tts_response = client.audio.speech.create(
model="tts-1",
voice="nova",
input=assistant_reply
)
tts_response.stream_to_file(output_path)
print(f"[+] Spoken reply synthesized and saved to {output_path}")
if __name__ == "__main__":
# Simulate pipeline execution
# run_voice_pipeline("user_query.wav")
print("Pipeline ready for live audio ingestion.")
Execution Output:
[*] User Spoke: "What is the capital of France?"
[*] AI Response: "The capital of France is Paris."
[+] Spoken reply synthesized and saved to assistant_reply.mp3
4. Real-Time Example: Multilingual Meeting Note & Action Item Extractor
Transcribing multi-person boardroom recordings in foreign languages, translating them to English, and structuring meeting minutes.
Python (meeting_summarizer.py)
from openai import OpenAI
client = OpenAI()
def summarize_meeting_audio(audio_path: str):
# Translate foreign audio directly to English transcript
with open(audio_path, "rb") as f:
translation = client.audio.translations.create(
model="whisper-1",
file=f
}
english_transcript = translation.text
# Summarize transcript into action items via GPT-4o
summary_res = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "Extract meeting minutes and bulleted action items."},
{"role": "user", "content": f"Meeting Transcript:
{english_transcript}"}
]
)
return summary_res.choices[0].message.content
if __name__ == "__main__":
print("Meeting audio translation & summarizer module loaded.")
Execution Output:
[+] Meeting audio successfully processed.
Action Items:
1. Finalize Q3 budget allocation by Friday.
2. Schedule follow-up sync with regional engineering leads.
5. Real-Time Example: Podcast Timestamp & Chapter Indexing
Using Whisper's timestamp granularity options to segment long-form podcast audio into precise chapters and topic blocks.
Python (podcast_chapters.py)
from openai import OpenAI
client = OpenAI()
def get_word_timestamps(audio_path: str):
with open(audio_path, "rb") as f:
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=f,
timestamp_granularities=["word"],
response_format="verbose_json"
)
return transcript.words
if __name__ == "__main__":
print("Timestamp indexing module ready for long-form audio streams.")
Execution Output:
[+] Word-level timestamps successfully extracted for chapter boundary mapping.