API Latency· 2025
Chat-bot API
Multi-modal chatbot API built with Flask and Google Gemini, featuring speech recognition, image understanding, and context-aware conversations through a clean REST interface.
Technical case study by Rojit Pokharel — Full-Stack Web Developer & System Architect, Kathmandu, Nepal
FlaskGemini APIPythonREST API
Stack diagram
Application
FlaskGemini APIPythonREST API
Client / Problem
$
- ›Teams wanted a single API to power chatbots that accept text, speech, and images — not separate integrations per modality.
- ›Multi-modal interactions need a clean, consistent REST interface over the raw Gemini calls.
My Role
$
- ›Backend developer — built a Flask REST API wrapping Google Gemini for text, speech, and image inputs.
- ›Designed the request/response contract and prompt engineering for context-aware conversations.
Architecture
$
- ›Flask REST API with blueprint-based route organization.
- ›Gemini API integration for multimodal generation — text, audio, and vision inputs.
- ›Speech recognition layer converts audio to text before prompting.
- ›Conversation context passed across turns for coherent multi-turn dialogue.
API Architecture
$
- ›Clean REST endpoints for chat, speech-to-text, and image understanding.
- ›Centralized prompt builder keeps system instructions consistent across modalities.
Authentication
$
- ›API-key authentication guarding the chat endpoints.
Deployment
$
- ›Flask app behind Nginx with SSL; environment-managed Gemini API keys.
Performance Optimization
$
- ›Streamed responses where supported to keep perceived latency low.
- ›Request timeouts and retries around Gemini to handle transient API failures.
Problems Encountered
$
- ›Gemini latency varies with input modality, making consistent API response times hard.
- ›Raw model calls scattered through routes made prompts and keys hard to manage.
How I Solved Them
$
- ›Centralized Gemini client with streaming and timeout handling per modality.
- ›A single prompt-builder module standardizes system context and message history.
Results
$
- ›A production-ready multi-modal chatbot API handling text, speech, and image conversations through one REST interface.
Lessons Learned
$
- ›Multimodal apps need a clean API contract before the model calls.
- ›Centralize provider integration — it will be replaced or versioned.
Related Reading
Computer Vision
Building an AI Referee: Computer Vision for Football Player Detection and Tracking
Build an AI-powered football analytics pipeline using YOLOv5, SORT tracking, SSIM-based team classification, pitch mapping, and a Flask dashboard.
Answers
Read the Answers
Direct answers to questions about Rojit Pokharel and the technologies used across these projects.
Want a production system like this?
I build production web applications end-to-end. Let's discuss your project.
Get in Touch