Skip to content
<Rojit />
API Latency· 2025

Chat-bot API

Multi-modal chatbot API built with Flask and Google Gemini, featuring speech recognition, image understanding, and context-aware conversations through a clean REST interface.

Technical case study by Rojit Pokharel — Full-Stack Web Developer & System Architect, Kathmandu, Nepal

FlaskGemini APIPythonREST API

Stack diagram

Application
FlaskGemini APIPythonREST API
Client / Problem
$
  • Teams wanted a single API to power chatbots that accept text, speech, and images — not separate integrations per modality.
  • Multi-modal interactions need a clean, consistent REST interface over the raw Gemini calls.
My Role
$
  • Backend developer — built a Flask REST API wrapping Google Gemini for text, speech, and image inputs.
  • Designed the request/response contract and prompt engineering for context-aware conversations.
Architecture
$
  • Flask REST API with blueprint-based route organization.
  • Gemini API integration for multimodal generation — text, audio, and vision inputs.
  • Speech recognition layer converts audio to text before prompting.
  • Conversation context passed across turns for coherent multi-turn dialogue.
API Architecture
$
  • Clean REST endpoints for chat, speech-to-text, and image understanding.
  • Centralized prompt builder keeps system instructions consistent across modalities.
Authentication
$
  • API-key authentication guarding the chat endpoints.
Deployment
$
  • Flask app behind Nginx with SSL; environment-managed Gemini API keys.
Performance Optimization
$
  • Streamed responses where supported to keep perceived latency low.
  • Request timeouts and retries around Gemini to handle transient API failures.
Problems Encountered
$
  • Gemini latency varies with input modality, making consistent API response times hard.
  • Raw model calls scattered through routes made prompts and keys hard to manage.
How I Solved Them
$
  • Centralized Gemini client with streaming and timeout handling per modality.
  • A single prompt-builder module standardizes system context and message history.
Results
$
  • A production-ready multi-modal chatbot API handling text, speech, and image conversations through one REST interface.
Lessons Learned
$
  • Multimodal apps need a clean API contract before the model calls.
  • Centralize provider integration — it will be replaced or versioned.

Want a production system like this?

I build production web applications end-to-end. Let's discuss your project.

Get in Touch