Skip to content
<Rojit />
FPS: 30+· 2025

AI Referee: Football Analytics

Computer vision pipeline using YOLOv5 + SORT tracking for player detection, SSIM template matching for team classification, and pitch mapping for real-time football analytics with a Flask dashboard.

Technical case study by Rojit Pokharel — Full-Stack Web Developer & System Architect, Kathmandu, Nepal

PythonYOLOv5OpenCVSORTFlaskPyTorch

Stack diagram

Application
PythonFlask
AI/ML
YOLOv5OpenCVSORTPyTorch
Client / Problem
$
  • Football coaches and analysts need objective, real-time player data — positions, speed, and team identity — without manual video review.
  • Detecting and classifying players by team from broadcast footage is noisy: occlusion, jersey similarity, and fast motion break naive detectors.
My Role
$
  • Computer vision engineer — built the full YOLOv5 detection and SORT tracking pipeline with OpenCV frame processing and a Flask dashboard.
  • Designed the team-classification and pitch-mapping logic.
Architecture
$
  • YOLOv5 for player detection, SORT (Kalman filter + IoU) for multi-object tracking across frames.
  • SSIM template matching for team classification against known jersey templates.
  • Perspective transform for pitch mapping, with a Flask dashboard serving live analytics.
API Architecture
$
  • Flask backend streaming processed video and aggregated tracking stats to the dashboard.
Real-time Systems
$
  • Real-time video inference pipeline sustaining 30+ FPS with detection, tracking, and classification per frame.
Deployment
$
  • GPU-backed inference environment running the PyTorch pipeline, with the Flask dashboard served over HTTP.
Performance Optimization
$
  • Frame-by-frame YOLOv5 inference optimized to sustain real-time rates.
  • SORT tracking keeps player identities stable between detections, reducing per-frame compute.
Problems Encountered
$
  • Occlusion and similar jersey colors confuse both detection and classification.
  • Tracking IDs must remain stable when players cross paths or leave frame briefly.
How I Solved Them
$
  • Combined detection with SORT tracking so identity persists through brief occlusions.
  • SSIM template matching on player crops assigns team affiliation before downstream analytics.
Results
$
  • End-to-end AI football analytics pipeline: detect, track, classify, and map players in real time.
  • Demonstrates production-grade computer vision with YOLOv5, OpenCV, SORT, and PyTorch.
Lessons Learned
$
  • Tracking is the bridge that turns detection frames into useful sports analytics.
  • Template-based classification is robust when lighting and jerseys are controlled.

Want a production system like this?

I build production web applications end-to-end. Let's discuss your project.

Get in Touch