wtf is drishti-ai?

Drishti AI is an assistive navigation system for visually impaired users that runs entirely inside a web browser, with no app to install, working from a laptop, an Android phone, or an iPhone. Instead of just spotting objects in front of the camera, it combines object detection, distance estimation, direction awareness, spoken narration in multiple languages, and a conversational AI that can answer follow up questions about the surroundings. The system works by streaming live camera frames from the browser to a backend server. There, a model called YOLOv8 identifies nearby objects such as cars, people, bicycles, and traffic signs, while a separate depth estimation model called MiDaS judges roughly how far away each object is, all from a single camera image with no extra sensors needed. Each detected object is placed into one of three zones, left, center, or right, and the closest or most relevant obstacle is announced first so the user is not overwhelmed with constant chatter about everything the camera sees. That spoken guidance is available in English, Hindi, Telugu, Tamil, Kannada, and Malayalam, using a text to speech model called Sarvam, with an automatic fallback to Google's text to speech service if Sarvam is unavailable. Beyond simple obstacle callouts, users can ask natural questions like whether there is a pedestrian crossing ahead, and a language model called Groq LLaMA 3 answers using a short scene summary generated by the vision models rather than raw images, which keeps responses fast and efficient. The frontend is built with React and the backend with FastAPI, with the backend deployed on Hugging Face Spaces and the frontend on Vercel. The project describes itself as production ready and aims to be more capable and more accessible than traditional assistive navigation hardware, which tends to be expensive, limited to basic obstacle detection, and hard to deploy across different devices. Drishti AI instead tries to build a continuous, spoken understanding of the user's surroundings that adjusts to obstacles, distance, and direction in something closer to real time.