Maitri: AI for Early Mental Health Detection

Maitri is a privacy-preserving, multimodal AI-powered web application designed for the AI for Early Mental Health Detection hackathon. It acts as a personal mental wellness companion, empowering users to understand, track, and improve their emotional well-being through analysis of text, audio, and video cues.

⚠️ Disclaimer: Maitri is not a diagnostic tool. It provides early insights and awareness, encouraging users to seek professional help if distress indicators persist.

🧠 Overview

Maitri integrates multiple data sources and modern AI technologies to detect early indicators of stress, anxiety, and depression. It uses transformer-based models, facial and vocal emotion recognition, and explainable AI (XAI) to provide users with clear, personalized emotional insights—without compromising privacy.

🚀 Core Features

1. 🧩 Multi-Modal Emotion Analysis

Maitri combines insights from three key modalities for holistic emotional evaluation:

🎥 Video Sentiment Analysis

• Users can record short video journals.
• Facial emotions are analyzed using Vision Transformer (ViT) and Facenet-PyTorch, identifying emotions such as joy, sadness, anger, and neutrality.
• Facial features are extracted through MTCNN and processed in real-time using OpenCV.

🎙️ Voice Emotion Analysis

• Voice diaries are processed via Wav2Vec2 and SpeechBrain to evaluate tone, pitch, and speech patterns.
• Detects markers of stress, sadness, or calmness from vocal cues.

📝 Intelligent Journaling

• A secure journaling space for daily reflections and emotional check-ins.
• Text is processed using SpaCy for PII removal and Transformers (DepRoBERTa & DistilRoBERTa) for emotion and depression likelihood detection.
• Optionally, users can connect social media posts for extended sentiment tracking.

🧩 Daily Mental Health Quiz

• Guided daily quiz sourced from health_quiz.json, inspired by validated psychological screening frameworks.
• Adds quantitative data to the daily emotional trend.

2. 📊 Comprehensive Wellness Dashboard

All emotional data is aggregated and visualized in an interactive, user-friendly dashboard built with Recharts.

Three Core Scores:

• Stress
• Anxiety
• Depression

Daily Emotion Index (DEI): Combines text, voice, and video analyses into a single metric.

Visual Trends: Weekly and monthly graphs help users recognize emotional patterns, progress, and triggers over time.

3. 💬 "Talk with Maitri" — Empathetic AI Chatbot

• Built with the Groq API for lightning-fast LLM inference.
• Offers a safe, empathetic space for users to express emotions freely.
• Provides personalized coping suggestions, self-care reminders, and grounding exercises.

4. 🧘 Panic Button & Calming Zone

A dedicated safety feature designed for moments of acute distress.

Activates a guided calming session featuring:

• Breathing exercises
• Soothing background audio

5. 🎮 Gamification & Progress Tracking

To encourage consistent mental health engagement:

• Activity Heatmap: Tracks journaling, video uploads, and quiz completions (GitHub-style contribution graph).
• Badges & Rewards: Recognizes milestones and consistency to reinforce self-care habits.

6. 🧭 Personalized Insights & Proactive Alerts

• Tailored recommendations based on user trends.
• If elevated distress indicators persist, Maitri suggests professional consultation or evidence-based coping strategies.

7. 🧩 Explainable AI (XAI)

To promote transparency and trust:

• EigenCAM visualizations highlight facial regions influencing emotion predictions.
• Helps users understand how the AI interprets their data.

🧰 Technology Stack

Maitri integrates a modern, AI-first, privacy-centric stack across frontend, backend, and ML pipelines.

🖥️ Frontend

Component	Technology
Framework	React (Vite)
Styling	Tailwind CSS
Routing	React Router
State Management	Zustand
Charts	Recharts
Animations	Rive (@rive-app/react-canvas)
Data Fetching	Axios

⚙️ Backend

Component	Technology
Framework	FastAPI
Server	Uvicorn
Database	MongoDB (via motor and pymongo)
Authentication	JWT (python-jose[cryptography]) & bcrypt (passlib[bcrypt])
Environment Config	python-dotenv
Validation	Pydantic

🧠 AI & Machine Learning

Task	Library / Model
NLP Sentiment & Emotion Detection	SpaCy, Transformers (DepRoBERTa, DistilRoBERTa)
Voice Emotion Recognition	Torchaudio, SpeechBrain, Wav2Vec2
Video/Facial Emotion Detection	Facenet-PyTorch, Vision Transformer (ViT), OpenCV, Pillow
Explainable AI	EigenCAM
Chatbot	Groq API
Media Processing	FFmpeg

🧩 Advanced AI Workflow

User Input Collection

Users can record a video, speak an audio journal, or write text entries.

Privacy-First Preprocessing

• Text Anonymization: PII removed with SpaCy NER before analysis.
• Face Cropping: MTCNN extracts standardized facial regions.
• Audio Extraction: FFmpeg converts video to .wav for cleaner analysis.

Multimodal Inference

• ViT → Emotion from facial data
• Wav2Vec2 → Emotion from voice
• DepRoBERTa / DistilRoBERTa → Emotion and depression likelihood from text

Score Aggregation

All predictions are combined into the Daily Emotion Index (DEI), generating stress, anxiety, and depression scores.

Visualization & Feedback

The data is sent to the frontend for dashboard visualization and chatbot personalization.

📂 Project Structure

Maitri/
├── backend/                           # FastAPI server, auth, AI pipelines
│   ├── main.py                       # App entrypoint
│   ├── auth.py                       # JWT authentication
│   ├── database.py                   # MongoDB connection
│   ├── pipeline.py                   # Multimodal data processing
│   ├── emotion_detector.py           # ViT/Facenet emotion inference
│   ├── emotional_support_chatbot.py  # Groq chatbot logic
│   └── requirements.txt              # Backend dependencies
│
├── frontend/                         # React web app
│   ├── src/
│   │   ├── pages/                   # Dashboard, Chat, Calming Zone, etc.
│   │   ├── components/              # Charts, Modals, Widgets
│   │   ├── context/                 # Auth & state management
│   │   └── main.jsx                 # React app entry
│   └── package.json                 # Frontend dependencies
│
├── audio/                           # Audio processing scripts (Wav2Vec2)
├── video/                           # Video analysis scripts (ViT + MTCNN)
├── notebooks/                       # Model training & R&D notebooks
├── quiz/                            # Daily quiz data
│   └── health_quiz.json
└── requirements.txt                 # Root ML dependencies

⚙️ Installation & Setup

🧩 Prerequisites

• Python 3.10+
• Node.js & npm
• MongoDB (local or cloud)

🧠 Backend Setup

git clone https://github.com/ritovan03/Maitri.git
cd Maitri
python -m venv venv
source venv/bin/activate # (or venv\Scripts\activate on Windows)
pip install -r requirements.txt

Create a .env file inside backend/:

MONGO_DETAILS="<your_mongodb_uri>"
SECRET_KEY="<your_jwt_secret>"
ALGORITHM="HS256"
GROQ_API_KEY="<your_groq_api_key>"

Run the backend:

cd backend
uvicorn main:app --reload

💻 Frontend Setup

cd frontend
npm install
npm run dev

Open your browser at: 👉 http://localhost:5173

⚖️ Ethical Framework & Privacy Commitment

Maitri is founded on a "Privacy-First, Empathy-Driven" philosophy.

🔒 Privacy Protection

• No raw data (text, audio, video) is stored.
• Only anonymized emotion scores are saved in MongoDB.
• PII removal is handled in-memory using SpaCy NER.

🧍 Informed Consent

• Users can choose which modalities (text, voice, video) to use.
• Data deletion on demand is supported.

🔍 Transparency & Explainability

• EigenCAM visualizations show which facial features influenced emotion detection.
• Clear explanations foster user trust and demystify AI processes.

⚖️ Bias Mitigation

• Models trained on diverse datasets to minimize gender, cultural, and linguistic bias.
• Periodic audits ensure fairness and inclusivity.

❤️ Ethical Use

Maitri strictly adheres to ethical AI standards:

• It does not diagnose mental disorders.
• It promotes early awareness, emotional literacy, and professional engagement.

🌟 Summary

Maitri combines cutting-edge AI models, empathetic design, and ethical safeguards to create an early mental health detection platform that empowers users while protecting their privacy.

It’s more than just an app—it’s a compassionate companion built to understand emotions, promote well-being, and support a healthier, more self-aware future. 💚