Skip to content

Jyotibrat/BotMed

Repository files navigation

πŸ§‘β€βš•οΈ BotMed

Project Logo

A comprehensive AI-powered medical assistant that provides intelligent health consultations through multiple input modalities including text, voice, and medical image analysis.

πŸ† Hackathon Project

This project was developed for the VITB-JHU Health Hack, combining cutting-edge AI technologies to create an accessible and intelligent medical consultation platform.

πŸ“ Repository Structure

BotMed/
β”œβ”€β”€ .github/
β”‚   β”œβ”€β”€ Assets/
β”‚   β”‚   β”œβ”€β”€ Contributors/
β”‚   β”‚   β”‚   β”œβ”€β”€ Rana_Talukdar.png
β”‚   β”‚   β”‚   β”œβ”€β”€ Bindupautra_Jyotibrat.png
β”‚   β”‚   β”‚   β”œβ”€β”€ Arunim_Gogoi.png
β”‚   β”‚   β”‚   β”œβ”€β”€ Ansh_Gaur.jpg
β”‚   β”‚   β”‚   └── Akshit_Joshi.jpeg
β”‚   β”‚   β”‚
β”‚   β”‚   └── logo.png
β”‚   β”‚
β”‚   β”œβ”€β”€ ISSUE_TEMPLATES/
β”‚   β”‚   β”œβ”€β”€ bug_report.md
β”‚   β”‚   └── feature_request.md
β”‚   β”‚   
β”‚   β”œβ”€β”€ CODE_OF_CONDUCT.md
β”‚   └── READMD.md
β”‚
β”œβ”€β”€ Archive/
β”‚   β”œβ”€β”€ Notebooks/
β”‚   β”‚   β”œβ”€β”€ API.ipynb
β”‚   β”‚   β”œβ”€β”€ deepseek_fine_tune.ipynb
β”‚   β”‚   β”œβ”€β”€ Openai_Whisper_STT_with_NLP.ipynb
β”‚   β”‚   └── Testing.ipynb
β”‚   β”‚
β”‚   β”œβ”€β”€ Python Scripts/
β”‚   β”‚   β”œβ”€β”€ aud2text.py
β”‚   β”‚   β”œβ”€β”€ breath_heart_sound.py
β”‚   β”‚   β”œβ”€β”€ generate_audio.py
β”‚   β”‚   └── heart_breath_sound.py
β”‚   β”‚
β”‚   β”œβ”€β”€ Testing Data/
β”‚   β”‚   └── Audio Data/
β”‚   β”‚       β”œβ”€β”€ OPUS and MP3 Audio Files/
β”‚   β”‚       β”‚   β”œβ”€β”€ test1.opus
β”‚   β”‚       β”‚   β”œβ”€β”€ test2.mp3
β”‚   β”‚       β”‚   β”œβ”€β”€ test3.opus
β”‚   β”‚       β”‚   └── test4.opus
β”‚   β”‚       β”‚   
β”‚   β”‚       └── WAV Audio Files/
β”‚   β”‚           β”œβ”€β”€ test1.wav
β”‚   β”‚           β”œβ”€β”€ test2.wav
β”‚   β”‚           └── test3.wav
β”‚   β”‚       
β”‚   └── README.md
β”‚
β”œβ”€β”€ Backend/
β”‚   β”œβ”€β”€ Notebooks/
β”‚   β”‚   β”œβ”€β”€ Main Backend/
β”‚   β”‚   β”‚   β”œβ”€β”€ llama_8b_integrated_aud.ipynb
β”‚   β”‚   β”‚   β”œβ”€β”€ tumour_class.ipynb
β”‚   β”‚   β”‚   └── whisper_aud_text.ipynb
β”‚   β”‚   β”‚
β”‚   β”‚   └── Speech To Text/
β”‚   β”‚       └── Faster_Whisper_STT_with_NLP.ipynb
β”‚   β”‚
β”‚   └── README.md
β”‚
β”œβ”€β”€ public/
β”‚   └── favicon.ico
β”‚
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ App.tsx
β”‚   β”œβ”€β”€ index.css
β”‚   β”œβ”€β”€ main.tsx
β”‚   └── vite-env.d.ts
β”‚
β”œβ”€β”€ .gitattributes
β”œβ”€β”€ .gitignore
β”œβ”€β”€ package.json
β”œβ”€β”€ package-lock.json
β”œβ”€β”€ index.html
β”œβ”€β”€ eslint.config.js
β”œβ”€β”€ postcss.config.js
β”œβ”€β”€ tailwind.config.js
β”œβ”€β”€ tsconfig.app.json
β”œβ”€β”€ tsconfig.json
β”œβ”€β”€ tsconfig.node.json
└── LICENSE

πŸš€ Features

Multi-Modal Input Support

  • Text Input: Direct text-based medical queries
  • Voice Input: Audio-to-text conversion using OpenAI WhisperX
  • Image Analysis: Medical image classification for tumor detection

Advanced AI Capabilities

  • Fine-tuned LLaMA 7B: Specialized medical knowledge base
  • NLP Health Filter: Intelligent detection of health-related queries
  • Medical Image Classification: ML model for tumor type identification
  • Contextual Responses: Relevant and accurate medical information

πŸ›  Tech Stack

Backend

  • Python: Core backend development
  • LLaMA 7B: Fine-tuned large language model
  • OpenAI WhisperX: Audio-to-text conversion
  • Custom ML Models: Image classification and NLP filtering

Frontend

  • TypeScript: Type-safe development
  • React: Component-based UI framework
  • Vite: Fast build tool and development server
  • HTML5: Semantic markup
  • Tailwind CSS: Utility-first styling framework

πŸ”§ Installation

Prerequisites

Backend Setup

  • The Models must be run on google colab or kaggle for the development setup.
  • Clone the repository and navigate to the backend directory and then to Notebooks directory.
    git clone https://github.com/Jyotibrat/BotMed.git
    cd "backend/Notebooks"
  • Upload the Notebooks to the google colab or kaggle to run them (You can run them in your system also if you have a very good GPU).

Note: Currently the Backend is in development and the project is not fully deployed yet.

Frontend Setup

  • Clone the repository and navigate to the root directory.
    git clone https://github.com/Jyotibrat/BotMed.git
  • Install the node modules using npm (you can also download using other package manager like yarn).
    npm install
  • If the packages in needing funding.
    npm fund
  • If packages need to be fixed.
    npm audit fix
  • If the fixing of package still persists after running the above command then run this command.
    npm audit fix --force
  • Run the development server.
    npm run dev
  • Open your browser and navigate to http://localhost:5173/ to see the application.

Note: Check the frontend deployment here

πŸ§ͺ Model Performance

  • LLaMA 7B Fine-tuning: Achieved high accuracy on medical query responses
  • Image Classification: Demonstrates reliable tumor type detection
  • NLP Filter: Effectively distinguishes health-related vs. non-health queries
  • WhisperX: High-quality audio transcription with medical terminology support

🀝 Contributing

We welcome contributions to improve the BotMed! Please read our Code of Conduct before contributing.

Development Workflow

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

πŸ“œ License

This project is licensed under the GPL-3.0 License - see the LICENSE file for details.

⚠️ Disclaimer

Important: This medical chatbot is designed for educational and informational purposes only. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult with qualified healthcare professionals for medical concerns.

πŸ‘₯ Team

This project was made possible by the contributions of these amazing individuals:

πŸ™ Acknowledgments

  • VITB-JHU Health Hack organizers
  • OpenAI for WhisperX technology
  • Meta AI for LLaMA foundation model
  • Healthcare professionals who provided domain expertise

πŸ“§ Contact

For questions, suggestions, or collaboration opportunities, please reach out to us @here.

About

This is Project Repository of VIT Bhopal University - John Hopkins (VITB-JHU) Health Hack 2025

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Contributors 3

  •  
  •  
  •