Tired of long doctor’s visits? Introducing MedScribe! Answer your post-visit questions in an instant!

MedScribe revolutionizes the patient experience by enabling patients to chat with recordings of their doctor visits. This allows patients to fully process a doctor’s instructions with additional medical knowledge, takes some of burden off of the doctor’s shoulders, and breaks down geographical barriers for rural patients seeking healthcare. As a real doctor who sees, hears, and reads, we utilize the power of multimodal large language models and audio embeddings to process text databases, audio recordings, and image inputs from a patient’s visit to get the full picture of a patient’s condition.

Simply upload a video of your visit, our chatbot will automatically ingest the video and understand what happened at your last visit. Combined with our AI’s knowledge of healthcare, you can have a conversation regarding your condition and even upload images to chat about your concerns.

We utilize GPT-4 Vision and OpenAI Whisper to embed images, text, and audio clips into our vector databases, which then is used for chatting with our healthcare-informed chatbot using retrieval augmented generation (RAG).

The frontend is powered by NextJS and React to deliver a seamless user experience.

Edit autogen.sh and replace the paths to the frontend and backend to your own paths on your machine. Then, simply run:

chmod +x autogen.sh
./autogen.sh

View the result on http://localhost:3000!