AR Surgical Guidance @HackMIT

AR Surgical Guidance was my first hackathon project, built at HackMIT with Ryan Mago, an electrical engineer student at Stanford. It's a real-time augmented-reality system that lets a remote surgical expert assist an on-site medic through live vision, voice, and spatial tracking -- streaming the medic's field of view to a surgeon who can annotate directly on it and guide a procedure in real time.

The motivation: in conflict and low-resource regions, medics with limited surgical training are forced into high-stakes procedures, and the expertise to guide them usually can't physically get there. Over 100,000 surgeries a day need guidance and most surgeons who want to help can't travel -- so the platform removes physical presence as the barrier and brings the expertise to the patient instead.

On the medic's side, XREAL One AR glasses and a Raspberry Pi 5 with an Arducam and microphone capture and stream the field of view over low-latency sockets. A central server routes the video and sensor data to the remote "doctor" client -- a UI that shows the live stream and overlays and accepts voice and annotation input. The system is modular enough to run on the constrained Pi hardware or be tested locally on a laptop.

We demoed it live at HackMIT. To get the prototype working on the day we tunneled our API endpoints through Ngrok and ran a split deployment -- MongoDB for data, frontend on Vercel, backend on Replit -- pivoting the architecture from our first prototype as we hit deployment walls.