Using Google Streetview to Map detected Potholes
Click the Poster to View Full Screen, Right click to save image
Shazab Ali
College:
The Dorothy and George Hennings College of Science, Mathematics, and Technology
Major:
Computer Science
Faculty Research Advisor(s):
Daehan Kwak
Abstract:
In medical diagnostics, this project aims to revolutionize chest X-ray interpretation using
medical visual question answering. The goal is to streamline radiological evaluations, providing
quicker and more consistent results for patients. Traditional methods relying on individual
radiologists may lead to inconsistencies and delays. Our solution integrates object detection,
Natural Language Processing (NLP), and stacked attention networks, enabling users to upload a
chest X-ray, pose related questions, and receive accurate predicted answers.
Our image preprocessing pipeline resizes input images and questions before passing them
through an attention layer. This attention mechanism allows the network to dynamically focus on
specific regions of the image, simulating a radiologist's attention to particular areas in an X-ray.
Leveraging a dataset of 329 chest X-rays, we meticulously organized resulting question-answer
pairs, question IDs, answer types, X-ray image locations, and annotations into a structured JSON
file.
Our system, integrating object detection, stacked attention networks, and a diverse dataset, has
shown promising results. We observed adaptability in question answering and enhanced accuracy
with the inclusion of a stacked attention network. In the future, we plan to address potential
biases and improve performance, we plan to add a new dataset for brain tumors. Additionally,
we'll apply bias mitigation techniques to account for abnormal images in both chest and brain
X-ray datasets. This approach aligns with our commitment to developing a powerful and
adaptable system for medical image interpretation.
Integrating AI into radiology addresses the variability in human interpretations, evidenced by
studies showing disagreement among experienced radiologists. Built on a diverse dataset, our
system aims to reduce discrepancies and enhance diagnostic accuracy. This research underscores
the potential of combining computer vision and NLP to transform medical diagnostics, with
ongoing efforts to provide healthcare providers and patients more reliable and expedited
diagnoses in the medical field.