Visual Question Answering (VQA)  is the task of answering free-text questions about the content of images. For example, given an image of a room – ‘What colour is the chair standing in the corner?’ – answering this question requires joint understanding of vision, language, spatial reasoning, and common sense. Thanks to advances in deep learning the state of the art in computer vision has been improving steadily over the past few years and language models experienced rapid progress recently, while spatial reasoning and common sense remain the most elusive.
Medical imaging offers plenty of interesting domain-specific challenges for VQA research . For example, the images can be 3-dimensional scans, specialist vocabulary can be used in questions and answers (such as anatomical directional terms for spatial relations), but on the other hand, we have ontologies such as UMLS which can support medical common sense knowledge.
This project will focus on a range of methodological approaches in computer vision and natural language processing, which collectively support answering questions about 2D and 3D radiology images.
Also, the interpretability of VQA will be addressed here – it is both a compelling basic research direction, and a desired consideration for healthcare applications.
 Antol, Stanislaw, et al. “VQA: Visual question answering.” Proceedings of the IEEE international conference on computer vision. 2015.
 Lau, Jason J., et al. “A dataset of clinically generated visual questions and answers about radiology images.” Scientific data 5 (2018): 180251.
- Programming and numerical skills
- Basic machine learning knowledge (practical experience in Python)
- Interest in both computer vision and natural language processing
- Written and oral communication skills
- Quick to adapt and learn new technical concepts
- Ability to work both individually and as part of a team
- Computing background in algorithms and data structures
- Deep learning experience
- Familiarity with PyTorch/TensorFlow/Keras
Canon Medical Research Europe Ltd, based in Edinburgh, designs and develops leading edge software in the medical domain. We visualise, process, and generate insights from multi-modal data which includes imaging and text. We are an R&D centre of excellence with strong academic and research links.
While based in the Canon Medical office the student will be part of the Artificial Intelligence Research team with their own workstation and access to Canon Medical software libraries. Working with a team of scientists who work on applying and adapting machine learning techniques to real-world healthcare applications, they will be expected to be an active member of the team, presenting their findings to the team as well as the wider company, and attending regular meetings. They will also be encouraged to network and collaborate with team members and other students on placement within Canon Medical.
Flexible Research Working
We aim to offer an inclusive, flexible and balanced working environment, by being an employer that cares for and respects its employees. To this end, Canon Medical’s Flexible Working Policy will be applicable to the successful applicant while based here. The policy outlines process and possible changes to fit working patterns to the student’s circumstances for shorter or longer periods of time.