I am a part-time PhD Student working on evaluation of NLP datasets using tools such as taxonomies, inter-rater reliability statistics and item response models. In [2] we investigate how to automatically evaluate taxonomies using their intrinsic properties. In [1], we systematically compare distances that can be used to calculate annotator agreement in open-ended biomedical annotations. More recently, I have been working on leveraging item-response models to inform model training on the basis of annotations.

Research Interests

NLP · Ontologies · Language models · RL · Annotation · Biomedical text mining · Deep learning