How machines learn to distinguish a person from a place
In short: When you read a sentence, your brain instantly categorizes names, dates, and organizations. Teaching computers to do the same—a process called Named-Entity Recognition—is far more complex than simply identifying words. It requires navigating linguistic ambiguities where a single name could represent a person, an airport, or even a political group.
A component of information extraction, named-entity recognition (NER) aims to identify and categorise unstructured text within set labels. These include people (PER), organizations (ORG), locations (LOC), and even quantities or temporal expressions like '2006'. The process typically involves two distinct phases: the detection of names through segmentation, and the classification of those names into an organized ontology. Beyond names, these systems can also identify monetary values, percentages, and vehicle types. This segmentation often treats names as contiguous spans of tokens without any internal nesting.
The task is fraught with ambiguity. A name like 'JFK' might refer to a former US president or his son, while 'IRA' could denote a retirement account, a reading association, or an army. Even 'The White House' can function as a metonym, referring to an organization rather than a physical location. To manage this, researchers use various hierarchies, such as the 29-type BBN categories from 2002 or Sekine's 200-subtype hierarchy. Researchers also utilize the 2011 Ritter hierarchy, which is based on Freebase entity types. This complexity is further complicated by the fact that some temporal expressions, such as 'June', may lack a specific year.
Modern approaches vary from hand-crafted grammar rules, which offer high precision but require months of expert labor, to statistical models that rely on large datasets. In specialized fields like microbiology, deep learning models like BioBERT have achieved an F1-score of 96%. Similarly, in traditional Chinese medicine, 'knowledge distillation'—transferring semantic knowledge from a teacher model to a student model—has been used to improve precision and recall in sparse datasets. In the context of traditional Chinese medicine, the TBTC model has demonstrated an F1-score of 77.01. Meanwhile, in microbiome research, models can process an entire full-text document in just 7 seconds.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Named-entity recognition
- Also: Knowledge distillation for named entity recognition in traditional chinese medicine.
- Also: Microbial named entity recognition and normalisation for AI-assisted literature review and meta-analysis.
To cite this card, link to https://intellectplus.co.uk/app/health-named-entity-recognition.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
