Core ML

Part-of-Speech Tagging

The grammatical foundation of NLP, where every word in a sentence is classified by its syntactic role (noun, verb, adjective, etc.).

POS tagging assigns a grammatical label to each token, resolving ambiguity based on the surrounding context.

Why Does This Exist?

In English, many words are highly ambiguous. The word "book" can be a noun ("Read the book") or a verb ("Book the flight"). The word "back" can be an adjective, an adverb, a noun, or a verb depending entirely on the context.

If you are building an application to translate English to French, or a system to extract the subject of a sentence, you cannot treat the word "book" the same way in all cases. You need to know its grammatical role.

Part-of-Speech (POS) tagging is the process of reading a sequence of words and assigning a grammatical category (like Noun, Verb, Adjective, Adverb, Pronoun, Preposition) to each one.

Think of It Like This

Imagine you are a director casting actors for a play.

You have an actor named "Will". Sometimes Will plays the hero (Noun), and sometimes Will plays the villain (Verb). Just knowing his name doesn't tell you what he's doing in a specific scene. You have to read the script to figure out what role he was assigned.

POS tagging is the process of looking at the script (the sentence) and explicitly labelling every actor with their role so the rest of the production team knows what is going on.

How It Actually Works

Like Named Entity Recognition, POS tagging is a Token Classification task. Every word in the sequence must receive exactly one label.

The standard set of labels comes from the Penn Treebank tagset, which includes 36 distinct tags (e.g., NN for singular noun, NNS for plural noun, VB for base verb, VBD for past-tense verb).

1. The Classical Approach: Hidden Markov Models

Before deep learning, POS tagging was dominated by Hidden Markov Models (HMMs) and the Viterbi algorithm.

HMMs rely on two probabilities:

Transition Probability: How likely is it that a Noun follows an Adjective? (Very high). How likely is it that a Verb follows a Determiner? (Very low).
Emission Probability: How likely is the word "run" to be a Verb versus a Noun?

The algorithm calculates the most probable path of tags through the entire sentence by combining these probabilities. If it sees "The run", it knows "The" is a Determiner. Since Determiner $\rightarrow$ Noun is highly probable, it correctly tags "run" as a Noun, overriding the fact that "run" is usually a Verb.

2. The Modern Approach: Transformers

Today, POS tagging is typically handled as a byproduct of contextual embeddings. When a sentence is passed through a model like BERT, the embedding generated for the word "book" in "Read the book" is mathematically completely different from the embedding generated in "Book the flight".

Because the contextual embeddings already encode the syntactic role, a simple linear layer placed on top of the embeddings can predict the POS tag with near-perfect accuracy (>97%).

Show Me the Code

In Python, spaCy handles POS tagging automatically as part of its standard pipeline.

1import spacy2
3# Load the small English pipeline4nlp = spacy.load("en_core_web_sm")5
6# A sentence with ambiguous words7# "saw" (verb vs noun), "duck" (noun vs verb)8text = "I saw the duck with a saw."9
10doc = nlp(text)11
12print(f"{'Word':<10} | {'POS Tag':<8} | {'Description'}")13print("-" * 40)14
15for token in doc:16    print(f"{token.text:<10} | {token.pos_:<8} | {spacy.explain(token.pos_)}")17
18# -> I          | PRON     | pronoun19# -> saw        | VERB     | verb20# -> the        | DET      | determiner21# -> duck       | NOUN     | noun22# -> with       | ADP      | adposition (preposition)23# -> a          | DET      | determiner24# -> saw        | NOUN     | noun25# -> .          | PUNCT    | punctuation

Notice how spaCy correctly tags the first "saw" as a Verb and the second "saw" as a Noun!

Watch Out For

POS Tagging is often a solved problem. For standard English, POS taggers achieve over 97% accuracy, which is roughly equivalent to human inter-annotator agreement. You rarely need to train your own POS tagger from scratch unless you are working with a highly specialized domain (like obscure medical shorthand) or a low-resource language.

The Quick Version

POS tagging assigns a grammatical role (noun, verb, adjective, etc.) to every word in a sentence.
It resolves the ambiguity of words that can act as different parts of speech depending on context.
Classical methods used Hidden Markov Models, while modern methods use contextual embeddings from Transformers.
It is a foundational preprocessing step for more complex syntactic tasks like Dependency Parsing.

Related concepts

Dependency Parsing