A transformer processes a sentence containing an ambiguous pronoun. Which component lets a token representation incorporate information from other positions in that sentence?

1. The filesystem directory containing the weights.
2. Self-attention over the sequence.
3. An alphabetical sort of the input tokens.
4. The HTTP status code returned by the endpoint.