pysnippets.lists.unique_elements
Remove duplicates and keep order
unique_elements(items: Iterable[H]) -> list[H]
De-duplicate a list while keeping the first occurrence of each value, in order.
list(set(items)) is the usual answer, and it scrambles the order. dict.fromkeys keeps it.
unique_elements.py✓ ruff · mypy --strict
from collections.abc import Hashable, Iterable
from typing import TypeVar
H = TypeVar("H", bound=Hashable)
def unique_elements(items: Iterable[H]) -> list[H]:
"""Remove duplicates from ``items``, keeping the first occurrence of each."""
return list(dict.fromkeys(items))- dict keys are unique and remember insertion order; a set doesn't
examples✓ 3 doctests · 3.11–3.14
passed on Python 3.11 to 3.14: >>> unique_elements([3, 1, 3, 2, 1])[3, 1, 2]passed on Python 3.11 to 3.14: >>> unique_elements("mississippi")['m', 'i', 's', 'p']passed on Python 3.11 to 3.14: >>> unique_elements([])[]Edit the function or the examples and run the doctests here, on Python 3.14. Python loads only when you click (about 7 MB, cached after the first time).
python 3.14 · in your browser
Loading Python 3.14 in your browser (≈7 MB, cached after the first time)…
Tab inserts four spaces. Press Escape, then Tab, to leave the editor. Control or Command plus Enter runs the examples.
How it works
- Dict keys are unique, and since Python 3.7 dicts keep insertion order.
dict.fromkeys(items)keeps the first occurrence of each value;list()takes the keys back out.- Values must be hashable. For lists of dicts, de-duplicate on a key instead.
Complexity: O(n) time, O(n) space
- 3.11✓passes
- 3.12✓passes
- 3.13✓passes
- 3.14✓passes
Import
from pysnippets.lists.unique_elements import unique_elementsNo dependencies: copy unique_elements.py into your project, or clone the repo.Adapted from unique-elements in 30-seconds-of-python by the 30 seconds of code contributors, CC BY 4.0. Changed: Rewritten to keep order; typed. Suggest an edit.