pysnippets.lists.duplicates
Find duplicate and single values
duplicates(items: Iterable[H]) -> list[H]Return the values that appear more than once, in first-seen order.singles(items: Iterable[H]) -> list[H]Return the values that appear exactly once, in first-seen order.
List the values that repeat, or the ones that appear exactly once.
Which IDs appear twice in the import? Which appear only once? One Counter answers both.
duplicates.py✓ ruff · mypy --strict
from collections import Counter
from collections.abc import Hashable, Iterable
from typing import TypeVar
H = TypeVar("H", bound=Hashable)
def duplicates(items: Iterable[H]) -> list[H]:
"""Return the values that appear more than once, in first-seen order."""
return [item for item, count in Counter(items).items() if count > 1]
def singles(items: Iterable[H]) -> list[H]:
"""Return the values that appear exactly once, in first-seen order."""
return [item for item, count in Counter(items).items() if count == 1]- Counter keeps first-seen order, so the result does too
examples✓ 4 doctests · 3.11–3.14
passed on Python 3.11 to 3.14: >>> duplicates([1, 2, 2, 3, 4, 4, 4, 5])[2, 4]passed on Python 3.11 to 3.14: >>> duplicates([1, 2, 3])[]passed on Python 3.11 to 3.14: >>> singles([1, 2, 2, 3, 4, 4, 4, 5])[1, 3, 5]passed on Python 3.11 to 3.14: >>> singles("aabbc")['c']Edit the function or the examples and run the doctests here, on Python 3.14. Python loads only when you click (about 7 MB, cached after the first time).
python 3.14 · in your browser
Loading Python 3.14 in your browser (≈7 MB, cached after the first time)…
Tab inserts four spaces. Press Escape, then Tab, to leave the editor. Control or Command plus Enter runs the examples.
How it works
Counter(items)counts every value in one pass.- Its items come back in first-seen order, so the results do too.
duplicateskeeps counts above 1;singleskeeps counts of exactly 1.
Complexity: O(n) time, O(n) space
- 3.11✓passes
- 3.12✓passes
- 3.13✓passes
- 3.14✓passes
Import
from pysnippets.lists.duplicates import duplicates, singlesNo dependencies: copy duplicates.py into your project, or clone the repo.Adapted from filter-unique and filter-non-unique in 30-seconds-of-python by the 30 seconds of code contributors, CC BY 4.0. Changed: Merged and renamed (filter_unique → duplicates, filter_non_unique → singles); typed. Suggest an edit.