fluttersmith/py-snippetsGitHub ↗
pysnippets.lists.has_duplicates

Check a list for duplicates

Return True as soon as any value repeats, even in a huge or endless iterable.

Lists#dedupe#validationO(n) time✓ 3 examples pass on 3.11–3.14

len(items) != len(set(items)) builds the whole set before it can answer. This returns at the first repeat.

has_duplicates.py✓ ruff · mypy --strict
from collections.abc import Hashable, Iterable


def has_duplicates(items: Iterable[Hashable]) -> bool:
    """Return ``True`` if any value occurs more than once in ``items``."""
    seen = set()
    for item in items:
        if item in seen:
            return True
        seen.add(item)
    return False
  1. stops at the first repeat, so it works on huge or endless input
examples✓ 3 doctests · 3.11–3.14
>>> has_duplicates([1, 2, 3, 4, 5, 5])True>>> has_duplicates([1, 2, 3, 4, 5])False>>> has_duplicates(n % 7 for n in range(10**9))True
Edit the function or the examples and run the doctests here, on Python 3.14. Python loads only when you click (about 7 MB, cached after the first time).

How it works

  • Each item is checked against the values seen so far.
  • It stops at the first repeat, which is why the example can scan a billion-item generator instantly.
  • Values must be hashable.

Complexity: O(n) time worst case, O(n) space

Importfrom pysnippets.lists.has_duplicates import has_duplicatesNo dependencies: copy has_duplicates.py into your project, or clone the repo.

Adapted from has-duplicates in 30-seconds-of-python by the 30 seconds of code contributors, CC BY 4.0. Changed: Rewritten to exit early and accept any iterable; typed. Suggest an edit.