DEV: Align mypy Makefile target with strict mode

[costajohnt]

Summary

Moves the --strict, --ignore-missing-imports, and --check-untyped-defs flags from the Makefile into pyproject.toml [tool.mypy], so that make mypy and direct mypy pypdf invocations use the same configuration.

Fixes all 55 strict-mode violations across 18 source files:

• 48 no-any-return: added cast() for return values originating from untyped libraries (pycryptodome), dict lookups, and stream reads
• 5 comparison-overlap: fixed a latent bytes-indexing bug (data[-1] returns int, not bytes — changed to data[-1:]) and added type: ignore where runtime comparisons are intentional (FloatObject == 0, PdfObject == string)
• 2 assignment/index: used bytearray for mutable byte buffers in filters.py

The data[-1] → data[-1:] fix corrects a real bug: comparing int to bytes was always True, causing incorrect newline detection in array-based content streams. The test assertion for output byte count was updated accordingly.

Test plan

• mypy pypdf reports 0 errors with strict mode
• make mypy works with simplified command (just mypy pypdf)
• All non-sample-file-dependent tests pass (1037 passed, 14 skipped)
• Updated test_content_stream__array_based__output_length assertion to match corrected byte count

Closes #3688

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.59%. Comparing base (c77d479) to head (309a37e).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3690      +/-   ##
==========================================
+ Coverage   97.48%   97.59%   +0.10%     
==========================================
  Files          55       55              
  Lines       10211    10218       +7     
  Branches     1875     1875              
==========================================
+ Hits         9954     9972      +18     
+ Misses        150      141       -9     
+ Partials      107      105       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stefan6419846]

Thanks for the PR. I have left some inline comments where I think that the chosen fix is not appropriate or could be improved.

[costajohnt]

Addressed all the review feedback — here's a summary of the two items I kept as-is:

_rectangle.py (#16): The typed locals on the property getters (value: FloatObject = self[0]) are needed because ArrayObject.__getitem__ returns Any in mypy strict. I tried adding __getitem__ overloads with _ensure_is_number always returning FloatObject, but that converts NumberObject(0) to FloatObject(0) which changes serialization from b'0' to b'0.0' — affecting nearly every PDF's page boxes. The typed locals avoid that behavioral change while satisfying mypy.

_page.py (#18): The tuple (operands, b"cm") is correct — ContentStream.operations is typed as list[tuple[Any, bytes]], so using a list [operands, b"cm"] would be a type error.

[costajohnt]

Thanks for the thorough review! I've rebased on main and reworked this PR to address all feedback. Here's a summary of changes:

Rebased on main (5 commits behind) — resolved the merge conflict in pyproject.toml and dropped the accidental _writer.py reverts. The compress_identical_objects() deprecation refactor is now preserved as intended.

pyproject.toml — kept strict = true but now preserves the test overrides from #3660 (tests., tests.generic., tests.scripts.*). The individual mypy flags are replaced by strict mode which is the superset.

Crypto providers — reverted the typed locals approach. Instead, added cryptography and pycryptodome as additional_dependencies in .pre-commit-config.yaml so mypy has the actual stubs during pre-commit runs.

Addressing specific comments:

• _writer.py — dropped all changes. The diff was reverting upstream work, not adding type annotations. Sorry about that.
• _text_state_manager.py — removed the unnecessary explicit type on result. The return type of raw_transform is sufficient for mypy to infer it.
• test_annotations.py — replaced the __new__/__init__ pattern with Text(rect=..., text=...) to construct via a concrete subclass.
• _page.py — used content_stream as the variable name (avoiding abbreviation per your feedback). The tuple is needed because operations is typed as list[tuple[Any, bytes]]. The int() wrapper on lst.index() is needed because lst returns Any from the property chain.
• filters.py — used str_data_mut as the intermediate variable name to avoid type-incompatible reassignment of str_data from bytes to bytearray.
• _page_labels.py — switched to cast(str, ...) as suggested.
• _base.py — kept the assertion message.
• _rectangle.py Union vs | — Union[FloatObject, NumberObject] is pre-existing syntax, not introduced by this PR. Since pypdf supports Python 3.9+ and the file doesn't use from __future__ import annotations, Union is the correct syntax. Happy to modernize it if you'd like, but that might be better as a separate cleanup.
• _data_structures.py binary stream type — agreed this should be a separate PR. Used BinaryStreamType (already defined in _utils.py) for _read_unsized_from_stream as an incremental step.

Bug fix note: the data[-1] != b"\n" to data[-1:] != b"\n" change fixes a latent bug where bytes[int] returns an int, so int != bytes was always True (newline was unconditionally appended). The slice correctly compares bytes-to-bytes. This changes the output byte count for array-based content streams, reflected in the updated test expectation.

Public API types preserved: color, font_format, and associated_file_relationship all keep their original return types (Optional[ArrayObject], Optional[OutlineFontFlag], str). Only internal casts were added to satisfy strict mypy.

[costajohnt]

Follow-up: pushed additional commits to fix strict mypy errors surfaced in CI.

Key finding: pycryptodome does not ship type stubs (unlike cryptography which does). So installing it in pre-commit/CI doesn't help mypy — the types remain Any. Added a mypy module override to disable warn_return_any for the pycryptodome provider specifically:

[[tool.mypy.overrides]]
module = "pypdf._crypt_providers._pycryptodome"
warn_return_any = false

Also added type: ignore comments for comparison-overlap in _writer.py (comparing PdfObject with string literals) and a couple test files, plus attr-defined for PIL.Image.__version__ — all pre-existing code that strict mode's strict_equality and no_implicit_reexport flags now catch.

Only cryptography remains in pre-commit additional_dependencies since it actually has type stubs.

[stefan6419846]

pycryptodome does not ship type stubs (unlike cryptography which does).

This is wrong - the latest wheel contains corresponding stub files besides the regular files. What are the messages you receive when enabling it in the pre-commit configuration?

Currently, the trove classifier is missing, which I have created a corresponding PR for, although the docs suggest that this should not be required: Legrandin/pycryptodome#909

[stefan6419846]

Besides the new remarks, could you please have a look at the merge conflict?

[costajohnt]

Thanks for the follow-up. Pushed a commit addressing three of the four April 16 remarks plus the rebase:

• _data_structures.py:773 got the "mypy" assertion message.
• _files.py:209 now defaults to NameObject("/Unspecified") with a NameObject cast.
• test_image_xobject.py:245 swapped to PIL.__version__, which is in the PIL stubs, so the type: ignore is gone. pypdf/__init__.py already uses that pattern.
• Rebased on main. The filters.py conflict was just combining the row_length rename with the str_data_mut rename.

For _base.py:388 (typing self.pdf), I tried pdf: Union["PdfReader", "PdfWriter"] but it ripples into 25 mypy errors across _page.py and _doc_common.py. PdfWriterProtocol is passed to IndirectObject() in several places, PdfDocCommon is a third caller type, and _page.py accesses _objects and _add_object which only exist on PdfWriter. That feels like a follow-up PR rather than an expansion of this one. Happy to file it separately if that works.

On the older threads:

• _data_structures.py dedicated binary stream type: same separate-PR logic, still happy to pick it up.
• _rectangle.py Union vs |: equivalent semantically, but | in runtime positions needs Python 3.10+, and pyproject.toml still pins requires-python = ">=3.9". _rectangle.py doesn't have from __future__ import annotations, so Union is load-bearing there. If you'd like a repo-wide move to | with a future import, happy to do that as a separate cleanup.
• _text_state_manager.py explicit type: the change swaps dict[int, float] for TextStateManagerDictType on raw_transform so that new_transform's result.update({"is_text": is_text, ...}) type-checks, since the updated dict has str keys and bool values that don't fit the narrower dict[int, float]. Let me know if you'd prefer another approach.

[stefan6419846]

For _base.py:388 (typing self.pdf), I tried pdf: Union["PdfReader", "PdfWriter"] but it ripples into 25 mypy errors across _page.py and _doc_common.py. PdfWriterProtocol is passed to IndirectObject() in several places, PdfDocCommon is a third caller type, and _page.py accesses _objects and _add_object which only exist on PdfWriter. That feels like a follow-up PR rather than an expansion of this one. Happy to file it separately if that works.

In theory, we lately guarded the code enough to let _page.py correctly handle PdfReader vs. PdfWriter. While it would be nice to avoid the explicit typing of self.pdf, I guess we can live with it for now until something starts to break or someone wants to clean this up properly.

_rectangle.py Union vs |: equivalent semantically, but | in runtime positions needs Python 3.10+, and pyproject.toml still pins requires-python = ">=3.9". _rectangle.py doesn't have from __future__ import annotations, so Union is load-bearing there. If you'd like a repo-wide move to | with a future import, happy to do that as a separate cleanup.

If I remember correctly, you exchanged Union[...] with "... | ...", which did not make much sense for me. I do not plan to explicitly change the annotation handling on the short term unless pyupgrade automatically considers this after dropping Python 3.9 support, which is not yet scheduled at a fixed date.

[stefan6419846]

Thanks for your patience. For now, I do not see any obvious issues any more. If you think that future cleanups make sense, consider opening dedicated PRs.