Merge pull request #476 from yukinarit/new-strict-typecheck

pyserde is powered by beartype

.pre-commit-config.yaml

@@ -14,7 +14,7 @@ repos:
         args:
           - .
   - repo: https://github.com/RobertCraigie/pyright-python
-    rev: v1.1.309
+    rev: v1.1.345
     hooks:
     - id: pyright
       additional_dependencies: ['pyyaml', 'msgpack', 'msgpack-types']

docs/en/SUMMARY.md

@@ -8,6 +8,6 @@
 - [Class Attributes](class-attributes.md)
 - [Field Attributes](field-attributes.md)
 - [Union](union.md)
-- [Type Check](type-check.md)
+- [Type Checking](type-check.md)
 - [Extension](extension.md)
 - [FAQ](faq.md)

docs/en/decorators.md

@@ -91,11 +91,13 @@ class Wrapper(External):
 pyserde supports forward references. If you replace a nested class name with with string, pyserde looks up and evaluate the decorator after nested class is defined.
 
 ```python
+from __future__ import annotations # make sure to import annotations
+
 @dataclass
 class Foo:
     i: int
     s: str
-    bar: 'Bar'  # Specify type annotation in string.
+    bar: Bar  # Bar can be specified although it's declared afterward.
 
 @serde
 @dataclass

docs/en/type-check.md

@@ -1,23 +1,59 @@
 # Type Checking
 
-This is one of the most awaited features. `pyserde` v0.9 adds the experimental type checkers. As this feature is still experimental, the type checking is not perfect. Also, [@tbsexton](https://github.com/tbsexton) is looking into [more beautiful solution](https://github.com/yukinarit/pyserde/issues/237#issuecomment-1191714102), the entire backend of type checker may be replaced by [beartype](https://github.com/beartype/beartype) in the future.
+pyserde offers runtime type checking since v0.9. It was completely reworked at v0.14 using [beartype](https://github.com/beartype/beartype) and it became more sophisticated and reliable. It is highly recommended to enable type checking always as it helps writing type-safe and robust programs.
 
-### `NoCheck`
+## `strict`
 
-This is the default behavior until pyserde v0.8.3 and v0.9.x. No type coercion or checks are run. Even if a user puts a wrong value, pyserde doesn't complain anything.
+Strict type checking is to check every field value against the declared type during (de)serialization and object construction. This is the default type check mode since v0.14. What will happen with this mode is if you declare a class with `@serde` decorator without any class attributes, `@serde(type_check=strict)` is assumed and strict type checking is enabled.
 
 ```python
 @serde
-@dataclass
 class Foo
     s: str
+```
 
+If you call `Foo` with wrong type of object,
+```python
 foo = Foo(10)
-# pyserde doesn't complain anything. {"s": 10} will be printed.
+```
+
+you get an error
+```python
+beartype.roar.BeartypeCallHintParamViolation: Method __main__.Foo.__init__() parameter s=10 violates type hint <class 'str'>, as int 10 not instance of str.
+```
+
+> **NOTE:** beartype exception instead of SerdeError is raised from constructor because beartype does not provide post validation hook as of Feb. 2024.
+
+similarly, if you call (de)serialize APIs with wrong type of object,
+
+```python
 print(to_json(foo))
 ```
 
-### `Coerce`
+again you get an error
+
+```python
+serde.compat.SerdeError: Method __main__.Foo.__init__() parameter s=10 violates type hint <class 'str'>, as int 10 not instance of str.
+```
+
+> **NOTE:** There are several caveats regarding type checks by beartype.
+>
+> 1. beartype can not validate on mutated properties
+>
+> The following code mutates the property "s" at the bottom. beartype can not detect this case.
+> ```python
+> @serde
+> class Foo
+>     s: str
+>
+> f = Foo("foo")
+> f.s = 100
+> ```
+>
+> 2. beartype can not validate every one of elements in containers. This is not a bug. This is desgin principle of beartype. See [Does beartype actually do anything?](https://beartype.readthedocs.io/en/latest/faq/#faq-o1].
+> ```
+
+## `coerce`
 
 Type coercing automatically converts a value into the declared type during (de)serialization. If the value is incompatible e.g. value is "foo" and type is int, pyserde raises an `SerdeError`.
 
@@ -33,40 +69,17 @@ foo = Foo(10)
 print(to_json(foo))
 ```
 
-### `Strict`
+## `disabled`
 
-Strict type checking is to check every value against the declared type during (de)serialization. We plan to make `Strict` a default type checker in the future release.
+This is the default behavior until pyserde v0.8.3 and v0.9.x. No type coercion or checks are run. Even if a user puts a wrong value, pyserde doesn't complain anything.
 
 ```python
-@serde(type_check=Strict)
+@serde
 @dataclass
 class Foo
     s: str
 
 foo = Foo(10)
-# pyserde checks the value 10 is instance of `str`.
-# SerdeError will be raised in this case because of the type mismatch.
+# pyserde doesn't complain anything. {"s": 10} will be printed.
 print(to_json(foo))
 ```
-
-> **NOTE:** Since pyserde is a serialization framework, it provides type checks or coercing only during (de)serialization. For example, pyserde doesn't complain even if incompatible value is assigned in the object below.
->
-> ```python
-> @serde(type_check=Strict)
-> @dataclass
-> class Foo
->     s: str
->
-> f = Foo(100) # pyserde doesn't raise an error
-> ```
->
-> If you want to detect runtime type errors, I recommend to use [beartype](https://github.com/beartype/beartype).
-> ```python
-> @beartype
-> @serde(type_check=Strict)
-> @dataclass
-> class Foo
->     s: str
->
-> f = Foo(100) # beartype raises an error
-> ```

examples/enum34.py

@@ -43,9 +43,7 @@ def main() -> None:
     print(f)
     s = to_json(f)
 
-    # You can also pass an enum-compabitle value (in this case True for E.B).
-    # Caveat: Foo takes any value IE accepts. e.g., Foo(True) is also valid.
-    s = to_json(Foo(3))  # type: ignore
+    s = to_json(Foo(IE(3)))
     print(s)
 
 

examples/forward_reference.py

@@ -1,3 +1,4 @@
+from __future__ import annotations
 from dataclasses import dataclass
 
 from serde import serde
@@ -8,7 +9,7 @@
 class Foo:
     i: int
     s: str
-    bar: "Bar"  # Specify type annotation in string.
+    bar: Bar
 
 
 @serde

examples/generics_nested.py

@@ -44,11 +44,12 @@ def main() -> None:
     print(event_a)
     print(new_event_a)
 
-    payload = Payload(1, A("a_str"))
-    payload_dict = to_dict(payload)
-    new_payload = from_dict(Payload[A], payload_dict)
-    print(payload)
-    print(new_payload)
+    # This has a bug, see https://github.com/yukinarit/pyserde/issues/464
+    # payload = Payload(1, A("a_str"))
+    # payload_dict = to_dict(payload)
+    # new_payload = from_dict(Payload[A], payload_dict)
+    # print(payload)
+    # print(new_payload)
 
 
 if __name__ == "__main__":

examples/init_var.py

@@ -12,7 +12,7 @@ class Foo:
     b: Optional[int] = field(default=None, init=False)
     c: InitVar[Optional[int]] = 1000
 
-    def __post_init__(self, c: Optional[int]) -> None:
+    def __post_init__(self, c: Optional[int]) -> None:  # type: ignore
         self.b = self.a * 10
 
 

examples/runner.py

@@ -36,7 +36,7 @@
 import skip
 import tomlfile
 import type_check_coerce
-import type_check_strict
+import type_check_disabled
 import type_datetime
 import type_decimal
 import union
@@ -82,8 +82,8 @@ def run_all() -> None:
     run(nested)
     run(lazy_type_evaluation)
     run(literal)
-    run(type_check_strict)
     run(type_check_coerce)
+    run(type_check_disabled)
     run(user_exception)
     run(pep681)
     run(variable_length_tuple)

examples/type_check_coerce.py

@@ -1,17 +1,17 @@
 from dataclasses import dataclass
 from typing import Dict, List, Optional
 
-from serde import Coerce, serde
+from serde import coerce, serde
 from serde.json import from_json, to_json
 
 
-@serde(type_check=Coerce)
+@serde(type_check=coerce)
 @dataclass
 class Bar:
     e: int
 
 
-@serde(type_check=Coerce)
+@serde(type_check=coerce)
 @dataclass
 class Foo:
     a: int

examples/type_check_disabled.py

@@ -0,0 +1,25 @@
+from typing import Dict, List
+
+from serde import disabled, serde
+from serde.json import from_json, to_json
+
+
+@serde(type_check=disabled)
+class Foo:
+    a: int
+    b: List[int]
+    c: List[Dict[str, int]]
+
+
+def main() -> None:
+    # Foo is instantiated with wrong types but serde won't complain
+    f = Foo(a=1.0, b=[1.0], c=[{"k": 1.0}])  # type: ignore
+    print(f"Into Json: {to_json(f)}")
+
+    # Also, JSON contains wrong types of values but serde won't complain
+    s = '{"a": 1, "b": [1], "c": [{"k": 1.0}]}'
+    print(f"From Json: {from_json(Foo, s)}")
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -42,6 +42,7 @@ numpy = [
 ]
 orjson = { version = "*", markers = "extra == 'orjson' or extra == 'all'", optional = true }
 plum-dispatch = ">=2,<2.3"
+beartype = "*"
 
 [tool.poetry.dev-dependencies]
 pyyaml = "*"
@@ -53,7 +54,7 @@ numpy = [
     { version = ">1.21.0", markers = "python_version ~= '3.9.0'" },
     { version = ">1.22.0", markers = "python_version ~= '3.10'" },
 ]
-mypy = "==1.3.0"
+mypy = "==1.8.0"
 pytest = "*"
 pytest-cov = "*"
 pytest-watch = "*"
@@ -104,6 +105,7 @@ include = [
   "tests/test_kwonly.py",
   "tests/test_flatten.py",
   "tests/test_lazy_type_evaluation.py",
+  "tests/test_type_check.py",
   "tests/data.py",
 ]
 exclude = [

serde/__init__.py

@@ -31,12 +31,12 @@
     ClassSerializer,
     ClassDeserializer,
     AdjacentTagging,
-    Coerce,
+    coerce,
     DefaultTagging,
     ExternalTagging,
     InternalTagging,
-    NoCheck,
-    Strict,
+    disabled,
+    strict,
     Tagging,
     TypeCheck,
     Untagged,
@@ -82,9 +82,9 @@
     "ExternalTagging",
     "InternalTagging",
     "Untagged",
-    "NoCheck",
-    "Strict",
-    "Coerce",
+    "disabled",
+    "strict",
+    "coerce",
     "field",
     "default_deserializer",
     "asdict",
@@ -119,7 +119,7 @@ def serde(
     serializer: Optional[SerializeFunc] = None,
     deserializer: Optional[DeserializeFunc] = None,
     tagging: Tagging = DefaultTagging,
-    type_check: TypeCheck = NoCheck,
+    type_check: TypeCheck = strict,
     serialize_class_var: bool = False,
     class_serializer: Optional[ClassSerializer] = None,
     class_deserializer: Optional[ClassDeserializer] = None,
@@ -135,7 +135,7 @@ def serde(
     serializer: Optional[SerializeFunc] = None,
     deserializer: Optional[DeserializeFunc] = None,
     tagging: Tagging = DefaultTagging,
-    type_check: TypeCheck = NoCheck,
+    type_check: TypeCheck = strict,
     serialize_class_var: bool = False,
     class_serializer: Optional[ClassSerializer] = None,
     class_deserializer: Optional[ClassDeserializer] = None,
@@ -152,7 +152,7 @@ def serde(
     serializer: Optional[SerializeFunc] = None,
     deserializer: Optional[DeserializeFunc] = None,
     tagging: Tagging = DefaultTagging,
-    type_check: TypeCheck = NoCheck,
+    type_check: TypeCheck = strict,
     serialize_class_var: bool = False,
     class_serializer: Optional[ClassSerializer] = None,
     class_deserializer: Optional[ClassDeserializer] = None,