Type Annotations in Python 3.8

One reason why Python is so easy to get started with is that it has dynamic types. You don’t have to specify the type of a variable, you just use variables as labels for containers of data. But in bigger projects, having types is helpful. If you have an undocumented function without types and maybe crappy variable naming, new developers will have a hard time. Luckily, variable annotations were added in Python 3.6 with PEP 526 🎉

This article is written in such a way that you can easily stop after the “mypy” section and take only a look at individual sections then.

Hello, Typed Annotated World!

So you can simply use the pattern

def some_function(param_name : typename) -> return_type_name:
    ...  # whatever the function does

Having type annotations is nice, but you need to check them! The Python runtimes do not do that, no matter if you use CPython, pypy, or something more exotic.

Type Checking with mypy

Install mypy via pip install mypy and run it:

$ mypy . --ignore-missing-imports
Success: no issues found in 1 source file

The --ignore-missing-imports flag is necessary because otherwise you will get a lot of messages like this:

error: Skipping analyzing ‘setuptools’: found module but no type hints or library stubs

To make it more convenient, I usually add a setup.cfg file in which I specify that I always want this flag to be applied:

[mypy]
ignore_missing_imports=true

Then you can pip install pytest-mypy and make sure mypy is always executed when you run pytest by adding this section to your setup.cfg:

[tool:pytest]
addopts = --mypy

It is important to note that the Python community and also mypy assumes that you come from a non-type annotated codebase. They want to make it easy for you to switch to an annotated code and thus support gradual typing. However, this means that you might miss errors if you don’t annotate your code! Mypy has a lot of flags to help you to make the move. You don’t need to annotate everything.

typing: List, Dict, Tuple, Any

The typing module adds support for type hints. It contains some of the types you will use most often: List, Dict, and Tuple.

Similarly, you can annotate that a dictionary maps strings to integers by Dict[str, int] . So List, Dict, and Tuple are generics. Any is just a way to specify that you could have arbitrary data in those containers. It is reasonable to use Any in the beginning when you start to add type annotations to a bigger codebase.

Stop Type Checking

As mentioned before, mypy and Python support gradual typing. And sometimes you need to silence the type checker to be able to continue (and hopefully fix it later 🤞). There are a couple of ways to do this with typing :

typing.Any : Every type is of type Any.

typing.cast(SomeClass, variable) : Sometimes mypy is not smart enough, so you can tell it which type you have. I did that a couple of times before I knew about typing.overload . Alternatively, you can also add assert isinstance(variable, Someclass)

# type: ingore : Explicitly tell the type-checker to ignore that line

typing: Union and Optional

Pretty often, you want to accept multiple types. Then you use Union:

As it happens pretty often that you need to accept some type and None , there is also typing.Optional . Optional[SomeType] is the same as Union[SomeType, None] .

typing: List vs Sequence

The type typing.List represents list . A typing.Sequence is “an iterable with random access” as Jochen Ritzel put it so nicely. For example, a string is a Sequence[Any] , but not a List[Any] .

typing: Dict vs Mapping

Similarly to the example List vs Sequence, the typing.Dict is meant mainly to represent a dict whereas typing.Mapping is more general. Stacksonstacks gives a good answer.

typing: TypedDict

TypedDict was introduced in PEP-589 and provides the possibility to denote which keys a dictionary should have and which type the values for those keys have. The example in the PEP shows this well:

from typing import TypedDict

class Movie(TypedDict):
    name: str
    year: int

movie: Movie = {'name': 'Blade Runner',
                'year': 1982}

By default, it must have all of the keys. You can make the keys optional by setting the totality: class Movie(TypedDict, total=False)

Many more Types

The typing module knows many more types and they are sometimes a bit confusing to distinguish. For example, what is the difference between a List, a Sequence, and an Iterable? Have a look in the documentation of collections.

There is also the types module which contains the ModuleType and the TracebackType .

Custom Types: Not all Strings are Created Equal

Not all strings contain the same type of content. They can represent anuser_id , a user_name , a password_hash , …

Especially for IDs, I have seen this to become messy. I think it’s pretty ridiculous to create an own class for those different string types as creating a class is usually development and maintenance overhead. So, what do you do?

Don’t worry, typing got you covered!

from typing import NewType
UserId = NewType("UserId", str)

typing.TypeVar: Define Generics in Python

You can define a type variable with TypeVar like this:

T = TypeVar('T')  # Can be anything
A = TypeVar('A', str, bytes)  # Must be str or bytes

It looks similar to NewType but is very different. NewType is used to create a meaningful alias for a given type. TypeVar is used to specify a generic, e.g. in the following example the two variables x and y are guaranteed to be of the same type, but it could be any type:

def is_smaller(x: T, y: T): ...

typing.overload

typing.Union is an anti-pattern sometimes, because you can also overload a function as Josh Reed shows:

Type checking only imports

I’ve recently seen myself in the position that I made a pretty heavy import on module level, just because of type checking. This felt wrong, so I asked for help. The solution was simple: typing.TYPE_CHECKING . This is true when running a type-checker but False during normal runs ❤️

Protocols

PEP 544 introduced structural subtyping and was introduced in Python 3.8. It feels like Interfaces in Java and works like this:

Note that there is no function body. After that definition, you can then use SupportsClose like any type.

The cool part is that the class Foo has no explicit relationship to SupportsClose ! It is only related by its structure!

Type comments

Type hints which are given as comments like this are outdated since Python 3.6:

However, you might want to disable type checking for single lines:

# type: ignore

Stub files

Stub files end in .pyi . If mypy finds a .py file and a .pyi file, it only loads the .pyi file. They are like header files in C++, but for Python. Instead of a function body, you use an Ellipsis ... :

def fib_list(n: int) -> List[int]: ...

There are also libraries like boto3-stubs which provide type annotations.

pyright, pyre, pytype

pyright is a Python static type checker written by Microsoft, pyre is one by Facebook, and pytype is one by Google. All of them claim to be faster than mypy, all of them have lower adoption than mypy. I haven’t used them so far.

Install them:

$ pip install pyre-check pytype

# Yes, pyright is written in TypeScript...
$ npm install -g pyright

Run them:

$ pyright .
$ pytype .

pyright was complaining a lot about stuff that is actually correct.

pydantic

Variable annotations can also be used to remove a lot of boilerplate code. For example, pydantic can help you with serialization/deserialization:

Which gives:

[User(name='user1', age=15), User(name='user2', age=28)]
[{"name": "user1", "age": 15}, {"name": "user2", "age": 28}]

FastAPI uses pydantic directly.

A cool thing about pydantic are the constrained types: PositiveFloat, NegativeInt, constr, …