Type hints: what and why

Intro

I was immediately drawn to Python when I first encountered it due to its dynamic language features. Python uses the duck typing design, which means that the type of an object is not as important as its behavior. This feature allows for faster development and a reduction in burdensome type declarations. Additionally, the support of powerful third-party libraries solidifies Python as my preferred programming language.😺

With the proposal of PEP 484¹, Python adopted type hints, which resemble static typing. It’s not true though, Python’s type hints are optional, and they have no runtime effect.

It seems that writing this blog post specifically to introduce Python’s type hints is unnecessary, but I have found that using them in my code still provides several benefits:

• Static type checker can check your code. For example, Mypy
• The code completion in IDE will become more intelligent. It will also report a bug if we use the wrong APIs. This is probably the biggest motivation for me to choose to write type hints
• Manage code complexity. Type hints expose useful information about APIs. As a developer, we can get a general idea by just looking at the signature of such an annotated function, without having to check the docstrings frequently.

Function annotations

Shipped with Python 3.0, the syntax of type hints has been established²:

• Arguments: name[: type] [ = default_val ], the [] means optional
• Return type: the syntax is -> return_type
• We can access the __annotations__ attribute to get type hints informatios. It returns a dict with {ParamName: ParamType}. It’s a bad practice to access this attribute directly. We can use the functions inside the inspect module(Python 3.10+) or the typing module(Python 3.5 ~ 3.9). See the following example:

def maximum(a: float, b: float) -> float:
    """ A simple function to return the maximum elements of two floats"""
    return max(a, b)


# >= Python 3.10, do this
import inspect
assert inspect.get_annotations(maximum) == maximum.__annotations__

# Python 3.5 ~ 3.9
import typing
assert typing.get_type_hints(maximum) == maximum.__annotations__

inspect.get_annotations(maximum)
# {'a': float, 'b': float, 'return': float}

# returns the maximum of two strings
# , but we declared the arguments should be float!
maximum('hello', 'world')     
# 'world'

Variables annotations

Before Python 3.6, the only way to annotate a variable was to include this information in comments, such as # type ...¹. This is referred to as “type comments.” With the introduction of PEP 526, a new syntax for variable annotations was established, which is similar to the syntax used for annotating function arguments³

a: int        # undefined typed value
a: int = 0    # typed value with default value

Common usage

Simple built-in types

The simple built-in types refer to int, str etc. They can also be types defined in third-party packages. See the previous maximum function as an example.

Any type

The Any type denotes that it can be any type. But it’s different from the object type¹

Conceptually, an unannotated function can be treated as if it were annotated with the Any type.

def foo(x):
    ...

# it assumes:
def foo(x: Any) -> Any:
    ...

Never type

When the Never is used as the return type, it means that the function would never return normally, that is:

1. This function calls sys.exit or similar functions to halt the program without returning.
2. This function always throws exceptions.

For example

from typing import Never

def foobar() -> Never:
    raise ValueError("This is a NoReturn function")

Collections and Mappings

We refer to each element inside a collection as an item. To add type hints for both the collection type and the item type, Python uses the [] notation. For example, to express a list of strings, we would write list[str]. This notation makes it clear that the list contains elements of the str type.

string_list: list[str] = ['hello', 'world']
    
# tuple[type1, type2, ..., typen] with fixed size
date: tuple[int, int, int] = (2023, 1, 11)
    
string_count: dict[str, int] = {
    'hello': 1,
    'world': 2,
}

The following join_str_list function accepts a list of strings and uses the whitespace to join them.

def join_str_list(string_list: list[str]) -> str:
    """ join all string in a list"""
    return ' '.join(string_list)

print(join_str_list(string_list))
# hello world
print(inspect.get_annotations(join_str_list))
# {'string_list': list[str], 'return': <class 'str'>}

def sum_variable_integers(data: tuple[int, ...]):
    """ Sum all integers of a tuple"""
    sum_val = 0
    for integer in data:
        sum_val += integer
    
    return sum_val

print(sum_variable_integers((1, 2, 3)))  # 6
print(sum_variable_integers((3,)))       # 3

What if we want to denote a list with different types of values? We can exploit the Any type:

list[Any]

A better solution is just using list.

collections.abc’s abstract type

Understanding how to use list, dict, set is relatively straightforward. However, a more effective practice is to utilize abstract types in collections.abc. One may question the advantages of this approach. The answer is that it can distinguish between mutable and immutable types.

For example, dict has 2 abstract types in collections.abc: Mapping and MutableMapping. The former represents immutable mapping, while the latter denotes mutable mappings. The following code demonstrates this idea:

from collections.abc import Mapping
def read_only(v: Mapping[str, int]):
    v.pop()

The read_only function tries to modify the input v although the type hint says Mapping[str, int]. The Mypy catches this type error :0

$ mypy .
main.py:17: error: "Mapping[str, int]" has no attribute "pop"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)

Type alias

Sometimes, the type will be so complicated that we don’t want to write it everywhere. So what do we do? Well, we can give it an alias with a meaningful name. The syntax is simple:

AliasName = Type

Take the previously defined date type as an example. A date is a tuple containing three int types. The type list[tuple[int, int, int] would denote a list of dates. To make the code more readable, it can be beneficial to give this type an alias, such as Date. See the following example:

Date = tuple[int, int, int]
DateList = list[Date]

def print_date_list(l: DateList):
    """ Print all dates in the format `year-month-day` in the date list"""
    for year, month, day in l:
        print(f'{year}-{month}-{day}')
    
print_date_list([(2022, 1, 1), (2023, 1, 3)])
# 2022-1-1 
# 2023-1-3
print(inspect.get_annotations(print_date_list))
# {'l': list[tuple[int, int, int]]}

The syntax of type alias is quite similar to defining a global variable. To make it more explicit, PEP 613(Python 3.10+) proposes a better way⁵:

AliasName: TypeAlias = Type

from typing import TypeAlias

Date: TypeAlias = tuple[int, int, int]

Parameterized generic types

In other programming languages, usually, they use an uppercase letter like T to denote a parameterized generic type. In Python, we use TypeVar to do the same thing. As the docs say:

T = TypeVar('T')             # Can be anything
S = TypeVar('S', bound=str)  # Can be any subtype of str
A = TypeVar('A', str, bytes) # Must be exactly str or bytes

To summarize, TypeVar provides two ways for us to restrict the generic types:

1. use bound=some_type, then we can only pass the subtype of some_type.
2. specify the allowed types directly

from typing import TypeVar

GenericString = TypeVar('GenericString', str, bytes)

def process(s: GenericString):
    """ The GenericString can be either str or bytes"""
    ...

Union type

The Union[type1, type2, ...] means that the allowed type is one of the types we specified, which is the logical or relationship. Using the symbol | to indicate the logical or relationship is quite common in other programming languages. For example, the OCaml’s variant type uses | to separate different variants.

def parse(s: str | int) -> int | None:
    """ Parse `s` and get an integer value. The `s` may be a string.
    Return None if fail
    """
    if isinstance(s, str):
        if not s.isdigit():
            return None
        else:
            return int(s)
    elif isinstance(s, int):
        return s

inspect.get_annotations(parse)
# {'s': str | int, 'return': int | None}

Optional type

The Optional[type1] represents a type that can be either type1 or None. You may have seen the optional type in other programming languages. For example, the Haskell has a Maybe type.

A classic example is a division function which may divide by zero.

def divide(a: int, b: int) -> int | None:
    if b == 0:
        return None

    return a // b

Callables

We use a Python function as an example of a callable. In Python, functions are first-class objects, meaning they can be passed as parameters or return values from other functions. Type hints support this with the following syntax:

Callable[[ParamType1, ParamType2], ReturnType]

Define an apply function that applies a given function to data.

# from typing import Callable   # Python < 3.9
from collections.abc import Callable

def apply(f: Callable[[str | int], int | None], data: list):
    """ Apply callable object on data. The `Callable[[str | int], int | None]` 
    is the type hints of `parse` we aforementioned
    """
    for d in data:
        print(f(d))

apply(parse, ['hello', 123])
# None
# 123

Class

In Python 3.11⁷, the Self type is proposed, which represents an instance of the current class. We can use Self within the class definition everywhere :) No need to use TypeVar anymore

from typing import Self

class Shape:
    def set_scale(self, scale: float) -> Self:
        self.scale = scale
        return self

Summary

Well, that’s the entire content of this blog. I have covered some of the most common and practical uses of type hints that I have found useful. However, it is not an exhaustive guide, and there are more advanced features such as static protocols⁸ that readers can explore.

Personally, I use type hints only when the benefits outweigh the added complexity. If type hints become too complex, I avoid them unless they provide clear value.

I will also give you some advice based on my experiences🎯:

1. When choosing type annotations, consider what the type can do⁸. The static protocols⁸ are well suited to this requirement. This reminds me of Rust’traits. :)
2. Always make the return type as precise as possible.
3. Even if you pass the type checker, the program is not free of bugs. Software testing is the standard practice to make your programs work as expected.
4. Add the cheatsheet to your bookmark 👍

Refs