How to accept dicts with extra keys?

[john375639]

A lot of times I pass data around using dictionaries. When I receive the data, for example as a function argument, what I want is a dict with that contains some specifics keys. I don't mind if the dict has more data.

So, I tried typed dicts:

Movie = TypedDict('Movie', {'name': str, 'year': int})

This works:

movie = {'name': 'Blade Runner', 'year': 1982}  # type: Movie

But this throws the error Extra key 'has_sequel' for TypedDict "Movie":

movie = {'name': 'Blade Runner', 'year': 1982, 'has_sequel': True}  # type: Movie

I can understand that you can't replace for the first value, because the result of keys or items is different.

But if I am only interested in having those keys, not iterating or other stuff, what are my options (if any)?

[ilevkivskyi]

There is a cool new feature called total=False, does it work for you?
Here are some docs http://mypy.readthedocs.io/en/latest/kinds_of_types.html#totality

[gvanrossum]

This looks like a feature request for TypedDict.

[ilevkivskyi]

@gvanrossum Actually I am quite sure total=False covers this, or am I missing something?

[ilevkivskyi]

Hm, I see one problem: potentially, the list of optional keys can be very long (since one still needs to list them).

[gvanrossum]

Sadly no -- total=False allows omitting keys, it doesn't allow extra keys.

# ...imports...
A = TypedDict('A', {'x': int}, total=False)
def f(a: A) -> None:
  print(a['x'])
b: A = {'x': 0, 'y': 0}  # E: Extra key 'y' for TypedDict "A"

[emmatyping]

Yes, I think this is a valid feature request, as enumerating keys in the TypedDict could be painful.

[JukkaL]

This is already supported for function arguments:

from mypy_extensions import TypedDict

A = TypedDict('A', {'x': int})
B = TypedDict('B', {'x': int, 'y': str})

def f(x: A) -> None: ...

b: B = {'x': 1, 'y': 'foo'}
f(b)  # Ok

The issues seems specific to creating a typed dict -- mypy makes sure that no extra keys are provided (as these could be accidental typos or obsolete key names, for example).

I don't think that changing the semantics of total=False is a good idea, since we'd lose a lot of type safety. One option would be to introduce a new flag. Let's call it allow_extra=True for now. Here is an attempt to define it:

• Don't complain about extra keys when creating a typed dict object if allow_extra is true for the type. Allow arbitrary extra keys and values.
• Allow accessing arbitrary keys, such as x['whatever']. The value type would have to be Any unless the key is explicitly defined for the typed dict.
• Allow setting arbitrary keys with arbitrary values, such as x['whatever'] = <anything>.

These semantics seem kind of arbitrary to me. To seriously consider such as feature I'd like to see more concrete examples where the current behavior is not sufficient. If the structure of a typed dict object is not well specified, the current recommendation is to use Dict[str, Any] instead. TypedDict is pretty restricted by design.

If we had intersection types, a similar effect to my proposed allow_extra flag could perhaps be achieved through Intersection[MyTypedDict, Dict[str, Any]]. We don't have any concrete plans to introduce intersection types, however.

[john375639]

Thanks.

I think that I can work with the current behavior.

JukkaL's example is what I am trying to do. Something like:

from typing import Iterable
from mypy_extensions import TypedDict

NamedThing = TypedDict('NamedThing', {'name': str})
Movie = TypedDict('Movie', {'name': str, 'year': int})
Replicant = TypedDict('Replicant', {'name': str, 'model': str})

def slug(x: NamedThing) -> str:
    return x['name'].lower().replace(' ', '-')

blade_runner: Movie = {'name': 'Blade Runner', 'year': 1982}
roy: Replicant = {'name': 'Roy', 'model': 'Nexus-6'}
things: Iterable[NamedThing] = [blade_runner, roy]

for thing in things:
    print(slug(thing))

When trying Mypy I was directly assigning the values or using variables without annotations, like:

blade_runner: NamedThing = {'name': 'Blade Runner', 'year': 1982}

slug({'name': 'Blade Runner', 'year': 1982})

blade_runner = {'name': 'Blade Runner', 'year': 1982}
slug(blade_runner)

[JukkaL]

I'm closing this since it seems that the current behavior is mostly good enough, and the solution to the larger issue would be messy.