Installation¶

$ pip install typeit

Using CLI tool¶

Once installed, typeit provides you with a CLI tool that allows you to generate a prototype Python structure of a JSON/YAML data that your app operates with.

For example, try the following snippet in your shell:

$ echo '{"first-name": "Hello", "initial": null, "last_name": "World"}' | typeit gen

You should see output similar to this:

from typing import Any, NamedTuple, Optional, Sequence
from typeit import TypeConstructor


class Main(NamedTuple):
    first_name: str
    initial: Optional[Any]
    last_name: str


overrides = {
    Main.first_name: 'first-name',
}


mk_main, serialize_main = TypeConstructor & overrides ^ Main

You can use this snippet as a starting point to improve further. For instance, you can clarify the Optional type of the Main.initial attribute, and rename the whole structure to better indicate the nature of the data:

# ... imports ...

class Person(NamedTuple):
    first_name: str
    initial: Optional[str]
    last_name: str


overrides = {
    Person.first_name: 'first-name',
}


mk_person, serialize_person = TypeConstructor & overrides ^ Person

typeit will handle creation of the constructor mk_person :: Dict -> Person and the serializer serialize_person :: Person -> Dict for you.

TypeConstructor & overrides produces a new type constructor that takes overrides into consideration, and TypeConstructor ^ Person reads as “type constructor applied on the Person structure” and essentially is the same as TypeConstructor(Person), but doesn’t require parentheses around overrides (and extensions):

(TypeConstructor & overrides & extension & ...)(Person)

If you don’t like this combinator syntax, you can use a more verbose version that does exactly the same thing:

TypeConstructor.override(overrides).override(extension).apply_on(Person)

Overrides¶

As you might have noticed in the example above, typeit generated a snippet with a dictionary called overrides, which is passed to the TypeConstructor alongside our Person type:

overrides = {
    Person.first_name: 'first-name',
}


mk_person, serialize_person = TypeConstructor & overrides ^ Person

This is the way we can indicate that our Python structure has different field names than the original JSON payload. typeit code generator created this dictionary for us because the first-name attribute of the JSON payload is not a valid Python variable name (dashes are not allowed in Python variables).

Instead of relying on automatic dasherizing of this attribute (for instance, with a help of inflection package), which rarely works consistently across all possible corner cases, typeit explicitly provides you with a reference point in the code, that you can track and refactor with Intelligent Code Completion tools, should that necessity arise (but this doesn’t meant that you cannot apply a global rule to override all attribute names, please refer to the Constructor Flags section of this manual for more details).

You can use the same overrides object to specify rules for attributes of any nested types, for instance:

class Address(NamedTuple):
    street: str
    city: str
    postal_code: str


class Person(NamedTuple):
    first_name: str
    initial: Optional[str]
    last_name: str
    address: Optional[Address]


overrides = {
    Person.first_name: 'first-name',
    Address.postal_code: 'postal-code',
}


mk_person, serialize_person = TypeConstructor & overrides ^ Person

Note

Because dataclasses do not provide class-level property attributes (Person.first_name in the example above), the syntax for their overrides needs to be slightly different:

@dataclass
class Person:
    first_name: str
    initial: Optional[str]
    last_name: str
    address: Optional[Address]


overrides = {
    (Person, 'first_name'): 'first-name',
    (Address, 'postal_code'): 'postal-code',
}

Handling errors¶

Let’s take the snippet above and use it with incorrect input data. Here is how we would handle the errors:

invalid_data = {'initial': True}

try:
    person = mk_person(invalid_data)
except typeit.Error as err:
    for e in err:
        print(f'Invalid data for `{e.path}`; {e.reason}: {repr(e.sample)} was passed')

If you run it, you will see an output similar to this:

Invalid data for `first-name`; Required: None was passed
Invalid data for `initial`; None of the expected variants matches provided data: True was passed
Invalid data for `last_name`; Required: None was passed

Instances of typeit.Error adhere iterator interface that you can use to iterate over all parsing errors that caused the exception.

Supported types¶

bool
int
float
bytes
str
dict
set and frozenset
typing.Any passes any value as is
typing.NewType
typing.Union including nested structures
typing.Sequence, typing.List including generic collections with typing.TypeVar;
typing.Set and typing.FrozenSet
typing.Tuple
typing.Dict
typing.Mapping
typing.Literal (typing_extensions.Literal on Python prior 3.8);
typing.Generic[T, U, ...]
typeit.sums.SumType
typeit.custom_types.JsonString - helpful when dealing with JSON strings encoded into JSON strings;
enum.Enum derivatives
pathlib.Path derivatives
pyrsistent.typing.PVector
pyrsistent.typing.PMap
Forward references and recursive definitions
Regular classes with annotated __init__ methods (dataclasses.dataclass are supported as a consequence of this).

Sum Type¶

There are many ways to describe what a Sum Type (Tagged Union) is. Here’s just a few of them:

Wikipedia describes it as “a data structure used to hold a value that could take on several different, but fixed, types. Only one of the types can be in use at any one time, and a tag explicitly indicates which one is in use. It can be thought of as a type that has several “cases”, each of which should be handled correctly when that type is manipulated”;
or you can think of Sum Types as data types that have more than one constructor, where each constructor accepts its own set of input data;
or even simpler, as a generalized version of Enums, with some extra features.

typeit provides a limited implementation of Sum Types, that have functionality similar to default Python Enums, plus the ability of each tag to hold a value.

A new SumType is defined with the following signature:

from typeit.sums import SumType

class Payment(SumType):
    class Cash:
        amount: Money

    class Card:
        amount: Money
        card: CardCredentials

    class Phone:
        amount: Money
        provider: MobilePaymentProvider

    class JustThankYou:
        pass

Payment is a new Tagged Union (which is another name for a Sum Type, remember), that consists of four distinct possibilities: Cash, Card, Phone, and JustThankYou. These possibilities are called tags (or variants, or constructors) of Payment. In other words, any instance of Payment is either Cash or Card or Phone or JustThankYou, and is never two or more of them at the same time.

Now, let’s observe the properties of this new type:

>>> adam_paid = Payment.Cash(amount=Money('USD', 10))
>>> jane_paid = Payment.Card(amount=Money('GBP', 8),
...                          card=CardCredentials(number='1234 5678 9012 3456',
...                                               holder='Jane Austen',
...                                               validity='12/24',
...                                               secret='***'))
>>> fred_paid = Payment.JustThankYou()
>>>
>>> assert type(adam_paid) is type(jane_paid) is type(fred_paid) is Payment
>>>
>>> assert isinstance(adam_paid, Payment)
>>> assert isinstance(jane_paid, Payment)
>>> assert isinstance(fred_paid, Payment)
>>>
>>> assert isinstance(adam_paid, Payment.Cash)
>>> assert isinstance(jane_paid, Payment.Card)
>>> assert isinstance(fred_paid, Payment.JustThankYou)
>>>
>>> assert not isinstance(adam_paid, Payment.Card)
>>> assert not isinstance(adam_paid, Payment.JustThankYou)
>>>
>>> assert not isinstance(jane_paid, Payment.Cash)
>>> assert not isinstance(jane_paid, Payment.JustThankYou)
>>>
>>> assert not isinstance(fred_paid, Payment.Cash)
>>> assert not isinstance(fred_paid, Payment.Card)
>>>
>>> assert not isinstance(adam_paid, Payment.Phone)
>>> assert not isinstance(jane_paid, Payment.Phone)
>>> assert not isinstance(fred_paid, Payment.Phone)
>>>
>>> assert Payment('Phone') is Payment.Phone
>>> assert Payment('phone') is Payment.Phone
>>> assert Payment(Payment.Phone) is Payment.Phone
>>>
>>> paid = Payment(adam_paid)
>>> assert paid is adam_paid

As you can see, every variant constructs an instance of the same type Payment, and yet, every instance is identified with its own tag. You can use this tag to branch your business logic, like in a function below:

def notify_restaurant_owner(channel: Broadcaster, payment: Payment):
    if isinstance(payment, Payment.JustThankYou):
        channel.push(f'A customer said Big Thank You!')
    else:  # Cash, Card, Phone instances have the `payment.amount` attribute
        channel.push(f'A customer left {payment.amount}!')

And, of course, you can use Sum Types in signatures of your serializable data:

from typing import NamedTuple, Sequence
from typeit import TypeConstructor

class Payments(NamedTuple):
    latest: Sequence[Payment]

mk_payments, serialize_payments = TypeConstructor ^ Payments

json_ready = serialize_payments(Payments(latest=[adam_paid, jane_paid, fred_paid]))
payments = mk_payments(json_ready)

Constructor Flags¶

Constructor flags allow you to define global overrides that affect all structures (toplevel and nested) in a uniform fashion.

typeit.flags.GlobalNameOverride - useful when you want to globally modify output field names from pythonic snake_style to another naming convention scheme (camelCase, dasherized-names, etc). Here’s a few examples:

import inflection

class FoldedData(NamedTuple):
    field_three: str

class Data(NamedTuple):
    field_one: str
    field_two: FoldedData

constructor, to_serializable = TypeConstructor & GlobalNameOverride(inflection.camelize) ^ Data

data = Data(field_one='one',
            field_two=FoldedData(field_three='three'))

serialized = to_serializable(data)

the serialized dictionary will look like

{
    'FieldOne': 'one',
    'FieldTwo': {
        'FieldThree': 'three'
    }
}

typeit.flags.NonStrictPrimitives - disables strict checking of primitive types. With this flag, a type constructor for a structure with a x: int attribute annotation would allow input values of x to be strings that could be parsed as integer numbers. Without this flag, the type constructor will reject those values. The same rule is applicable to combinations of floats, ints, and bools:

construct, deconstruct = TypeConstructor ^ int
nonstrict_construct, nonstrict_deconstruct = TypeConstructor & NonStrictPrimitives ^ int

construct('1')            # raises typeit.Error
construct(1)              # OK
nonstrict_construct('1')  # OK
nonstrict_construct(1)    # OK

typeit.flags.SumTypeDict - switches the way SumType is parsed and serialized. By default, SumType is represented as a tuple of (<tag>, <payload>) in a serialized form. With this flag, it will be represented and parsed from a dictionary:

{
    <TAG_KEY>: <tag>,
    <payload>
}

i.e. the tag and the payload attributes will be merged into a single mapping, where <TAG_KEY> is the key by which the <tag> could be retrieved and set while parsing and serializing. The default value for TAG_KEY is type, but you can override it with the following syntax:

# Use "_type" as the key by which SumType's tag can be found in the mapping
mk_sum, serialize_sum = TypeConstructor & SumTypeDict('_type') ^ int

Here’s an example how this flag changes the behaviour of the parser:

>>> class Payment(typeit.sums.SumType):
...    class Cash:
...        amount: str
...    class Card:
...        number: str
...        amount: str
...
>>> _, serialize_std_payment = typeit.TypeConstructor ^ Payment
>>> _, serialize_dict_payment = typeit.TypeConstructor & typeit.flags.SumTypeDict ^ Payment
>>> _, serialize_dict_v2_payment = typeit.TypeConstructor & typeit.flags.SumTypeDict('$type') ^ Payment
>>>
>>> payment = Payment.Card(number='1111 1111 1111 1111', amount='10')
>>>
>>> print(serialize_std_payment(payment))
('card', {'number': '1111 1111 1111 1111', 'amount': '10'})

>>> print(serialize_dict_payment(payment))
{'type': 'card', 'number': '1111 1111 1111 1111', 'amount': '10'}

>>> print(serialize_dict_v2_payment(payment))
{'$type': 'card', 'number': '1111 1111 1111 1111', 'amount': '10'}

Extensions¶

See a cookbook for Structuring Docker Compose Config.

Installation¶

Using CLI tool¶

Overrides¶

Handling errors¶

Supported types¶

Sum Type¶

Constructor Flags¶

Extensions¶

typeit

Navigation

Related Topics