marp
true

footer: Python と型ヒントとその使い方 - PyCon Kyushu in Okinawa 2019 May 18th slidenumbers: true

Python と型ヒント

とその使い方

PyCon Kyushu in Okinawa 2019 May 18th

in Ryukyu University, Okinawa

[fit] こんにちは

Who are you(お前誰よ)?

中村真人(masahito)
株式会社ヌーラボ Typetalk チーム所属 Software Developer
- use Scala and JavaScript and Python 🐍
Python ❤️ and ☕ ❤️

^ May I take a picture from the stage? ^ How many people come from Japan?

Masato Nakamura(masahito)

Speaker
- Python Conference Hong Kong 2018
- Python Conference Kyushu 2018 in Japan
- Python Conference Taiwan 2017

Nulab provide collaboration softwares.

Today's Goal

Type Hints とは?について説明できる Type Hints を今日から使えるように Python2 to 3 の話も少しだけします

Not covering today

型とは? Technical Detail

Today's Outline

Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints

Today's Outline

👉 Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints

Introduction of Type Hints

A. What is Type Hints function
B. History of Type Hints
C. typing module

What is Type Hints function

We can write more Readable Codes.

We can annotate our codes only.

^ I want to explain using examples to everyone

What is Type Hints function

Traditional Python code

def twice(num):
    return num * 2

using Type Hints

def twice(num: int) -> int:
    return num * 2

What is Type Hints function

also we can use Class!

class Car:
    def __init__(self) -> None:
        self.seats = 4
c: Car = Car()

What is Type Hints function

using Type Hints

def twice(num: int) -> int:
    return num * 2

C-language Code

int twice(int num) {
   return num * 2;
}

Difference of Type Hints and C code

If you set wrong Type,We cannot show runtime error

def twice(num: int) -> int:
    return num * 2.0

# expected `int` but use `str` 😓
print(twice("hello"))

$ python set_wrong_type.py
$ hellohello ## 😑

[fit] hmm, What is TypeHints 🤔???

[fit] We can annotate our codes only.

But You use library, e.g. mypy

$ mypy --python-version 3.7 example.py
example.py:5: error: Argument 1 to "twice" has incompatible type
"str"; expected "int"

[fit] What is good point for checking type🤔???

[fit] We can check our code without run codes.

so we can re-write our Python2 code to Python3 😄 more easily
We can do it !!

History of Type Hints

implement	PEP
Python 3.0	3107
Python 3.5	484
Ptyhon 3.6	526
Python 3.7	560, 561, 562,563

Traditional Python code(Python2)

def greeting(name):
    return 'Hello ' + name

greet = greeting("Guido")

What is name🤔?
What does greeting return🤔?

PEP 3107(Python 3.0)

パラメータと戻り値の両方で、オプションで関数アノテーションを追加することができます。フリーテキストを書くことができます。

def compile(source: "something compilable",
            filename: "where the compilable thing comes from",
            mode: "is this a single statement or a suite?"):

セマンティクスは意図的に未定義に

PEP 484(Python 3.5)

コミュニティが標準ライブラリ内の標準化された語彙およびベースラインツールから恩恵を受けることができるように、静的タイプ分析のための十分なサードパーティの使用方法を用意した

def greeting(name: str) -> str:
    return 'Hello ' + name

PEP-484 Type Hints

Points in PEP484

Python は dynamically typed language のままであり、作者は型のヒントを必須にしたいという欲求を全く持っていません。

コード生成ではありません
コンパイラがコードをどのように準拠しているかには影響しません。
型チェック後、実行時にコードが壊れる可能性があります。
Python を静的型にするつもりはない

inconvinient of PEP484

def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!

Hmm , でももうちょい簡潔に定義を書きたい

PEP 526(Python 3.6)

# PEP 484
def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!

# pep 526
def greeting(name: str) -> str:
    return 'Hello ' + name

greet:str = greeting("Masato") # make greeting message!

typing module

What is the `typing module`

PEP 484 が 3.5 で実装されたときに導入されたモジュール
List,Dict,Union などのよく使われる型を Python に提供します
namedtupleのようなデータ構造を簡単に導入できる

from typing import List

a: List[int] = [1,2,3]
path: Optional[str] = None # Path to module sourde

Union

i = 1           # Infer type "int" for i
l = [1, 2]      # Infer type "List[int]" for l


from typing import Union

x: Union[int, str] = 1
x: Union[int, str] = 1.1  # Error

NamedTuple

In Python 3.5(PEP 484)

Point = namedtuple('Point', ['x', 'y'])
p = Point(x=1, y=2)

In Python 3.6(PEP 526)

from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int
p = Point(x=1, y=2)

Today's Outline

Ⅰ. Introduction of Type Hints 👉 Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints

Usage Of Type Hints

A. How to write
B. How to get Type Hints Info

How to write

PEP526 style

def greeting(name: str) -> str:
    return 'Hello ' + name

greet: str = greeting("Masato")

difference of PEP484 and PEP526

# pep 484
child # type: bool # cannot allow
if age < 18:
    child = True  # type: bool # It's OK
else:
    child = False # type: bool # It's OK

# pep 526
child: bool # OK
if age < 18:
    child = True # No need to Write bool
else:
    child = False

backward compatibility

# We can write pep484 style code in 3.6 backward compatiblity
hour = 24 # type: int

# PEP 526 style
hour: int; hour = 24

hour: int = 24

recommend using Type Hints

PEP3107 style is also OK

>>> alice: 'well done' = 'A+'
>>> bob: 'what a shame' = 'F-'
>>> __annotations__
{'alice': 'well done', 'bob': 'what a shame'}

でもなるべくこういう書き方は Type Hints でのみ使いましょうね.

How to get Type Hints Info

Q. where is `variable annotation`

A. at `annotaitons`

How to get Type Hints Info

example

>>> answer:int = 42
>>> __annotations__
{'answer': <class 'int'>}

Note for Class

We can only find Class variables.

>>> class Car:
...     stats: ClassVar[Dict[str, int]] = {}
...     def __init__(self) -> None:
...         self.seats = 4
>>> c: Car = Car()
>>> c.__annotations__
{'stats': typing.ClassVar[typing.Dict[str, int]]}
# only ClassVar!

We cannot get instance's variables!

Today's Outline

Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints 👉 Ⅲ. Benefits of Type Hints

Benefits of Type Hints

A. Code Style B. code completion C. statistic type analysis D. Our story

Code Style

Simple is best
Explicit is better than Implicit.

def greeting(name: str) -> str:
    return 'Hello ' + name

We can check this by Type Hints

def greeting(name: str) -> str:
    return 42

Code Completion

We can use code completion in

Editor
- Visual Studio code
IDE
- PyCharm(IntelliJ)

example

statistic type analysis

for check variable type(no runable code)
python has some tools
- mypy
- pytypes (not talk)

example

mypy

https://github.com/python/mypy
made by JukkaL & Guido
can output report
We can use!(PEP526 style)

How to install mypy

$ pip install mypy

Notice requires Python 3.5 or later to run.

mypy example

code(example.py)

from typing import NamedTuple

class Employee(NamedTuple):
    name: str
    id: int

emp: Employee = Employee(name='Guido', id='x')

$ mypy --python-version 3.7 example.py
example.py:16: error: Argument 2 to "Employee" has incompatible type
"str"; expected "int"

mypy can use in Python2

class MyClass(object):

    # For instance methods, omit `self`.
    def my_method(self, num, str1):
        # type: (int, str) -> str
        return num * str1

    def __init__(self):
        # type: () -> None
        pass

x = MyClass() # type: MyClass

Missing imports

import しているライブラリが type annotation を提供してないとエラーが出る 😢

main.py:1: error: No library stub file for standard library module 'antigravity'
main.py:2: error: No library stub file for module 'flask'
main.py:3: error: Cannot find module named 'this_module_does_not_exist'

このエラーは抑えられる(現状だとほぼ必須 😢)

mypy --ignore-missing-imports

Continuous Integration

Jenkins
Travis CI
CircleCi

Continuous Integration

we can run mypy on CI systems
we can get results
- If you develop in a team, there are many benefits
- ex: If someone issues a pull request for your project, you can review their code

sample for CI

Sample

example(error)

example(success)

Benefits for us(Nulab)

We use Fabric v1 and Ansible for deployment
But it is for Only Python2
Python2 will finished at 2020(Tokyo Olympic Year)
We thought we should convert Python2 to Python3

Recently, that date has been updated to January 1, 2020.

We want to use Python3

We use Fabric3
- It's Fabric for Python2.7 and Python3
- https://pypi.python.org/pypi/Fabric3
We try to check this, That's good for run

We(Nulab) use Python3 & Fabric3 & mypy

We use TypeHints for rewriting our code
We got it! -> Only 2days
We deploy the new version of Out App every week.
beta version every day!

Python2 to 3 で主にやったこと

まずは Python2.7 で動くか検証する
six を入れる(まずは Python2 と 3 で動くように)
2to3 コマンドで何が変わるかをまずは確認する

Python2 to 3 で主にやったこと

重要な処理はリファクタリングを入れて、テスト追加、関数化を進める
- デプロイスクリプトはべたっとコードを書きがち
- なるべく関数に分割していく
- ここで型ヒントを入れる-> 型チェックするテスト書くの辛い
  - でも頑張りすぎない

def is_modified(path, backup='bak'):
    # type: (Text, str) -> bool
    """
    該当の path とバックアップファイルを比較し変更があったかを返す
    """
    with settings(hide('warnings'), warn_only=True):
        result = sudo('diff -u %s.%s %s' % (path, backup, path))
    return result.return_code == 1
```

---

# Those are the Benefits of Type Hints

---

# Today's Outline

✅ 1. Introduction of Type Hints
✅ 2. Usage of Type Hints
✅ 3. Benefits of Type Hints

---

# 型ヒントってどんなものか理解できました?

---

# Thank you！

@masahito

---

## おまけ

- type-hints 周りの 今議論されている機能 pep
  - PEP 586 -- Literal Types
  - PEP 591 -- Adding a final qualifier to typing
- mypy の機能拡張 (plugin)
- typing module の拡張の話

---

## PEP 586 -- Literal Types

```python
from typing import Literal

def accepts_only_four(x: Literal[4]) -> None:
    pass

accepts_only_four(4)   # OK
accepts_only_four(19)  # Rejected

PEP 591 -- Adding a final qualifier to typing

from typing import Final

RATE: Final = 3000

class Base:
    DEFAULT_ID: Final = 0

RATE = 300  # Error: can't assign to final attribute
Base.DEFAULT_ID = 1  # Error: can't override a final attribute

mypy plugin

mypy に色々機能拡張する方法が議論され、ている
plugin として mypy が機能を提供、拡張したいときは plugin を書いて読み込ませる
将来的に sqlAlchemy のテーブル定義を取り出したり etc 用の plugin を書いたりとかもできそう

mypy plugin

dropbox が SqlAlchemy 用の mypy-stub を提供してくれてます
- python/mypy#3538
- https://github.com/dropbox/sqlalchemy-stubs

mypy と PEP561

PEP 561 is
- 各ライブラリごとの型情報を外だしできるようにするための取り決め
- これのおかげで numpy の stub も提供されている
- https://github.com/numpy/numpy-stubs

typing extentions

議論されている typing の機能を追加して検証するための場所 https://github.com/python/typing/tree/master/typing_extensions
Protocol とかとか

User Defined Protocol

from typing import Iterable
from typing_extensions import Protocol

class SupportsClose(Protocol):
    def close(self) -> None:
       ...  # Empty method body (explicit '...')

class Resource:  # No SupportsClose base class!
    # ... some methods ...

    def close(self) -> None:
       self.resource.release()

def close_all(items: Iterable[SupportsClose]) -> None:
    for item in items:
        item.close()

close_all([Resource(), open('some/file')])  # Okay!

DuckTyping 的なメソッドのチェックができるように！！！

masahitojp/test.md

Python と型ヒント

とその使い方

PyCon Kyushu in Okinawa 2019 May 18th

in Ryukyu University, Okinawa

[fit] こんにちは

Who are you(お前誰よ)?

Masato Nakamura(masahito)

Nulab provide collaboration softwares.

Today's Goal

Not covering today

Today's Outline

Today's Outline

Introduction of Type Hints

What is Type Hints function

We can write more Readable Codes.

We can annotate our codes only.

What is Type Hints function

What is Type Hints function

What is Type Hints function

Difference of Type Hints and C code

[fit] hmm, What is TypeHints 🤔???

[fit] We can annotate our codes only.

But You use library, e.g. mypy

[fit] What is good point for checking type🤔???

[fit] We can check our code without run codes.

History of Type Hints

History of Type Hints

Traditional Python code(Python2)

PEP 3107(Python 3.0)

PEP 484(Python 3.5)

Points in PEP484

inconvinient of PEP484

PEP 526(Python 3.6)

typing module

What is the typing module

Today's Outline

Usage Of Type Hints

How to write

difference of PEP484 and PEP526

backward compatibility

recommend using Type Hints

How to get Type Hints Info

Q. where is variable annotation

A. at __annotaitons__

How to get Type Hints Info

Note for Class

Today's Outline

Benefits of Type Hints

Code Style

Code Completion

example

statistic type analysis

statistic type analysis

example

mypy

How to install mypy

mypy example

mypy can use in Python2

Missing imports

Continuous Integration

Continuous Integration

sample for CI

example(error)

example(success)

Benefits for us(Nulab)

We want to use Python3

We(Nulab) use Python3 & Fabric3 & mypy

Python2 to 3 で主にやったこと

Python2 to 3 で主にやったこと

PEP 591 -- Adding a final qualifier to typing

mypy plugin

mypy plugin

mypy と PEP561

typing extentions

User Defined Protocol

Thanks again!

What is the `typing module`

Q. where is `variable annotation`

A. at `annotaitons`