Skip to content
Commits on Source (4)
Metadata-Version: 2.1
Name: mypy
Version: 0.590
Version: 0.600
Summary: Optional static typing for Python
Home-page: http://www.mypy-lang.org/
Author: Jukka Lehtosalo
......
mypy (0.600-1) unstable; urgency=medium
* New upstream release.
* Add manpages for dmypy subcommands, as they are now officially in beta
-- Michael R. Crusoe <michael.crusoe@gmail.com> Sun, 06 May 2018 02:51:30 -0700
mypy (0.590-1) unstable; urgency=medium
* Add basic autopkgtest and enable autopkgtest-pkg-python
......
#!/usr/bin/make -f
SHELL=bash # needed for the <(echo …) process subsitution temporary file
# tricks with help2man
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
......@@ -21,6 +23,43 @@ override_dh_auto_build:
PYTHONPATH=$(PPATH) help2man scripts/dmypy --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--name 'Client for mypy daemon mode' > debian/dmypy.1
sed -i '/\.\.\./d' debian/dmypy.1 # Delete the "..."
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy start' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_start \- Start a mypy daemon") \
> debian/dmypy-start.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy restart' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_restart \- Restart a mypy daemon") \
> debian/dmypy-restart.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy status' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_status \- Show a mypy daemon status") \
> debian/dmypy-status.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy stop' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_stop \- Stop a mypy daemon") \
> debian/dmypy-stop.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy kill' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_kill \- Kill a mypy daemon") \
> debian/dmypy-kill.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy check' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_check \- Type check some Python files with a mypy daemon") \
> debian/dmypy-check.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy recheck' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_recheck \- Type check the same files from the previous run") \
> debian/dmypy-recheck.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy hang' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_hang \- Hang a mypy daemon, as a debug hack") \
> debian/dmypy-hang.1
PYTHONPATH=$(PPATH) help2man 'scripts/dmypy daemon' --no-info \
--version-string="${DEB_VERSION_UPSTREAM}" \
--include <(echo -e "[NAME]\ndmypy_daemon \- Run a mypy daemon in the foreground") \
> debian/dmypy-daemon.1
PYTHONPATH=$(PPATH) help2man 'python${PY3V} scripts/stubgen' --no-info \
--no-discard-stderr --version-string="${DEB_VERSION_UPSTREAM}" \
--name 'Generate draft stubs for Python modules.' > \
......
Additional features
-------------------
Several mypy features are not currently covered by this tutorial,
including the following:
- inheritance between generic classes
- compatibility and subtyping of generic types, including covariance of generic types
- ``super()``
This section discusses various features outside core mypy features.
.. _attrs_package:
......@@ -83,3 +77,157 @@ Caveats/Known Issues
* Method definitions added by mypy currently overwrite any existing method
definitions.
.. _remote-cache:
Using a remote cache to speed up mypy runs
******************************************
Mypy performs type checking *incrementally*, reusing results from
previous runs to speed up successive runs. If you are type checking a
large codebase, mypy can still be sometimes slower than desirable. For
example, if you create a new branch based on a much more recent commit
than the target of the previous mypy run, mypy may have to
process almost every file, as a large fraction of source files may
have changed. This can also happen after you've rebased a local
branch.
Mypy supports using a *remote cache* to improve performance in cases
such as the above. In a large codebase, remote caching can sometimes
speed up mypy runs by a factor of 10, or more.
Mypy doesn't include all components needed to set
this up -- generally you will have to perform some simple integration
with your Continuous Integration (CI) or build system to configure
mypy to use a remote cache. This discussion assumes you have a CI
system set up for the mypy build you want to speed up, and that you
are using a central git repository. Generalizing to different
environments should not be difficult.
Here are the main components needed:
* A shared repository for storing mypy cache files for all landed commits.
* CI build that uploads mypy incremental cache files to the shared repository for
each commit for which the CI build runs.
* A wrapper script around mypy that developers use to run mypy with remote
caching enabled.
Below we discuss each of these components in some detail.
Shared repository for cache files
=================================
You need a repository that allows you to upload mypy cache files from
your CI build and make the cache files available for download based on
a commit id. A simple approach would be to produce an archive of the
``.mypy_cache`` directory (which contains the mypy cache data) as a
downloadable *build artifact* from your CI build (depending on the
capabilities of your CI system). Alternatively, you could upload the
data to a web server or to S3, for example.
Continuous Integration build
============================
The CI build would run a regular mypy build and create an archive containing
the ``.mypy_cache`` directory produced by the build. Finally, it will produce
the cache as a build artifact or upload it to a repository where it is
accessible by the mypy wrapper script.
Your CI script might work like this:
* Run mypy normally. This will generate cache data under the
``.mypy_cache`` directory.
* Create a tarball from the ``.mypy_cache`` directory.
* Determine the current git master branch commit id (say, using
``git rev-parse HEAD``).
* Upload the tarball to the shared repository with a name derived from the
commit id.
Mypy wrapper script
===================
The wrapper script is used by developers to run mypy locally during
development instead of invoking mypy directly. The wrapper first
populates the local ``.mypy_cache`` directory from the shared
repository and then runs a normal incremental build.
The wrapper script needs some logic to determine the most recent
central repository commit (by convention, the ``origin/master`` branch
for git) the local development branch is based on. In a typical git
setup you can do it like this:
.. code::
git merge-base HEAD origin/master
The next step is to download the cache data (contents of the
``.mypy_cache`` directory) from the shared repository based on the
commit id of the merge base produced by the git command above. The
script will decompress the data so that mypy will start with a fresh
``.mypy_cache``. Finally, the script runs mypy normally. And that's all!
Caching with mypy daemon
========================
You can also use remote caching with the :ref:`mypy daemon <mypy_daemon>`.
The remote cache will significantly speed up the the first ``dmypy check``
run after starting or restarting the daemon.
The mypy daemon requires extra fine-grained dependency data in
the cache files which aren't included by default. To use caching with
the mypy daemon, use the ``--cache-fine-grained`` option in your CI
build::
$ mypy --cache-fine-grained <args...>
This flag adds extra information for the daemon to the cache. In
order to use this extra information, you will also need to use the
``--use-fine-grained-cache`` option with ``dymypy start`` or
``dmypy restart``. Example::
$ dmypy start -- --use-fine-grained-cache <options...>
Now your first ``dmypy check`` run should be much faster, as it can use
cache information to avoid processing the whole program.
Refinements
===========
There are several optional refinements that may improve things further,
at least if your codebase is hundreds of thousands of lines or more:
* If the wrapper script determines that the merge base hasn't changed
from a previous run, there's no need to download the cache data and
it's better to instead reuse the existing local cache data.
* If you use the mypy daemon, you may want to restart the daemon each time
after the merge base or local branch has changed to avoid processing a
potentially large number of changes in an incremental build, as this can
be much slower than downloading cache data and restarting the daemon.
* If the current local branch is based on a very recent master commit,
the remote cache data may not yet be available for that commit, as
there will necessarily be some latency to build the cache files. It
may be a good idea to look for cache data for, say, the 5 latest
master commits and use the most recent data that is available.
* If the remote cache is not accessible for some reason (say, from a public
network), the script can still fall back to a normal incremental build.
* You can have multiple local cache directories for different local branches
using the ``--cache-dir`` option. If the user switches to an existing
branch where downloaded cache data is already available, you can continue
to use the existing cache data instead of redownloading the data.
* You can set up your CI build to use a remote cache to speed up the
CI build. This would be particularly useful if each CI build starts
from a fresh state without access to cache files from previous
builds. It's still recommended to run a full, non-incremental
mypy build to create the cache data, as repeatedly updating cache
data incrementally could result in drift over a long time period (due
to a mypy caching issue, perhaps).
......@@ -21,7 +21,7 @@ flag (or its long form ``--help``)::
[--warn-unused-ignores] [--warn-unused-configs]
[--show-error-context] [--no-implicit-optional] [--no-incremental]
[--quick-and-dirty] [--cache-dir DIR] [--cache-fine-grained]
[--skip-version-check] [--strict-optional]
[--skip-version-check] [--no-strict-optional]
[--strict-optional-whitelist [GLOB [GLOB ...]]]
[--always-true NAME] [--always-false NAME] [--junit-xml JUNIT_XML]
[--pdb] [--show-traceback] [--stats] [--inferstats]
......@@ -298,11 +298,14 @@ Here are some more useful flags:
- ``--ignore-missing-imports`` suppresses error messages about imports
that cannot be resolved (see :ref:`follow-imports` for some examples).
- ``--strict-optional`` enables strict checking of ``Optional[...]``
types and ``None`` values. Without this option, mypy doesn't
- ``--no-strict-optional`` disables strict checking of ``Optional[...]``
types and ``None`` values. With this option, mypy doesn't
generally check the use of ``None`` values -- they are valid
everywhere. See :ref:`strict_optional` for more about this feature.
This flag will become the default in the near future.
everywhere. See :ref:`no_strict_optional` for more about this feature.
**Note:** Strict optional checking was enabled by default starting in
mypy 0.600, and in previous versions it had to be explicitly enabled
using ``--strict-optional`` (which is still accepted).
- ``--disallow-untyped-defs`` reports an error whenever it encounters
a function definition without type annotations.
......@@ -352,7 +355,8 @@ Here are some more useful flags:
mode, in order to "warm" the cache. To disable writing the cache,
use ``--cache-dir=/dev/null`` (UNIX) or ``--cache-dir=nul``
(Windows). Cache files belonging to a different mypy version are
ignored.
ignored. This flag can be useful for controlling cache use when using
:ref:`remote caching <remote-cache>`.
.. _quick-mode:
......@@ -428,9 +432,10 @@ Here are some more useful flags:
- ``--config-file CONFIG_FILE`` causes configuration settings to be
read from the given file. By default settings are read from ``mypy.ini``
or ``setup.cfg`` in the current directory. Settings override mypy's
built-in defaults and command line flags can override settings.
See :ref:`config-file` for the syntax of configuration files.
or ``setup.cfg`` in the current directory, or ``.mypy.ini`` in the user home
directory. Settings override mypy's built-in defaults and command line flags
can override settings. See :ref:`config-file` for the syntax of configuration
files.
- ``--junit-xml JUNIT_XML`` will make mypy generate a JUnit XML test
result document with type checking results. This can make it easier
......
......@@ -51,7 +51,7 @@ flagged as an error.
If you don't know what types to add, you can use ``Any``, but beware:
- **One of the values involved has type ``Any``.** Extending the above
- **One of the values involved has type 'Any'.** Extending the above
example, if we were to leave out the annotation for ``a``, we'd get
no error:
......@@ -85,7 +85,7 @@ flagged as an error.
clarity about the latter use ``--follow-imports=error``. You can
read up about these and other useful flags in :ref:`command-line`.
- **A function annotated as returning a non-optional type returns ``None``
- **A function annotated as returning a non-optional type returns 'None'
and mypy doesn't complain**.
.. code-block:: python
......@@ -93,9 +93,8 @@ flagged as an error.
def foo() -> str:
return None # No error!
By default, the ``None`` value is considered compatible with everything. See
:ref:`optional` for details on strict optional checking, which allows mypy to
check ``None`` values precisely, and will soon become default.
You may have disabled strict optional checking (see
:ref:`no_strict_optional` for more).
.. _silencing_checker:
......@@ -131,6 +130,17 @@ The second line is now fine, since the ignore comment causes the name
if we did have a stub available for ``frobnicate`` then mypy would
ignore the ``# type: ignore`` comment and typecheck the stub as usual.
Unexpected errors about 'None' and/or 'Optional' types
------------------------------------------------------
Starting from mypy 0.600, mypy uses
:ref:`strict optional checking <strict_optional>` by default,
and ``None`` is not compatible with non-optional types. It's
easy to switch back to the older behavior where ``None`` was
compatible with arbitrary types (see :ref:`no_strict_optional`).
Types of empty collections
--------------------------
......
......@@ -4,9 +4,10 @@ The mypy configuration file
===========================
Mypy supports reading configuration settings from a file. By default
it uses the file ``mypy.ini`` (with fallback to ``setup.cfg``) in the
current directory; the ``--config-file`` command-line flag can be used to
read a different file instead (see :ref:`--config-file <config-file-flag>`).
it uses the file ``mypy.ini`` with fallback to ``setup.cfg`` in the current
directory, or ``.mypy.ini`` in the user home directory if none of them are
found; the ``--config-file`` command-line flag can be used to read a different
file instead (see :ref:`--config-file <config-file-flag>`).
It is important to understand that there is no merging of configuration
files, as it would lead to ambiguity. The ``--config-file`` flag
......@@ -29,10 +30,15 @@ characters.
the global flags. The ``setup.cfg`` file is an exception to this.
- Additional sections named ``[mypy-PATTERN1,PATTERN2,...]`` may be
present, where ``PATTERN1``, ``PATTERN2`` etc. are `fnmatch patterns
<https://docs.python.org/3.6/library/fnmatch.html>`_
separated by commas. These sections specify additional flags that
only apply to *modules* whose name matches at least one of the patterns.
present, where ``PATTERN1``, ``PATTERN2``, etc., are comma-separated
patterns of the form ``dotted_module_name`` or ``dotted_module_name.*``.
These sections specify additional flags that only apply to *modules*
whose name matches at least one of the patterns.
A pattern of the form ``dotted_module_name`` matches only the named module,
while ``dotted_module_name.*`` matches ``dotted_module_name`` and any
submodules (so ``foo.bar.*`` would match all of ``foo.bar``,
``foo.bar.baz``, and ``foo.bar.baz.quux``).
.. note::
......@@ -86,9 +92,6 @@ The following global flags may only be set in the global section
per-module sections in the config file that didn't match any
files processed in the current run.
- ``strict_optional`` (Boolean, default False) enables experimental
strict Optional checks.
- ``scripts_are_modules`` (Boolean, default False) makes script ``x``
become module ``x`` instead of ``__main__``. This is useful when
checking multiple scripts in a single run.
......@@ -137,8 +140,12 @@ overridden by the pattern sections matching the module name.
.. note::
If multiple pattern sections match a module they are processed in
order of their occurrence in the config file.
If multiple pattern sections match a module, the options from the
most specific section are used where they disagree. This means
that ``foo.bar`` will take values from sections with the patterns
``foo.bar``, ``foo.bar.*``, and ``foo.*``, but when they specify
different values, it will use values from ``foo.bar`` before
``foo.bar.*`` before ``foo.*``.
- ``follow_imports`` (string, default ``normal``) directs what to do
with imports when the imported module is found as a ``.py`` file and
......@@ -170,6 +177,13 @@ overridden by the pattern sections matching the module name.
- ``almost_silent`` (Boolean, deprecated) equivalent to
``follow_imports=skip``.
- ``strict_optional`` (Boolean, default True) enables or disables
strict Optional checks. If False, mypy treats ``None`` as
compatible with every type.
**Note::** This was False by default
in mypy versions earlier than 0.600.
- ``disallow_any_unimported`` (Boolean, default false) disallows usage of types that come
from unfollowed imports (such types become aliases for ``Any``).
......
......@@ -3,17 +3,13 @@ Duck type compatibility
In Python, certain types are compatible even though they aren't subclasses of
each other. For example, ``int`` objects are valid whenever ``float`` objects
are expected. Mypy supports this idiom via *duck type compatibility*. As of
now, this is only supported for a small set of built-in types:
are expected. Mypy supports this idiom via *duck type compatibility*. This is
supported for a small set of built-in types:
* ``int`` is duck type compatible with ``float`` and ``complex``.
* ``float`` is duck type compatible with ``complex``.
* In Python 2, ``str`` is duck type compatible with ``unicode``.
.. note::
Mypy support for Python 2 is still work in progress.
For example, mypy considers an ``int`` object to be valid whenever a
``float`` object is expected. Thus code like this is nice and clean
and also behaves as expected:
......@@ -26,6 +22,12 @@ and also behaves as expected:
n = 90 # Inferred type 'int'
print(degrees_to_radians(n)) # Okay!
You can also often use :ref:`protocol-types` to achieve a similar effect in
a more principled and extensible fashion. Protocols don't apply to
cases like ``int`` being compatible with ``float``, since ``float`` is not
a protocol class but a regular, concrete class, and many standard library
functions expect concrete instances of ``float`` (or ``int``).
.. note::
Note that in Python 2 a ``str`` object with non-ASCII characters is
......
......@@ -29,6 +29,7 @@ Mypy is a static type checker for Python.
additional_features
command_line
config_file
mypy_daemon
python36
installed_packages
faq
......
......@@ -345,139 +345,239 @@ narrow down the type to a specific type:
f('x') # OK
f(1.1) # Error
.. _optional:
.. _strict_optional:
The type of None and optional types
***********************************
Optional types and the None type
********************************
Mypy treats the type of ``None`` as special. ``None`` is a valid value
for every type, which resembles ``null`` in Java. Unlike Java, mypy
doesn't treat primitives types
specially: ``None`` is also valid for primitive types such as ``int``
and ``float``.
You can use the ``Optional`` type modifier to define a type variant
that allows ``None``, such as ``Optional[int]`` (``Optional[X]`` is
the preferred shorthand for ``Union[X, None]``):
.. note::
.. code-block:: python
from typing import Optional
def strlen(s: str) -> Optional[int]:
if not s:
return None # OK
return len(s)
See :ref:`strict_optional` for an experimental mode which allows
mypy to check ``None`` values precisely.
def strlen_invalid(s: str) -> int:
if not s:
return None # Error: None not compatible with int
return len(s)
Most operations will not be allowed on unguarded ``None`` or ``Optional``
values:
.. code-block:: python
def my_inc(x: Optional[int]) -> int:
return x + 1 # Error: Cannot add None and int
Instead, an explicit ``None`` check is required. Mypy has
powerful type inference that lets you use regular Python
idioms to guard against ``None`` values. For example, mypy
recognizes ``is None`` checks:
.. code-block:: python
def my_inc(x: Optional[int]) -> int:
if x is None:
return 0
else:
# The inferred type of x is just int here.
return x + 1
Mypy will infer the type of ``x`` to be ``int`` in the else block due to the
check against ``None`` in the if condition.
Other supported checks for guarding against a ``None`` value include
``if x is not None``, ``if x`` and ``if not x``. Additionally, mypy understands
``None`` checks within logical expressions:
.. code-block:: python
def concat(x: Optional[str], y: Optional[str]) -> Optional[str]:
if x is not None and y is not None:
# Both x and y are not None here
return x + y
else:
return None
Sometimes mypy doesn't realize that a value is never ``None``. This notably
happens when a class instance can exist in a partially defined state,
where some attribute is initialized to ``None`` during object
construction, but a method assumes that the attribute is no longer ``None``. Mypy
will complain about the possible ``None`` value. You can use
``assert x is not None`` to work around this in the method:
.. code-block:: python
class Resource:
path: Optional[str] = None
def initialize(self, path: str) -> None:
self.path = path
def read(self) -> str:
# We require that the object has been initialized.
assert self.path is not None
with open(self.path) as f: # OK
return f.read()
r = Resource()
r.initialize('/foo/bar')
r.read()
When initializing a variable as ``None``, ``None`` is usually an
empty place-holder value, and the actual value has a different type.
This is why you need to annotate an attribute in a case like this:
This is why you need to annotate an attribute in a cases like the class
``Resource`` above:
.. code-block:: python
class A:
class Resource:
path: Optional[str] = None
...
This also works for attributes defined within methods:
.. code-block:: python
class Counter:
def __init__(self) -> None:
self.count = None # type: int
self.count: Optional[int] = None
As a special case, you can use a non-optional type when initializing an
attribute to ``None`` inside a class body *and* using a type comment,
since when using a type comment, an initializer is syntacticaly required,
and ``None`` is used as a dummy, placeholder initializer:
.. code-block:: python
from typing import List
class Container:
items = None # type: List[str] # OK (only with type comment)
This is not a problem when using variable annotations, since no initializer
is needed:
.. code-block:: python
from typing import List
Mypy will complain if you omit the type annotation, as it wouldn't be
able to infer a non-trivial type for the ``count`` attribute
otherwise.
class Container:
items: List[str] # No initializer
Mypy generally uses the first assignment to a variable to
infer the type of the variable. However, if you assign both a ``None``
value and a non-``None`` value in the same scope, mypy can often do
the right thing:
value and a non-``None`` value in the same scope, mypy can usually do
the right thing without an annotation:
.. code-block:: python
def f(i: int) -> None:
n = None # Inferred type int because of the assignment below
n = None # Inferred type Optional[int] because of the assignment below
if i > 0:
n = i
...
Often it's useful to know whether a variable can be
``None``. For example, this function accepts a ``None`` argument,
but it's not obvious from its signature:
.. code-block:: python
def greeting(name: str) -> str:
if name:
return 'Hello, {}'.format(name)
else:
return 'Hello, stranger'
Sometimes you may get the error "Cannot determine type of <something>". In this
case you should add an explicit ``Optional[...]`` annotation (or type comment).
print(greeting('Python')) # Okay!
print(greeting(None)) # Also okay!
.. note::
Mypy lets you use ``Optional[t]`` to document that ``None`` is a
valid argument type:
``None`` is a type with only one value, ``None``. ``None`` is also used
as the return type for functions that don't return a value, i.e. functions
that implicitly return ``None``.
.. code-block:: python
.. note::
from typing import Optional
The Python interpreter internally uses the name ``NoneType`` for
the type of ``None``, but ``None`` is always used in type
annotations. The latter is shorter and reads better. (Besides,
``NoneType`` is not even defined in the standard library.)
def greeting(name: Optional[str]) -> str:
if name:
return 'Hello, {}'.format(name)
else:
return 'Hello, stranger'
.. note::
Mypy treats this as semantically equivalent to the previous example,
since ``None`` is implicitly valid for any type, but it's much more
useful for a programmer who is reading the code. You can equivalently
use ``Union[str, None]``, but ``Optional`` is shorter and more
idiomatic.
``Optional[...]`` *does not* mean a function argument with a default value.
However, if the default value of an argument is ``None``, you can use
an optional type for the argument, but it's not enforced by default.
You can use the ``--no-implicit-optional`` command-line option to stop
treating arguments with a ``None`` default value as having an implicit
``Optional[...]`` type. It's possible that this will become the default
behavior in the future.
.. note::
.. _no_strict_optional:
``None`` is also used as the return type for functions that don't
return a value, i.e. that implicitly return ``None``. Mypy doesn't
use ``NoneType`` for this, since it would
look awkward, even though that is the real name of the type of ``None``
(try ``type(None)`` in the interactive interpreter to see for yourself).
Disabling strict optional checking
**********************************
.. _strict_optional:
Mypy also has an option to treat ``None`` as a valid value for every
type (in case you know Java, it's useful to think of it as similar to
the Java ``null``). In this mode ``None`` is also valid for primitive
types such as ``int`` and ``float``, and ``Optional[...]`` types are
not required.
Strict optional type and None checking
***************************************************
The mode is enabled through the ``--no-strict-optional`` command-line
option. In mypy versions before 0.600 this was the default mode. You
can enable this option explicitly for backward compatibility with
earlier mypy versions, in case you don't want to introduce optional
types to your codebase yet.
Currently, ``None`` is a valid value for each type, similar to
``null`` or ``NULL`` in many languages. However, you can use the
``--strict-optional`` command line option
(which will become the default in the near future)
to tell mypy that types should not include ``None``
by default. The ``Optional`` type modifier is then used to define
a type variant that includes ``None``, such as ``Optional[int]``:
It will cause mypy to silently accept some buggy code, such as
this example -- it's not recommended if you can avoid it:
.. code-block:: python
from typing import Optional
def inc(x: int) -> int:
return x + 1
def f() -> Optional[int]:
return None # OK
x = inc(None) # No error reported by mypy if strict optional mode disabled!
def g() -> int:
...
return None # Error: None not compatible with int
However, making code "optional clean" can take some work! You can also use
:ref:`the mypy configuration file <config-file>` to migrate your code
to strict optional checking one file at a time, since there exists
the :ref:`per-module flag <per-module-flags>` ``strict_optional`` to
control strict optional mode.
Also, most operations will not be allowed on unguarded ``None``
or ``Optional`` values:
Often it's still useful to document whether a variable can be
``None``. For example, this function accepts a ``None`` argument,
but it's not obvious from its signature:
.. code-block:: python
def f(x: Optional[int]) -> int:
return x + 1 # Error: Cannot add None and int
def greeting(name: str) -> str:
if name:
return 'Hello, {}'.format(name)
else:
return 'Hello, stranger'
Instead, an explicit ``None`` check is required. Mypy has
powerful type inference that lets you use regular Python
idioms to guard against ``None`` values. For example, mypy
recognizes ``is None`` checks:
print(greeting('Python')) # Okay!
print(greeting(None)) # Also okay!
You can still use ``Optional[t]`` to document that ``None`` is a
valid argument type, even if strict ``None`` checking is not
enabled:
.. code-block:: python
def f(x: Optional[int]) -> int:
if x is None:
return 0
from typing import Optional
def greeting(name: Optional[str]) -> str:
if name:
return 'Hello, {}'.format(name)
else:
# The inferred type of x is just int here.
return x + 1
return 'Hello, stranger'
Mypy will infer the type of ``x`` to be ``int`` in the else block due to the
check against ``None`` in the if condition.
Mypy treats this as semantically equivalent to the previous example
if strict optional checking is disabled, since ``None`` is implicitly
valid for any type, but it's much more
useful for a programmer who is reading the code. This also makes
it easier to migrate to strict ``None`` checking in the future.
.. _noreturn:
......
.. _mypy_daemon:
Mypy daemon (mypy server)
=========================
Instead of running mypy as a command-line tool, you can also run it as
a long-running daemon (server) process and use a command-line client to
send type-checking requests to the server. This way mypy can perform type
checking much faster, since program state cached from previous runs is kept
in memory and doesn't have to be read from the file system on each run.
The server also uses finer-grained dependency tracking to reduce the amount
of work that needs to be done.
If you have a large codebase to check, running mypy using the mypy
daemon can be *10 or more times faster* than the regular command-line
``mypy`` tool, especially if your workflow involves running mypy
repeatedly after small edits -- which is often a good idea, as this way
you'll find errors sooner.
.. note::
The mypy daemon is experimental. In particular, the command-line
interface may change in future mypy releases.
.. note::
The mypy daemon currently supports macOS and Linux only.
.. note::
Each mypy daemon process supports one user and one set of source files,
and it can only process one type checking request at a time. You can
run multiple mypy daemon processes to type check multiple repositories.
Basic usage
***********
The client utility ``dmypy`` is used to control the mypy daemon.
Use ``dmypy start -- <flags>`` to start the daemon. You can use almost
arbitrary mypy flags after ``--``. The daemon will always run on the
current host. Example::
dmypy start -- --follow-imports=skip
.. note::
You'll need to use either the ``--follow-imports=skip`` or the
``--follow-imports=error`` option with dmypy because the current
implementation can't follow imports.
See :ref:`follow-imports` for details on how these work.
You can also define these using a
:ref:`configuration file <config-file>`.
The daemon will not type check anything when it's started.
Use ``dmypy check <files>`` to check some files (or directories)::
dmypy check prog.py pkg1/ pkg2/
You need to provide all files or directories you want to type check
(other than stubs) as arguments. This is a result of the
``--follow-imports`` restriction mentioned above.
The initial run will process all the code and may take a while to
finish, but subsequent runs will be quick, especially if you've only
changed a few files. You can use :ref:`remote caching <remote-cache>`
to speed up the initial run. The speedup can be significant if
you have a large codebase.
Additional features
*******************
You have precise control over the lifetime of the daemon process:
* ``dymypy stop`` stops the daemon.
* ``dmypy restart -- <flags>`` restarts the daemon. The flags are the same
as with ``dmypy start``. This is equivalent to a stop command followed
by a start.
* Use ``dmypy start --timeout SECONDS -- <flags>`` (or
``dmypy restart --timeout SECONDS -- <flags>``) to automatically
shut down the daemon after inactivity. By default, the daemon runs
until it's explicitly stopped.
Use ``dmypy --help`` for help on additional commands and command-line
options not discussed here, and ``dmypy <command> --help`` for help on
command-specific options.
Limitations
***********
* Changes related to protocol classes are not reliably propagated.
* You have to use either the ``--follow-imports=skip`` or
the ``--follow-imports=error`` option because of an implementation
limitation. This can be defined
through the command line or through a
:ref:`configuration file <config-file>`.
* Windows is not supported.
Revision history
================
List of major changes:
List of major changes (the `Mypy Blog <http://mypy-lang.blogspot.com/>`_ contains more
detailed release notes):
- May 2018
* Publish ``mypy`` version 0.600 on PyPI.
* Enable :ref:`strict optional checking <strict_optional>` by default.
* Document :ref:`disabling strict optional checking <no_strict_optional>`.
* Add :ref:`mypy_daemon`.
* Add :ref:`remote-cache`.
* Support user-specific configuration file (:ref:`docs <config-file>`).
* Changes to section pattern semantics in configuration files
(:ref:`docs <config-file>` and :ref:`more docs <per-module-flags>`).
- April 2018
* Publish ``mypy`` version 0.590 on PyPI.
......@@ -225,7 +242,7 @@ List of major changes:
- July 2016
* Publish ``mypy-lang`` version 0.4.3 on PyPI.
* Add :ref:`strict_optional`.
* Add :ref:`strict optional checking <strict_optional>`.
* Add :ref:`multi_line_annotation`.
......@@ -262,7 +279,7 @@ List of major changes:
- Mar 2015
Update documentation to reflect PEP 484:
* Add :ref:`named-tuples` and :ref:`optional`.
* Add :ref:`named-tuples` and :ref:`Optional types <strict_optional>`.
* Do not mention type application syntax (for
example, ``List[int]()``), as it's no longer supported,
......
Metadata-Version: 2.1
Name: mypy
Version: 0.590
Version: 0.600
Summary: Optional static typing for Python
Home-page: http://www.mypy-lang.org/
Author: Jukka Lehtosalo
......
......@@ -30,6 +30,7 @@ docs/source/index.rst
docs/source/installed_packages.rst
docs/source/introduction.rst
docs/source/kinds_of_types.rst
docs/source/mypy_daemon.rst
docs/source/python2.rst
docs/source/python36.rst
docs/source/revision_history.rst
......@@ -76,12 +77,14 @@ mypy/literals.py
mypy/main.py
mypy/maptype.py
mypy/meet.py
mypy/memprofile.py
mypy/messages.py
mypy/moduleinfo.py
mypy/nodes.py
mypy/options.py
mypy/parse.py
mypy/plugin.py
mypy/py.typed
mypy/report.py
mypy/sametypes.py
mypy/scope.py
......@@ -107,6 +110,7 @@ mypy/treetransform.py
mypy/tvar_scope.py
mypy/typeanal.py
mypy/types.py
mypy/typestate.py
mypy/typevars.py
mypy/util.py
mypy/version.py
......@@ -536,6 +540,8 @@ typeshed/stdlib/2/multiprocessing/__init__.pyi
typeshed/stdlib/2/multiprocessing/pool.pyi
typeshed/stdlib/2/multiprocessing/process.pyi
typeshed/stdlib/2/multiprocessing/util.pyi
typeshed/stdlib/2/multiprocessing/dummy/__init__.pyi
typeshed/stdlib/2/multiprocessing/dummy/connection.pyi
typeshed/stdlib/2/os/__init__.pyi
typeshed/stdlib/2/os/path.pyi
typeshed/stdlib/2/wsgiref/__init__.pyi
......@@ -876,6 +882,8 @@ typeshed/stdlib/3/multiprocessing/pool.pyi
typeshed/stdlib/3/multiprocessing/process.pyi
typeshed/stdlib/3/multiprocessing/queues.pyi
typeshed/stdlib/3/multiprocessing/synchronize.pyi
typeshed/stdlib/3/multiprocessing/dummy/__init__.pyi
typeshed/stdlib/3/multiprocessing/dummy/connection.pyi
typeshed/stdlib/3/os/__init__.pyi
typeshed/stdlib/3/os/path.pyi
typeshed/stdlib/3/tkinter/__init__.pyi
......
This diff is collapsed.
......@@ -44,7 +44,7 @@ from mypy.subtypes import (
)
from mypy.maptype import map_instance_to_supertype
from mypy.typevars import fill_typevars, has_no_typevars
from mypy.semanal import set_callable_name, refers_to_fullname
from mypy.semanal import set_callable_name, refers_to_fullname, calculate_mro
from mypy.erasetype import erase_typevars
from mypy.expandtype import expand_type, expand_type_by_instance
from mypy.visitor import NodeVisitor
......@@ -1390,6 +1390,9 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
if second_type is None:
self.msg.cannot_determine_type_in_base(name, base2.name(), ctx)
ok = True
# __slots__ is special and the type can vary across class hierarchy.
if name == '__slots__':
ok = True
if not ok:
self.msg.base_class_definitions_incompatible(name, base1, base2,
ctx)
......@@ -1466,7 +1469,7 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
else:
lvalue_type, index_lvalue, inferred = self.check_lvalue(lvalue)
if isinstance(lvalue, NameExpr):
if isinstance(lvalue, RefExpr):
if self.check_compatibility_all_supers(lvalue, lvalue_type, rvalue):
# We hit an error on this line; don't check for any others
return
......@@ -1531,13 +1534,12 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
self.infer_variable_type(inferred, lvalue, self.expr_checker.accept(rvalue),
rvalue)
def check_compatibility_all_supers(self, lvalue: NameExpr, lvalue_type: Optional[Type],
def check_compatibility_all_supers(self, lvalue: RefExpr, lvalue_type: Optional[Type],
rvalue: Expression) -> bool:
lvalue_node = lvalue.node
# Check if we are a class variable with at least one base class
if (isinstance(lvalue_node, Var) and
lvalue.kind == MDEF and
lvalue.kind in (MDEF, None) and # None for Vars defined via self
len(lvalue_node.info.bases) > 0):
for base in lvalue_node.info.mro[1:]:
......@@ -1575,7 +1577,7 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
break
return False
def check_compatibility_super(self, lvalue: NameExpr, lvalue_type: Optional[Type],
def check_compatibility_super(self, lvalue: RefExpr, lvalue_type: Optional[Type],
rvalue: Expression, base: TypeInfo, base_type: Type,
base_node: Node) -> bool:
lvalue_node = lvalue.node
......@@ -2515,57 +2517,47 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
def visit_for_stmt(self, s: ForStmt) -> None:
"""Type check a for statement."""
if s.is_async:
item_type = self.analyze_async_iterable_item_type(s.expr)
iterator_type, item_type = self.analyze_async_iterable_item_type(s.expr)
else:
item_type = self.analyze_iterable_item_type(s.expr)
iterator_type, item_type = self.analyze_iterable_item_type(s.expr)
s.inferred_item_type = item_type
s.inferred_iterator_type = iterator_type
self.analyze_index_variables(s.index, item_type, s.index_type is None, s)
self.accept_loop(s.body, s.else_body)
def analyze_async_iterable_item_type(self, expr: Expression) -> Type:
"""Analyse async iterable expression and return iterator item type."""
def analyze_async_iterable_item_type(self, expr: Expression) -> Tuple[Type, Type]:
"""Analyse async iterable expression and return iterator and iterator item types."""
echk = self.expr_checker
iterable = echk.accept(expr)
self.check_subtype(iterable,
self.named_generic_type('typing.AsyncIterable',
[AnyType(TypeOfAny.special_form)]),
expr, messages.ASYNC_ITERABLE_EXPECTED)
method = echk.analyze_external_member_access('__aiter__', iterable, expr)
iterator = echk.check_call(method, [], [], expr)[0]
method = echk.analyze_external_member_access('__anext__', iterator, expr)
awaitable = echk.check_call(method, [], [], expr)[0]
return echk.check_awaitable_expr(awaitable, expr,
item_type = echk.check_awaitable_expr(awaitable, expr,
messages.INCOMPATIBLE_TYPES_IN_ASYNC_FOR)
return iterator, item_type
def analyze_iterable_item_type(self, expr: Expression) -> Type:
"""Analyse iterable expression and return iterator item type."""
def analyze_iterable_item_type(self, expr: Expression) -> Tuple[Type, Type]:
"""Analyse iterable expression and return iterator and iterator item types."""
echk = self.expr_checker
iterable = echk.accept(expr)
method = echk.analyze_external_member_access('__iter__', iterable, expr)
iterator = echk.check_call(method, [], [], expr)[0]
if isinstance(iterable, TupleType):
joined = UninhabitedType() # type: Type
for item in iterable.items:
joined = join_types(joined, item)
return joined
return iterator, joined
else:
# Non-tuple iterable.
self.check_subtype(iterable,
self.named_generic_type('typing.Iterable',
[AnyType(TypeOfAny.special_form)]),
expr, messages.ITERABLE_EXPECTED)
method = echk.analyze_external_member_access('__iter__', iterable,
expr)
iterator = echk.check_call(method, [], [], expr)[0]
if self.options.python_version[0] >= 3:
nextmethod = '__next__'
else:
nextmethod = 'next'
method = echk.analyze_external_member_access(nextmethod, iterator,
expr)
return echk.check_call(method, [], [], expr)[0]
return iterator, echk.check_call(method, [], [], expr)[0]
def analyze_index_variables(self, index: Expression, item_type: Type,
infer_lvalue_type: bool, context: Context) -> None:
......@@ -2733,7 +2725,7 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
info = TypeInfo(SymbolTable(), cdef, cur_module.fullname())
cdef.info = info
info.bases = [typ]
info.calculate_mro()
calculate_mro(info)
info.calculate_metaclass_type()
# Build up a fake FuncDef so we can populate the symbol table.
......
......@@ -1422,7 +1422,7 @@ class ExpressionChecker(ExpressionVisitor[Type]):
elif (local_errors.is_errors() and
# is_valid_var_arg is True for any Iterable
self.is_valid_var_arg(right_type)):
itertype = self.chk.analyze_iterable_item_type(right)
_, itertype = self.chk.analyze_iterable_item_type(right)
method_type = CallableType(
[left_type],
[nodes.ARG_POS],
......@@ -2290,9 +2290,9 @@ class ExpressionChecker(ExpressionVisitor[Type]):
for index, sequence, conditions, is_async in zip(e.indices, e.sequences,
e.condlists, e.is_async):
if is_async:
sequence_type = self.chk.analyze_async_iterable_item_type(sequence)
_, sequence_type = self.chk.analyze_async_iterable_item_type(sequence)
else:
sequence_type = self.chk.analyze_iterable_item_type(sequence)
_, sequence_type = self.chk.analyze_iterable_item_type(sequence)
self.chk.analyze_index_variables(index, sequence_type, True, e)
for condition in conditions:
self.accept(condition)
......
......@@ -3,3 +3,6 @@ PYTHON3_VERSION = (3, 6)
PYTHON3_VERSION_MIN = (3, 3)
CACHE_DIR = '.mypy_cache'
CONFIG_FILE = 'mypy.ini'
SHARED_CONFIG_FILES = ('setup.cfg',)
USER_CONFIG_FILES = ('~/.mypy.ini',)
CONFIG_FILES = (CONFIG_FILE,) + SHARED_CONFIG_FILES + USER_CONFIG_FILES
......@@ -26,6 +26,10 @@ from mypy.gclogger import GcLogger
from mypy.fscache import FileSystemCache
from mypy.fswatcher import FileSystemWatcher, FileData
from mypy.options import Options
from mypy.typestate import TypeState
MEM_PROFILE = False # If True, dump memory profile after initialization
def daemonize(func: Callable[[], None], log_file: Optional[str] = None) -> int:
......@@ -119,13 +123,15 @@ class Server:
if os.path.isfile(STATUS_FILE):
os.unlink(STATUS_FILE)
self.fscache = FileSystemCache(self.options.python_version)
self.fscache = FileSystemCache()
options.incremental = True
options.fine_grained_incremental = True
options.show_traceback = True
if options.use_fine_grained_cache:
options.cache_fine_grained = True # set this so that cache options match
# Using fine_grained_cache implies generating and caring
# about the fine grained cache
options.cache_fine_grained = True
else:
options.cache_dir = os.devnull
# Fine-grained incremental doesn't support general partial types
......@@ -147,6 +153,7 @@ class Server:
conn, addr = sock.accept()
except socket.timeout:
print("Exiting due to inactivity.")
self.free_global_state()
sys.exit(0)
try:
data = receive(conn)
......@@ -177,6 +184,7 @@ class Server:
conn.close()
if command == 'stop':
sock.close()
self.free_global_state()
sys.exit(0)
finally:
os.unlink(STATUS_FILE)
......@@ -186,6 +194,9 @@ class Server:
if exc_info[0] and exc_info[0] is not SystemExit:
traceback.print_exception(*exc_info) # type: ignore
def free_global_state(self) -> None:
TypeState.reset_all_subtype_caches()
def create_listening_socket(self) -> socket.socket:
"""Create the socket and set it up for listening."""
self.sockname = os.path.abspath(SOCKET_NAME)
......@@ -289,19 +300,28 @@ class Server:
# Stores the initial state of sources as a side effect.
self.fswatcher.find_changed()
if MEM_PROFILE:
from mypy.memprofile import print_memory_profile
print_memory_profile(run_gc=False)
status = 1 if messages else 0
return {'out': ''.join(s + '\n' for s in messages), 'err': '', 'status': status}
def fine_grained_increment(self, sources: List[mypy.build.BuildSource]) -> Dict[str, Any]:
assert self.fine_grained_manager is not None
manager = self.fine_grained_manager.manager
t0 = time.time()
self.update_sources(sources)
changed, removed = self.find_changed(sources)
# Update the lib_path, which can change when sources do.
# TODO: This is slow.
manager.lib_path = tuple(mypy.build.compute_lib_path(
sources, manager.options, self.alt_lib_path, manager.data_dir, self.fscache))
t1 = time.time()
messages = self.fine_grained_manager.update(changed, removed)
t2 = time.time()
self.fine_grained_manager.manager.log(
manager.log(
"fine-grained increment: find_changed: {:.3f}s, update: {:.3f}s".format(
t1 - t0, t2 - t1))
status = 1 if messages else 0
......
......@@ -5,7 +5,7 @@ import os.path
from typing import List, Sequence, Set, Tuple, Optional, Dict
from mypy.build import BuildSource, PYTHON_EXTENSIONS
from mypy.fscache import FileSystemMetaCache
from mypy.fscache import FileSystemCache
from mypy.options import Options
......@@ -17,13 +17,13 @@ class InvalidSourceList(Exception):
def create_source_list(files: Sequence[str], options: Options,
fscache: Optional[FileSystemMetaCache] = None,
fscache: Optional[FileSystemCache] = None,
allow_empty_dir: bool = False) -> List[BuildSource]:
"""From a list of source files/directories, makes a list of BuildSources.
Raises InvalidSourceList on errors.
"""
fscache = fscache or FileSystemMetaCache()
fscache = fscache or FileSystemCache()
finder = SourceFinder(fscache)
targets = []
......@@ -56,7 +56,7 @@ def keyfunc(name: str) -> Tuple[int, str]:
class SourceFinder:
def __init__(self, fscache: FileSystemMetaCache) -> None:
def __init__(self, fscache: FileSystemCache) -> None:
self.fscache = fscache
# A cache for package names, mapping from module id to directory path
self.package_cache = {} # type: Dict[str, str]
......