Commit 164575da authored by Guillaume Binet's avatar Guillaume Binet

Python2 removal.

A lot of crap all around: try on imports / if PY2/Py3, tests on unicode,
src conversion, travis config, docs ...
parent a2882148
__pycache__
.venv
*.pyc
/.idea
/atlassian-ide-plugin.xml
/.settings
......
language: python
python:
- "2.7"
- "3.3"
- "3.4"
- "3.5"
......@@ -11,8 +10,7 @@ before_script:
install:
- pip install --upgrade pip
- pip install -e .
- pip install slackclient # Optional dependency but Slack tests are skipped without it
- pip install -e .[slack]
- pip freeze
script: ./run_tests.py
......
......@@ -106,8 +106,8 @@ Installation
Prerequisites
~~~~~~~~~~~~~
Errbot runs under Python 3.3+ and Python 2.7 on Linux, Windows and Mac. For some chatting systems you'll need a key or a login for your bot to access it.
Note: Python 2 support is going away ! Please check out the change log below for details.
Errbot runs under Python 3.3+ on Linux, Windows and Mac. For some chatting systems you'll need a key or a login for your bot to access it.
Note: Python 2 support is still supported in `errbot-4.2.x` but is it going away.
Quickstart
~~~~~~~~~~
......
......@@ -140,7 +140,7 @@ Lets go ahead and create ours. Place the following in a file called
file structure.
Lets look at what this does. We see three sections, `[Core]` ,
`[Python]` and `[Documentation]`. The `[Core]` section is what tells
and `[Documentation]`. The `[Core]` section is what tells
Errbot where it can actually find the code for this plugin.
The key `Module` should point to a module that Python can find and
......@@ -156,17 +156,6 @@ names can differ, doing so is not recommended.
the same as the class name anyway, this has to do with technical
limitations that we won't go into here.
While the items from the `[Core]` section tell Errbot where to find the
code, the `[Python]` section tells Errbot which versions of Python your
plugin is compatible with. As you are probably aware, Errbot runs on
both 2.7 and 3.\ *x* versions of Python, but maintaining compatibilty
to both can take some time and effort however. Effort you might not
want to put into your own plugins.
The `Version` key allows you to specify which versions of Python your
plugin works on. Supported values are `2` for Python 2, `3` for
Python 3 and `2+` for both Python 2 and 3.
The `[Documentation]` section will be explained in more detail
further on in this guide, but you should make sure to at least have
the `Description` item here with a short description of your plugin.
......
Plugin compatibility settings
=============================
Python compatibility
--------------------
Your plugin might not be compatible with Python 2 or Python 3.
In order to prevent users to install them under an incompatible environment,
you can add a **Python** section to your plug file:
.. code-block:: ini
[Core]
Name = MyPlugin
Module = myplugin
[Documentation]
Description = my plugin
[Python]
Version=2+
The possible choices for **Version** are:
- 2 : only compatible with Python 2
- 3 : only compatible with Python 3
- 2+: compatible with both
Of there is no Python section, your plugin will be restricted to Python 2 for backward compatibility.
Errbot compatibility
--------------------
......
......@@ -4,8 +4,7 @@ Setup
Prerequisites
-------------
Errbot runs under Python 3.2+ on Linux, Windows and Mac. You can still use it under Python 2.7 if you really
must but it is not recommended and the support for Python 2 will be removed soon.
Errbot runs under Python 3.3+ on Linux, Windows and Mac.
Installation
------------
......@@ -53,7 +52,6 @@ Errbot may be installed directly from PyPi using `pip`_ by issuing::
which means you need to have development headers for some libraries installed.
On Debian/Ubuntu these may be installed with
`apt-get install python3-dev libssl-dev libffi-dev`
(use `python-dev` instead of `python3-dev` if you're still stuck on Python 2).
Package names may differ on other OS's.
.. _configuration:
......
......@@ -11,8 +11,6 @@ from typing import Callable, Any, Tuple
from .core_plugins.wsview import bottle_app, WebView
from .backends.base import Message, ONLINE, OFFLINE, AWAY, DND # noqa
from .utils import compat_str
from .utils import PY2, PY3 # noqa gbin: this is now used by plugins
from .botplugin import BotPlugin, SeparatorArgParser, ShlexArgParser, CommandError, Command # noqa
from .flow import FlowRoot, BotFlow, Flow, FLOW_END
from .core_plugins.wsview import route, view # noqa
......@@ -438,7 +436,7 @@ def webhook(*args,
if isinstance(args[0], (str, bytes)): # first param is uri_rule.
return lambda func: _tag_webhook(func,
compat_str(args[0]).rstrip('/'), # trailing / is also be stripped on incoming.
str(args[0]).rstrip('/'), # trailing / is also be stripped on incoming.
methods=methods,
form_param=form_param,
raw=raw)
......
......@@ -2,7 +2,7 @@ import io
import logging
import random
import time
from typing import Any, Mapping, BinaryIO, List, Union, Sequence, Tuple
from typing import Any, Mapping, BinaryIO, List, Sequence, Tuple
from abc import abstractproperty, abstractmethod
from collections import deque, defaultdict
......@@ -11,18 +11,13 @@ import inspect
try:
from abc import ABC
except ImportError:
# 3.3 compatibility
# 3.3 backward compatibility
from abc import ABCMeta
class ABC(metaclass=ABCMeta):
"""Helper class that provides a standard way to create an ABC using
inheritance.
"""
pass
from errbot.utils import compat_str, deprecated
# Can't use __name__ because of Yapsy
log = logging.getLogger('errbot.backends.base')
......@@ -247,7 +242,7 @@ class Message(object):
:param flow:
The flow in which this message has been triggered.
"""
self._body = compat_str(body)
self._body = body
self._from = frm
self._to = to
self._delayed = delayed
......
......@@ -20,8 +20,8 @@ try:
from PySide.QtCore import Qt
except ImportError:
log.exception("Could not start the graphical backend")
log.fatal(""" To install PySide use:
pip install PySide
log.fatal(""" To install graphic support use:
pip install errbot[graphic]
""")
sys.exit(-1)
......
......@@ -3,23 +3,16 @@
import logging
import re
import sys
try:
from functools import lru_cache
except ImportError:
from backports.functools_lru_cache import lru_cache
from functools import lru_cache
from errbot.backends.base import Room, RoomDoesNotExistError, RoomOccupant
from errbot.backends.xmpp import XMPPRoomOccupant, XMPPBackend, XMPPConnection
from markdown import Markdown
from markdown.extensions.extra import ExtraExtension
from markdown.extensions import Extension
from markdown.treeprocessors import Treeprocessor
from errbot.backends.base import Room, RoomDoesNotExistError, Message, RoomOccupant
from errbot.backends.xmpp import (
XMPPPerson, XMPPRoomOccupant,
XMPPBackend, XMPPConnection,
split_identifier
)
# Can't use __name__ because of Yapsy
log = logging.getLogger('errbot.backends.hipchat')
......@@ -29,9 +22,9 @@ try:
except ImportError:
log.exception("Could not start the HipChat backend")
log.fatal(
"You need to install the hypchat package in order to use the HipChat "
"back-end. You should be able to install this package using: "
"pip install hypchat"
"You need to install the hipchat support in order to use the HipChat.\n "
"You should be able to install this package using:\n"
"pip install errbot[hipchat]"
)
sys.exit(1)
......
......@@ -57,18 +57,9 @@ try:
import irc.connection
from irc.client import ServerNotConnectedError, NickMask
from irc.bot import SingleServerIRCBot
except ImportError as _:
log.exception("Could not start the IRC backend")
log.fatal("""
If you intend to use the IRC backend please install the python irc package:
-> On debian-like systems
sudo apt-get install python-software-properties
sudo apt-get update
sudo apt-get install python-irc
-> On Gentoo
sudo emerge -av dev-python/irc
-> Generic
pip install irc
except ImportError:
log.fatal("""You need the IRC support to use IRC, you can install it with:
pip install errbot[IRC]
""")
sys.exit(-1)
......
......@@ -6,46 +6,29 @@ import re
import time
import sys
import pprint
from functools import lru_cache
from errbot.backends.base import Message, Presence, ONLINE, AWAY, Room, RoomError, RoomDoesNotExistError, \
UserDoesNotExistError, RoomOccupant, Person, Card
from errbot.core import ErrBot
from errbot.utils import PY3, split_string_after
from errbot.utils import split_string_after
from errbot.rendering.slack import slack_markdown_converter
# Can't use __name__ because of Yapsy
log = logging.getLogger('errbot.backends.slack')
try:
from functools import lru_cache
except ImportError:
from backports.functools_lru_cache import lru_cache
try:
from slackclient import SlackClient
except ImportError:
log.exception("Could not start the Slack back-end")
log.fatal(
"You need to install the slackclient package in order to use the Slack "
"back-end. You should be able to install this package using: "
"pip install slackclient"
)
sys.exit(1)
except SyntaxError:
if not PY3:
raise
log.exception("Could not start the Slack back-end")
log.fatal(
"I cannot start the Slack back-end because I cannot import the SlackClient. "
"Python 3 compatibility on SlackClient is still quite young, you may be "
"running an old version or perhaps they released a version with a Python "
"3 regression. As a last resort to fix this, you could try installing the "
"latest master version from them using: "
"pip install --upgrade https://github.com/slackhq/python-slackclient/archive/master.zip"
"You need to install the slackclient support in order to use the Slack.\n"
"You can do `pip install errbot[slack]` to install it"
)
sys.exit(1)
# The Slack client automatically turns a channel name into a clickable
# link if you prefix it with a #. Other clients receive this link as a
# token matching this regex.
......
import logging
import sys
from errbot import PY2
from errbot.backends.base import RoomError, Identifier, Person, RoomOccupant, ONLINE, Room
from errbot.core import ErrBot
from errbot.rendering import text
......@@ -12,17 +11,17 @@ from errbot.rendering.ansiext import enable_format, TEXT_CHRS
log = logging.getLogger('errbot.backends.telegram')
TELEGRAM_MESSAGE_SIZE_LIMIT = 1024
UPDATES_OFFSET_KEY = b'_telegram_updates_offset' if PY2 else '_telegram_updates_offset'
UPDATES_OFFSET_KEY = '_telegram_updates_offset'
try:
import telegram
except ImportError:
log.exception("Could not start the Telegram back-end")
log.fatal(
"You need to install the python-telegram-bot package in order "
"to use the Telegram back-end. "
"You should be able to install this package using: "
"pip install python-telegram-bot"
"You need to install the telegram support in order "
"to use the Telegram backend.\n"
"You should be able to install this package using:\n"
"pip install errbot[telegram]"
)
sys.exit(1)
......
import logging
import sys
from functools import lru_cache
from sleekxmpp.exceptions import IqError
try:
from functools import lru_cache
except ImportError:
from backports.functools_lru_cache import lru_cache
from threading import Thread
from time import sleep
......@@ -23,19 +18,11 @@ log = logging.getLogger('errbot.backends.xmpp')
try:
from sleekxmpp import ClientXMPP
from sleekxmpp.xmlstream import resolver, cert
except ImportError as _:
except ImportError:
log.exception("Could not start the XMPP backend")
log.fatal("""
If you intend to use the XMPP backend please install the python sleekxmpp package:
-> On debian-like systems
sudo apt-get install python-software-properties
sudo apt-get update
sudo apt-get install python-sleekxmpp
-> On Gentoo
sudo layman -a laurentb
sudo emerge -av dev-python/sleekxmpp
-> Generic
pip install sleekxmpp
If you intend to use the XMPP backend pleas install the support for XMPP with:
pip install errbot[XMPP]
""")
sys.exit(-1)
......
from os import path, makedirs
import logging
import sys
from errbot.core import ErrBot
from errbot.plugin_manager import BotPluginManager
from errbot.repo_manager import BotRepoManager
from errbot.specific_plugin_manager import SpecificPluginManager
import sys
from errbot.storage.base import StoragePluginBase
from errbot.utils import PLUGINS_SUBDIR, is_str
......
......@@ -5,17 +5,13 @@ from types import ModuleType
from typing import Tuple, Callable, Mapping, Sequence
from io import IOBase
from .utils import recurse_check_structure, PY2
from .utils import recurse_check_structure
from .storage import StoreMixin, StoreNotOpenError
from errbot.backends.base import Message, Presence, Stream, Room, Identifier, ONLINE, Card
log = logging.getLogger(__name__)
def compat_ascii(s):
return s.encode('ascii') if PY2 and isinstance(s, unicode) else s
class CommandError(Exception):
"""
Use this class to report an error condition from your commands, the command
......@@ -64,7 +60,7 @@ class Command(object):
cmd_kwargs = {}
if cmd_args is None:
cmd_args = ()
function.__name__ = compat_ascii(name)
function.__name__ = name
if doc:
function.__doc__ = doc
self.definition = cmd_type(*((function,) + cmd_args), **cmd_kwargs)
......@@ -239,9 +235,7 @@ class BotPluginBase(StoreMixin):
"""
if name in self._dynamic_plugins:
raise ValueError('Dynamic plugin %s already created.')
plugin_class = type(compat_ascii(name),
(BotPlugin,),
{command.name: command.definition for command in commands})
plugin_class = type(name, (BotPlugin,), {command.name: command.definition for command in commands})
plugin_class.__errdoc__ = doc
plugin = plugin_class(self._bot)
self._dynamic_plugins[name] = plugin
......
......@@ -15,7 +15,6 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
import argparse
import inspect
import locale
import logging
import os
......@@ -27,27 +26,6 @@ from errbot.logs import root_logger
from errbot.plugin_wizard import new_plugin_wizard
from errbot.version import VERSION
PY3 = sys.version_info[0] == 3
PY2 = not PY3
# Fail early if the user tries to run err under the incorrect interpreter
def foo(param='canary'):
pass
foo_src = inspect.getsourcelines(foo)[0][0]
if PY3 and "param=u'canary'" in foo_src:
print('Err has been converted to Python2 but you try to run it under Python3')
sys.exit(-1)
if PY2 and "param='canary'" in foo_src:
print('You are trying to run err under python2 without converting the source code to py2 first.')
print('Either use python3 or install err using ./setup.py develop.')
sys.exit(-1)
log = logging.getLogger(__name__)
if locale.getpreferredencoding().lower() != 'utf-8':
......
import fnmatch
from errbot import BotPlugin, cmdfilter
from errbot.backends.base import RoomOccupant
from errbot.utils import compat_str, is_str
from errbot.utils import is_str
BLOCK_COMMAND = (None, None, None)
......@@ -21,7 +21,7 @@ def glob(text, patterns):
"""
if is_str(patterns):
patterns = (patterns,)
return any(fnmatch.fnmatchcase(compat_str(text), compat_str(pattern)) for pattern in patterns)
return any(fnmatch.fnmatchcase(text, pattern) for pattern in patterns)
def ciglob(text, patterns):
......
......@@ -2,16 +2,8 @@ import logging
from errbot import BotPlugin, botcmd, SeparatorArgParser, ShlexArgParser
from errbot.backends.base import RoomNotJoinedError
from errbot.utils import compat_str, PY3
# 2to3 hack
# thanks to https://github.com/oxplot/fysom/issues/1
# which in turn references http://www.rfk.id.au/blog/entry/preparing-pyenchant-for-python-3/
if PY3:
basestring = (str, bytes)
log = logging.getLogger(__name__)
log.debug("LOADING CHATROOM")
class ChatRoom(BotPlugin):
......@@ -32,16 +24,14 @@ class ChatRoom(BotPlugin):
self.log.exception("Joining room %s failed", repr(room))
def _join_room(self, room):
username = self.bot_config.CHATROOM_FN
password = None
if isinstance(room, (tuple, list)):
room_name = compat_str(room[0])
room_password = compat_str(room[1])
room, username, password = (room_name, self.bot_config.CHATROOM_FN, room_password)
room, password = room # unpack
self.log.info("Joining room {} with username {} and password".format(room, username))
else:
room_name = compat_str(room)
room, username, password = (room_name, self.bot_config.CHATROOM_FN, None)
self.log.info("Joining room {} with username {}".format(room, username))
self.query_room(room).join(username=self.bot_config.CHATROOM_FN, password=password)
self.query_room(room).join(username=self.bot_config.CHATROOM_FN, password=password)
def deactivate(self):
self.connected = False
......
......@@ -6,18 +6,13 @@ from random import random
from webtest import TestApp
from errbot import botcmd, BotPlugin, webhook
from errbot.utils import PY3
from errbot.core_plugins.wsview import bottle_app
from rocket import Rocket
if PY3:
from urllib.request import unquote
else:
from urllib2 import unquote
from urllib.request import unquote
try:
from OpenSSL import crypto
has_crypto = True
except ImportError:
has_crypto = False
......@@ -54,7 +49,7 @@ def make_ssl_certificate(key_path, cert_path):
pkey = crypto.PKey()
pkey.generate_key(crypto.TYPE_RSA, 4096)
cert.set_pubkey(pkey)
cert.sign(pkey, 'sha256' if PY3 else b'sha256')
cert.sign(pkey, 'sha256')
f = open(cert_path, 'w')
f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert).decode('utf-8'))
......@@ -141,7 +136,7 @@ class Webserver(BotPlugin):
It triggers the notification and generate also a little test report.
"""
url = args[0] if PY3 else args[0].encode() # PY2 needs a str not unicode
url = args[0]
content = ' '.join(args[1:])
# try to guess the content-type of what has been passed
......
import inspect
import logging
import sys
......
......@@ -10,7 +10,7 @@ import pip
from errbot.flow import BotFlow
from .botplugin import BotPlugin
from .utils import (version2array, PY3, PY2, collect_roots, ensure_sys_path_contains)
from .utils import version2array, collect_roots, ensure_sys_path_contains
from .templating import remove_plugin_templates_path, add_plugin_templates_path
from .version import VERSION
from yapsy.PluginManager import PluginManager
......@@ -135,18 +135,12 @@ def check_python_plug_section(name: str, config: ConfigParser) -> bool:
'The Version can only be 2, 2+ and 3', name)
return False
if python_version == '2' and PY3:
if python_version == '2':
log.error(
'\nPlugin %s is made for python 2 only and you are running '
'err under python 3.\n\n'
'If the plugin can be run on python 2 and 3 please add this '
'section to its .plug descriptor :\n[Python]\nVersion=2+\n\n'
'Or if the plugin is Python 3 only:\n[Python]\nVersion=3\n\n', name)
'\nPlugin %s is made for python 2 only and Errbot is not compatible with Python 2 anymore.'
'Please contact the plugin developer or try to contribute to port the plugin.')
return False
if python_version == '3' and PY2:
log.error('\nPlugin %s is made for python 3 and you are running err under python 2.', name)
return False
return True
......@@ -216,8 +210,8 @@ class BotPluginManager(PluginManager, StoreMixin):
"""Customized yapsy PluginManager for ErrBot."""
# Storage names
CONFIGS = b'configs' if PY2 else 'configs'
BL_PLUGINS = b'bl_plugins' if PY2 else 'bl_plugins'
CONFIGS = 'configs'
BL_PLUGINS = 'bl_plugins'
def __init__(self, storage_plugin, repo_manager, extra, autoinstall_deps, core_plugins):
self.bot = None
......@@ -228,7 +222,7 @@ class BotPluginManager(PluginManager, StoreMixin):
self.repo_manager = repo_manager
# if this is the old format migrate the entries in repo_manager
ex_entry = b'repos' if PY2 else 'repos'
ex_entry = 'repos'
if ex_entry in self:
log.info('You are migrating from v3 to v4, porting your repo info...')
for name, url in self[ex_entry].items():
......@@ -329,8 +323,6 @@ class BotPluginManager(PluginManager, StoreMixin):
module_alias = plugin.plugin_object.__module__
module_old = __import__(module_alias)
f = module_old.__file__
if f.endswith('.pyc'):
f = f[:-1] # py2 compat : load the .py
module_new = imp.load_source(module_alias, f)
class_name = type(plugin.plugin_object).__name__
new_class = getattr(module_new, class_name)
......
......@@ -5,16 +5,10 @@ import jinja2
import os
import re
import sys
from configparser import ConfigParser
from errbot import PY2, PY3
from errbot.version import VERSION
if PY2:
from backports.configparser import ConfigParser
input = raw_input
else:
from configparser import ConfigParser
def new_plugin_wizard(directory=None):
"""
......@@ -41,17 +35,7 @@ def new_plugin_wizard(directory=None):
description = ask(
"What may I use as a short (one-line) description of your plugin?"
)
if PY2:
default_python_version = "2+"
else:
default_python_version = "3"
python_version = ask(
"Which python version will your plugin work with? 2, 2+ or 3? I will default to "
"{version} if you leave this blank.".format(version=default_python_version),
valid_responses=['2', '2+', '3', '']
)
if python_version.strip() == "":
python_version = default_python_version
python_version = "3"
errbot_min_version = ask(
"Which minimum version of errbot will your plugin work with? "
"Leave blank to support any version or input CURRENT to select the "
......@@ -89,8 +73,7 @@ def new_plugin_wizard(directory=None):
pyfile_path = os.path.join(plugin_path, module_name+".py")
try:
if PY3 or (PY2 and not os.path.isdir(plugin_path)):
os.makedirs(plugin_path, mode=0o700)
os.makedirs(plugin_path, mode=0o700)
except IOError as e:
if e.errno != errno.EEXIST:
raise
......
......@@ -20,13 +20,9 @@ log = logging.getLogger(__name__)
try:
from html import unescape # py3.5
except:
try:
from html.parser import HTMLParser # py3.4
except ImportError:
from HTMLParser import HTMLParser # py2
finally:
unescape = HTMLParser().unescape
except ImportError:
from html.parser import HTMLParser # py3.4
unescape = HTMLParser().unescape
# chr that should not count as a space
......
......@@ -15,15 +15,11 @@ import re
from errbot.plugin_manager import check_dependencies
from errbot.storage import StoreMixin
from .utils import PY2, which, compat_str
from .utils import which
log = logging.getLogger(__name__)
def timestamp(dt):
return (dt - datetime(1970, 1, 1)).total_seconds() if PY2 else dt.timestamp()
def human_name_for_git_url(url):
# try to humanize the last part of the git url as much as we can
s = url.split(':')[-1].split('/')[-2:]
......@@ -32,11 +28,11 @@ def human_name_for_git_url(url):
return str('/'.join(s))
INSTALLED_REPOS = b'installed_repos' if PY2 else 'installed_repos'
INSTALLED_REPOS = 'installed_repos'
REPO_INDEXES_CHECK_INTERVAL = timedelta(hours=1)
REPO_INDEX = b'repo_index' if PY2 else 'repo_index'
REPO_INDEX = 'repo_index'
LAST_UPDATE = 'last_update'
RepoEntry = namedtuple('RepoEntry', 'entry_name, name, python, repo, path, avatar_url, documentation')
......@@ -94,23 +90,22 @@ class BotRepoManager(StoreMixin):
self.index_update()
def index_update(self):
index = {LAST_UPDATE: timestamp(datetime.now())}
index = {LAST_UPDATE: datetime.now().timestamp()}
for source in reversed(self.plugin_indexes):
src_file = None
try:
if source.startswith('http'):
log.debug('Update from remote source %s...', source)
src_file = urlopen(url=source, timeout=10)
with urlopen(url=source, timeout=10) as request:
log.debug('Update from remote source %s...', source)
encoding = request.headers.get_content_charset()
content = request.read().decode(encoding if encoding else 'utf-8')
else:
log.debug('Update from local source %s...', source)
src_file = open(source, 'r')
index.update(json.loads(compat_str(src_file.read())))
with open(source, encoding='utf-8', mode='r') as src_file: