...
 
Commits (3)
- id: codespell
name: codespell
description: Checks for common misspellings in text files.
entry: codespell
language: python
types: [text]
args: []
require_serial: false
additional_dependencies: []
......@@ -6,7 +6,7 @@
language: python
sudo: false
dist: trusty
dist: xenial
cache:
directories:
......@@ -16,12 +16,12 @@ matrix:
include:
- os: linux
python: 2.7
- os: linux
python: 3.4
- os: linux
python: 3.5
- os: linux
python: 3.6
- os: linux
python: 3.7
before_install:
- source tools/travis_tools.sh
......@@ -31,7 +31,7 @@ before_install:
- source venv/bin/activate
- python --version # just to check
- pip install -U pip wheel # upgrade to latest pip find 3.5 wheels; wheel to avoid errors
- retry pip install pytest pytest-cov pytest-sugar flake8 coverage codecov chardet setuptools
- retry pip install pytest pytest-cov flake8 coverage codecov chardet setuptools
- cd $SRC_DIR
install:
......@@ -40,6 +40,7 @@ install:
script:
- codespell --help
- make check-dictionary
- codespell --skip="codespell_lib/tests/test_basic.py,codespell_lib/data/*" codespell_lib/
# this file has an error
- "! codespell codespell_lib/tests/test_basic.py"
- flake8
......
SORT_ARGS := -f
SORT_ARGS := -f -b
PHONY := all check check-dictionary sort-dictionary clean
DICTIONARY := codespell_lib/data/dictionary.txt
PHONY := all check check-dictionary sort-dictionary trim-dictionary clean
all: check-dictionary codespell.1
......@@ -9,13 +11,20 @@ codespell.1: codespell.1.include bin/codespell
sed -i '/\.SS \"Usage/,+2d' codespell.1
check-dictionary:
@if ! LC_ALL=C sort ${SORT_ARGS} -c codespell_lib/data/dictionary.txt; then \
@if ! LC_ALL=C sort ${SORT_ARGS} -c ${DICTIONARY}; then \
echo "Dictionary not sorted. Sort with 'make sort-dictionary'"; \
exit 1; \
fi
@if egrep -n "^\s*$$|\s$$|^\s" ${DICTIONARY}; then \
echo "Dictionary contains leading/trailing whitespace and/or blank lines. Trim with 'make trim-dictionary'"; \
exit 1; \
fi
sort-dictionary:
LC_ALL=C sort ${SORT_ARGS} -u -o codespell_lib/data/dictionary.txt codespell_lib/data/dictionary.txt
LC_ALL=C sort ${SORT_ARGS} -u -o ${DICTIONARY} ${DICTIONARY}
trim-dictionary:
sed -E -i.bak -e 's/^[[:space:]]+//; s/[[:space:]]+$$//; /^$$/d' ${DICTIONARY} && rm ${DICTIONARY}.bak
pypi:
python setup.py sdist register upload
......
......@@ -39,6 +39,14 @@ Some noteworthy flags::
The -w flag will actually implement the changes recommended by codespell. Not running with ``-w`` flag is the same as with doing a dry run. It is recommended to run this with the ``-i`` or ``--interactive`` flag.::
codespell -I FILE, --ignore-words=FILE
The -I flag can be used to whitelist certain words that are in the ``codespell_lib/data/dictionary.txt``. The format of the whitelist file is one word per line. Invoke using: ``codespell -I path/to/file.txt`` to execute codespell referencing said whitelist. **Important note:** The whitelist passed to ``-I`` is case-sensitive based on how it is listed in ``dictionary.txt``. ::
codespell -L word1,word2,word3,word4
The -L flag can be used to whitelist certain words that are comma-separated placed immediately after it. ::
codespell -S, --skip=
Comma-separated list of files to skip. It accepts globs as well. Examples:
......@@ -47,15 +55,13 @@ Comma-separated list of files to skip. It accepts globs as well. Examples:
* to skip directories, invoke ``codespell --skip="./src/3rd-Party,./src/Test"`` ::
codespell -I FILE, --ignore-words=FILE
This flag can be used to whitelist certain words that are in the ``codespell_lib/data/dictionary.txt``. The format of the whitelist file is one word per line. Invoke using: ``codespell -I path/to/file.txt`` to execute codespell referencing said whitelist. **Important note:** The whitelist passed to ``-I`` is case-sensitive based on how it is listed in ``dictionary.txt``.
Useful commands::
codespell -d -q 3 --skip="*.po,*.ts,./src/3rdParty,./src/Test"
List all typos found except translation files and some directories. Display them without terminal colors and with a quiet level of 3. ::
| List all typos found except translation files and some directories.
Display them without terminal colors and with a quiet level of 3. ::
codespell -i 3 -w
......@@ -67,7 +73,7 @@ after applying them in projects like Linux Kernel, EFL, oFono among others.
You can provide your own version of the dictionary, but patches for
new/different entries are very welcome.
Want to know if a word you're proposing exists in codespell already? It is possible to test a word against the current dictionary that exists in ``codespell_lib/data/dictionary.txt`` via:
Want to know if a word you're proposing exists in codespell already? It is possible to test a word against the current dictionary that exists in ``codespell_lib/data/dictionary.txt`` via::
echo "word" | codespell -
echo "1stword,2ndword" | codespell -
......@@ -139,6 +145,16 @@ To stay current with codespell developments it is possible to build codespell fr
* It has been reported that after installing from ``pip``, codespell can't be located. Please check the $PATH variable to see if ``~/.local/bin`` is present. If it isn't then add it to your path.
* If you decide to install via ``pip`` then be sure to remove any previously installed versions of codespell (via you OSs preferred app mananger).
Updating the dictionary
-----------------------
In the scenario where the user prefers not to follow the development version of codespell yet still opts to benefit from the frequently updated `dictionary.txt` file, we recommend running a simple set of commands to achieve this ::
wget https://raw.githubusercontent.com/codespell-project/codespell/master/codespell_lib/data/dictionary.txt
codespell -D dictionary.txt
The above simply downloads the latest ``dictionary.txt`` file and then by utilizing the -D flag allows the user to specify the freshly downloaded ``dictionary.txt`` as the custom dictionary instead of the default one.
License
-------
......
clone_depth: 50
environment:
global:
PYTHON: "C:\\conda"
CONDA_DEPENDENCIES: "pytest pytest-cov setuptools flake8 coverage chardet"
PIP_DEPENDENCIES: "codecov pytest-sugar"
matrix:
- PYTHON_VERSION: "2.7"
PYTHON_ARCH: "32"
- PYTHON_VERSION: "3.6"
PYTHON_ARCH: "64"
- PYTHON: C:\Python37-x64
PYTHON_VERSION: 3.7
PYTHON_ARCH: 64
- PYTHON: C:\Python27
PYTHON_VERSION: 2.7
PYTHON_ARCH: 32
cache:
# Cache downloaded pip packages and built wheels.
- '%LOCALAPPDATA%\pip\Cache\http'
- '%LOCALAPPDATA%\pip\Cache\wheels'
install:
- "git clone git://github.com/astropy/ci-helpers.git"
- "powershell ci-helpers/appveyor/install-miniconda.ps1"
- "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
- "activate test"
- "pip install pytest pytest-cov setuptools flake8 coverage chardet codecov"
- "python setup.py develop"
build: false # Not a C# project, build stuff at the test step instead.
......@@ -25,3 +28,10 @@ test_script:
on_success:
- "codecov"
# Remove old or huge cache files to hopefully not exceed the 1GB cache limit.
# (adapted from PyInstaller)
- C:\cygwin\bin\find "%LOCALAPPDATA%\pip" -type f -mtime +360 -delete
- C:\cygwin\bin\find "%LOCALAPPDATA%\pip" -type f -size +10M -delete
- C:\cygwin\bin\find "%LOCALAPPDATA%\pip" -empty -delete
# Show size of cache
- C:\cygwin\bin\du -hs "%LOCALAPPDATA%\pip\Cache"
......@@ -26,12 +26,12 @@ from optparse import OptionParser
import os
import fnmatch
word_regex_def = u"[\w\-'’`]+"
word_regex_def = u"[\\w\\-'’`]+"
encodings = ('utf-8', 'iso-8859-1')
USAGE = """
\t%prog [OPTIONS] [file1 file2 ... fileN]
"""
VERSION = '1.14.0'
VERSION = '1.15.0'
# Users might want to link this file into /usr/local/bin, so we resolve the
# symbolic link path to the real path if necessary.
......@@ -283,6 +283,12 @@ def parse_options(args):
action='store_true', default=False,
help='Check hidden files (those starting with ".") as '
'well.')
parser.add_option('-A', '--after-context', metavar='LINES',
help='print LINES of trailing context', type='int')
parser.add_option('-B', '--before-context', metavar='LINES',
help='print LINES of leading context', type='int')
parser.add_option('-C', '--context', metavar='LINES',
help='print LINES of surrounding context', type='int')
(o, args) = parser.parse_args(list(args))
......@@ -411,8 +417,15 @@ def ask_for_word_fix(line, wrongword, misspelling, interactivity):
return misspelling.fix, fix_case(wrongword, misspelling.data)
def print_context(lines, index, context):
# context = (context_before, context_after)
for i in range(index - context[0], index + context[1] + 1):
if 0 <= i < len(lines):
print('%s %s' % ('>' if i == index else ':', lines[i].rstrip()))
def parse_file(filename, colors, summary, misspellings, exclude_lines,
file_opener, word_regex, options):
file_opener, word_regex, context, options):
bad_count = 0
lines = None
changed = False
......@@ -479,10 +492,14 @@ def parse_file(filename, colors, summary, misspellings, exclude_lines,
for word in word_regex.findall(line):
lword = word.lower()
if lword in misspellings:
context_shown = False
fix = misspellings[lword].fix
fixword = fix_case(word, misspellings[lword].data)
if options.interactive and lword not in asked_for:
if context is not None:
context_shown = True
print_context(lines, i, context)
fix, fixword = ask_for_word_fix(
lines[i], word, misspellings[lword],
options.interactive)
......@@ -527,6 +544,8 @@ def parse_file(filename, colors, summary, misspellings, exclude_lines,
# our bad_count and thus return value
bad_count += 1
if (not context_shown) and (context is not None):
print_context(lines, i, context)
if filename != '-':
print("%(FILENAME)s:%(LINE)s: %(WRONGWORD)s "
" ==> %(RIGHTWORD)s%(REASON)s"
......@@ -613,6 +632,26 @@ def main(*args):
else:
summary = None
context = None
if options.context is not None:
if (options.before_context is not None) or \
(options.after_context is not None):
print('ERROR: --context/-C cannot be used together with '
'--context-before/-B or --context-after/-A')
parser.print_help()
return 1
context_both = max(0, options.context)
context = (context_both, context_both)
elif (options.before_context is not None) or \
(options.after_context is not None):
context_before = 0
context_after = 0
if options.before_context is not None:
context_before = max(0, options.before_context)
if options.after_context is not None:
context_after = max(0, options.after_context)
context = (context_before, context_after)
exclude_lines = set()
if options.exclude_file:
build_exclude_hashes(options.exclude_file, exclude_lines)
......@@ -642,7 +681,7 @@ def main(*args):
continue
bad_count += parse_file(
fname, colors, summary, misspellings, exclude_lines,
file_opener, word_regex, options)
file_opener, word_regex, context, options)
# skip (relative) directories
dirs[:] = [dir_ for dir_ in dirs if not glob_match.match(dir_)]
......@@ -650,7 +689,7 @@ def main(*args):
else:
bad_count += parse_file(
filename, colors, summary, misspellings, exclude_lines,
file_opener, word_regex, options)
file_opener, word_regex, context, options)
if summary:
print("\n-------8<-------\nSUMMARY:")
......
This diff is collapsed.
......@@ -302,6 +302,65 @@ def test_case_handling(tmpdir, capsys):
os.remove(f.name)
def test_context(tmpdir, capsys):
"""Test context options."""
d = str(tmpdir)
with open(op.join(d, 'context.txt'), 'w') as f:
f.write('line 1\nline 2\nline 3 abandonned\nline 4\nline 5')
# symmetric context, fully within file
cs.main('-C', '1', d)
lines = capsys.readouterr()[0].split('\n')
assert len(lines) == 5
assert lines[0] == ': line 2'
assert lines[1] == '> line 3 abandonned'
assert lines[2] == ': line 4'
# requested context is bigger than the file
cs.main('-C', '10', d)
lines = capsys.readouterr()[0].split('\n')
assert len(lines) == 7
assert lines[0] == ': line 1'
assert lines[1] == ': line 2'
assert lines[2] == '> line 3 abandonned'
assert lines[3] == ': line 4'
assert lines[4] == ': line 5'
# only before context
cs.main('-B', '2', d)
lines = capsys.readouterr()[0].split('\n')
assert len(lines) == 5
assert lines[0] == ': line 1'
assert lines[1] == ': line 2'
assert lines[2] == '> line 3 abandonned'
# only after context
cs.main('-A', '1', d)
lines = capsys.readouterr()[0].split('\n')
assert len(lines) == 4
assert lines[0] == '> line 3 abandonned'
assert lines[1] == ': line 4'
# asymmetric context
cs.main('-B', '2', '-A', '1', d)
lines = capsys.readouterr()[0].split('\n')
assert len(lines) == 6
assert lines[0] == ': line 1'
assert lines[1] == ': line 2'
assert lines[2] == '> line 3 abandonned'
assert lines[3] == ': line 4'
# both '-C' and '-A' on the command line
cs.main('-C', '2', '-A', '1', d)
lines = capsys.readouterr()[0].split('\n')
assert 'ERROR' in lines[0]
# both '-C' and '-B' on the command line
cs.main('-C', '2', '-B', '1', d)
lines = capsys.readouterr()[0].split('\n')
assert 'ERROR' in lines[0]
@contextlib.contextmanager
def FakeStdin(text):
if sys.version[0] == '2':
......
......@@ -23,17 +23,32 @@ def test_dictionary_formatting():
assert not re.match(r'^\s.*', rep), ('error %s: correction %r '
'cannot start with whitespace'
% (err, rep))
for (r, msg) in [
(r'^,', 'error %s: correction %r starts with a comma'),
(r'\s,', 'error %s: correction %r contains a whitespace '
'character followed by a comma'),
(r',\s\s', 'error %s: correction %r contains a comma followed '
'by multiple whitespace characters'),
(r',[^ ]', 'error %s: correction %r contains a comma *not* '
'followed by a space')
]:
assert not re.search(r, rep), (msg % (err, rep))
if rep.count(','):
if not rep.endswith(','):
assert 'disabled' in rep.split(',')[-1], \
('currently corrections must end with trailing "," (if'
' multiple corrections are available) or '
'have "disabled" in the comment')
err_dict[err] = rep
reps = [r.strip() for r in rep.lower().split(',')]
reps = [r for r in reps if len(r)]
err_dict[err] = reps
unique = list()
for r in reps:
if r not in unique:
unique.append(r)
assert reps == unique, 'entries are not (lower-case) unique'
# check for corrections that are errors (but not self replacements)
for err in err_dict:
for r in err_dict[err]:
assert (r not in err_dict) or (r in err_dict[r]), \
('error %s: correction %s is an error itself' % (err, r))
codespell (1.15.0-1~exp1) experimental; urgency=medium
* New upstream release
-- Sylvestre Ledru <sylvestre@debian.org> Fri, 17 May 2019 20:23:02 +0200
codespell (1.14.0-1) unstable; urgency=medium
* New upstream release
......