Commit 009aa709 authored by Antonio Valentino's avatar Antonio Valentino

Import Upstream version 0.8.6+ds

parents
[bumpversion]
current_version = 0.8.3
commit = True
tag = True
[bumpversion:file:pyspectral/version.py]
etc/pyspectral_rsr_data.tgz filter=lfs diff=lfs merge=lfs -text
pyspectral/etc/pyspectral_rsr_data.tgz filter=lfs diff=lfs merge=lfs -text
doc/_static/mersi2_rsr_band_0040_0070.png filter=lfs diff=lfs merge=lfs -text
doc/_static/mersi2_rsr_band_0040_0070_missingbands.png filter=lfs diff=lfs merge=lfs -text
pyspectral/version.py export-subst
# emacs temps
*~
*.py[cod~]
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
var
sdist
develop-eggs
.installed.cfg
lib
lib64
__pycache__
doc/_build
*png
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
## Version <RELEASE_VERSION> (2018/12/04)
### Issues Closed
* [Issue 38](https://github.com/pytroll/pyspectral/issues/38) - Download RSR data in vain
In this release 1 issue was closed.
### Pull Requests Merged
#### Features added
* [PR 60](https://github.com/pytroll/pyspectral/pull/60) - Fix readthedocs
In this release 1 pull request was closed.
## Version <RELEASE_VERSION> (2018/11/30)
### Issues Closed
* [Issue 50](https://github.com/pytroll/pyspectral/issues/50) - Re-download of RSR files ([PR 56](https://github.com/pytroll/pyspectral/pull/56))
In this release 1 issue was closed.
### Pull Requests Merged
#### Bugs fixed
* [PR 53](https://github.com/pytroll/pyspectral/pull/53) - Fix masking of calculated NIR reflectances
#### Features added
* [PR 57](https://github.com/pytroll/pyspectral/pull/57) - Script cleanup and document
* [PR 56](https://github.com/pytroll/pyspectral/pull/56) - Rsr download fix ([50](https://github.com/pytroll/pyspectral/issues/50))
In this release 3 pull requests were closed.
## Version <RELEASE_VERSION> (2018/11/30)
### Issues Closed
* [Issue 50](https://github.com/pytroll/pyspectral/issues/50) - Re-download of RSR files ([PR 56](https://github.com/pytroll/pyspectral/pull/56))
In this release 1 issue was closed.
### Pull Requests Merged
#### Bugs fixed
* [PR 53](https://github.com/pytroll/pyspectral/pull/53) - Fix masking of calculated NIR reflectances
#### Features added
* [PR 57](https://github.com/pytroll/pyspectral/pull/57) - Script cleanup and document
* [PR 56](https://github.com/pytroll/pyspectral/pull/56) - Rsr download fix ([50](https://github.com/pytroll/pyspectral/issues/50))
In this release 3 pull requests were closed.
This diff is collapsed.
include pyspectral/etc/*
include pyspectral/data/e490_00a.dat
include pyspectral/data/MSG_SEVIRI_Spectral_Response_Characterisation.XLS
include LICENSE.txt
include versioneer.py
include pyspectral/version.py
PySpectral
==========
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/9f039d7d640846ca89be8a78fa11e1f6)](https://www.codacy.com/app/adybbroe/pyspectral?utm_source=github.com&utm_medium=referral&utm_content=pytroll/pyspectral&utm_campaign=badger)
[![Build Status](https://travis-ci.org/pytroll/pyspectral.png?branch=master)](https://travis-ci.org/pytroll/pyspectral)
[![Build status](https://ci.appveyor.com/api/projects/status/5lm42n0l65l5o9xn?svg=true)](https://ci.appveyor.com/project/pytroll/pyspectral)
[![Coverage Status](https://coveralls.io/repos/github/pytroll/pyspectral/badge.svg?branch=master)](https://coveralls.io/github/pytroll/pyspectral?branch=master)
[![Code Health](https://landscape.io/github/pytroll/pyspectral/master/landscape.png)](https://landscape.io/github/pytroll/pyspectral/master)
[![PyPI version](https://badge.fury.io/py/pyspectral.svg)](https://badge.fury.io/py/pyspectral)
[![Research software impact](http://depsy.org/api/package/pypi/pyspectral/badge.svg)](http://depsy.org/package/python/pyspectral)
[![Code Climate](https://codeclimate.com/github/pytroll/pyspectral/badges/gpa.svg)](https://codeclimate.com/github/pytroll/pyspectral)
[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/pytroll/pyspectral/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/pytroll/pyspectral/?branch=master)
Given a passive sensor on a meteorological satellite PySpectral provides the
relative spectral response (rsr) function(s) and offer you some basic
operations like convolution with the solar spectrum to derive the in band solar
flux, for instance. The focus is on imaging sensors like AVHRR, VIIRS, MODIS, ABI,
AHI, OLCI and SEVIRI. But more sensors are included and if others are needed they can
be easily added. With PySpectral it is possible to derive the reflective and
emissive parts of the signal observed in any NIR band around 3-4 microns where
both passive terrestrial emission and solar backscatter mix the information
received by the satellite. Furthermore PySpectral allows correcting true color
imagery for the background (climatological) atmospheric signal due to Rayleigh
scattering of molecules, absorption by atmospheric gases and aerosols, and Mie
scattering of aerosols.
Adam Dybbroe
March 2018, Norrkoping, Sweden
# Releasing PySpectral
prerequisites: `pip install setuptools twine`
1. checkout master
2. pull from repo
3. run the unittests
4. run `loghub` and update the `CHANGELOG.md` file:
```
loghub pytroll/pyspectral -u <username> -st v0.8.3 -plg bug "Bugs fixed" -plg enhancement "Features added" -plg documentation "Documentation changes"
```
Don't forget to commit!
5. Create a tag with the new version number, starting with a 'v', eg:
```
git tag v0.8.4 -m "Version 0.8.4
```
See [semver.org](http://semver.org/) on how to write a version number.
6. push changes to github `git push --follow-tags`
7. Verify travis tests passed and deployed sdist and wheel to PyPI
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (c) 2016-2018 Adam.Dybbroe
# Author(s):
# Adam.Dybbroe <adam.dybbroe@smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""A very basic (example) plotting program to display spectral responses for a
set of satellite instruments for a give wavelength range
"""
import matplotlib.pyplot as plt
from pyspectral.rsr_reader import RelativeSpectralResponse
from pyspectral.utils import get_bandname_from_wavelength
from pyspectral.utils import logging_on, logging_off, get_logger
import numpy as np
def plot_band(plt_in, band_name, rsr_obj, **kwargs):
"""Do the plotting of one band"""
if 'platform_name_in_legend' in kwargs:
platform_name_in_legend = kwargs['platform_name_in_legend']
else:
platform_name_in_legend = False
detectors = rsr_obj.rsr[band_name].keys()
# for det in detectors:
det = sorted(detectors)[0]
resp = rsr_obj.rsr[band_name][det]['response']
wvl = rsr_obj.rsr[band_name][det]['wavelength']
resp = np.ma.masked_less_equal(resp, minimum_response)
wvl = np.ma.masked_array(wvl, resp.mask)
resp.compressed()
wvl.compressed()
if platform_name_in_legend:
plt_in.plot(wvl, resp, label='{platform} - {band}'.format(
platform=rsr_obj.platform_name, band=band_name))
else:
plt_in.plot(wvl, resp, label='{band}'.format(band=band_name))
return plt_in
def get_arguments():
"""Get the command line arguments"""
import argparse
parser = argparse.ArgumentParser(
description='Plot spectral responses for a set of satellite imagers')
parser.add_argument("--platform_name", '-p', nargs='*',
help="The Platform name",
type=str, required=True)
parser.add_argument("--sensor", '-s', nargs='*',
help="The sensor/instrument name",
type=str, required=True)
parser.add_argument("-x", "--xlimits", nargs=2,
help=("x-axis boundaries for plot"),
default=None, type=float)
parser.add_argument("-t", "--minimum_response",
help=("Minimum response: Any response lower than " +
"this will be ignored when plotting"),
default=0.015, type=float)
parser.add_argument("-no_platform_name_in_legend", help=("No platform name in legend"),
action='store_true')
parser.add_argument("--title", help=("Plot title"),
default=None, type=str)
parser.add_argument("--wavelength_resolution",
help=("The step in wavelength (nanometers) when scanning\n" +
" the spectral range trying to find bands"),
default=0.005, type=float)
parser.add_argument("-o", "--filename", help=("Output plot file name"),
default=None, type=str)
parser.add_argument(
"-v", '--verbose', help=("Turn logging on"), action='store_true')
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--bandname", '-b',
help="The sensor band name", type=str)
group.add_argument("--wavelength", "-w", type=float,
help='the approximate spectral wavelength in micron')
group.add_argument("--range", "-r", nargs='*',
help="The wavelength range for the plot",
default=[None, None], type=float)
return parser.parse_args()
if __name__ == "__main__":
import sys
args = get_arguments()
LOG = get_logger(__name__)
platform_names = args.platform_name
sensors = args.sensor
minimum_response = args.minimum_response
xlimits = args.xlimits
title = args.title
if not title:
title = 'Relative Spectral Responses'
filename = args.filename
no_platform_name_in_legend = args.no_platform_name_in_legend
wavel_res = args.wavelength_resolution
verbose = args.verbose
if verbose:
logging_on()
else:
logging_off()
req_wvl = None
band = None
wvlmin, wvlmax = args.range
if args.bandname:
band = args.bandname
elif args.wavelength:
req_wvl = args.wavelength
figscale = 1.0
if wvlmin:
figscale = (wvlmax - wvlmin) / 4.
figsize = (figscale * 1. + 10, figscale * 0.5 + 5)
plt.figure(figsize=figsize)
something2plot = False
for platform in platform_names:
for sensor in sensors:
try:
rsr = RelativeSpectralResponse(platform, sensor)
except IOError:
# LOG.exception('Failed getting the rsr data for platform %s ' +
# 'and sensor %s', platform, sensor)
rsr = None
else:
break
if not rsr:
continue
something2plot = True
LOG.debug("Platform name %s and Sensor: %s", str(rsr.platform_name), str(rsr.instrument))
if req_wvl:
bands = get_bandname_from_wavelength(sensor, req_wvl, rsr.rsr, 0.25, multiple_bands=True)
if not bands:
continue
if isinstance(bands, list):
for b__ in bands:
plt = plot_band(plt, b__, rsr,
platform_name_in_legend=(not no_platform_name_in_legend))
else:
plt = plot_band(plt, bands, rsr,
platform_name_in_legend=(not no_platform_name_in_legend))
elif band:
plt = plot_band(plt, band, rsr,
platform_name_in_legend=(not no_platform_name_in_legend))
else:
wvlx = wvlmin
prev_band = None
while wvlx < wvlmax:
bands = get_bandname_from_wavelength(sensor, wvlx, rsr.rsr, wavel_res, multiple_bands=True)
if isinstance(bands, list):
b__ = bands[0]
for b in bands[1:]:
LOG.warning("Skipping band %s", str(b))
else:
b__ = bands
wvlx = wvlx + wavel_res / 5.
if not b__:
continue
if b__ != prev_band:
plt = plot_band(plt, b__, rsr,
platform_name_in_legend=(not no_platform_name_in_legend))
prev_band = b__
if not something2plot:
LOG.error("Nothing to plot!")
sys.exit(0)
wmin, wmax = plt.xlim()
delta_x = (wmax - wmin)
wmax = wmax + delta_x / 4.0
if xlimits:
wmin = xlimits[0]
wmax = xlimits[1]
plt.xlim((wmin, wmax))
plt.title(title)
plt.legend(loc='lower right')
if filename:
plt.savefig(filename)
else:
if req_wvl:
plt.savefig('rsr_band_{:>04d}.png'.format(int(100 * req_wvl)))
elif wvlmin and wvlmax:
plt.savefig('rsr_band_{:>04d}_{:>04d}.png'.format(
int(100 * wvlmin), int(100 * wvlmax)))
else:
plt.savefig('rsr_band_{bandname}.png'.format(bandname=band))
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (c) 2018 Adam.Dybbroe
# Author(s):
# Adam.Dybbroe <adam.dybbroe@smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Script to download the the LUT files for atmospheric correction in the SW
spectral range
"""
import logging
import argparse
from pyspectral.utils import download_luts, AEROSOL_TYPES
from pyspectral.utils import logging_on, logging_off
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description='Download the atm correction LUT files')
parser.add_argument("--aerosol_types", '-a', nargs='*',
help="Aerosol types",
type=str, default=AEROSOL_TYPES)
parser.add_argument(
"-d", '--dry_run', help=("Dry run - no action"), action='store_true',
default=False)
parser.add_argument(
"-v", '--verbose', help=("Turn logging on"), action='store_true')
args = parser.parse_args()
verbose = args.verbose
dry_run = args.dry_run
aerosol_types = args.aerosol_types
if verbose:
logging_on(logging.DEBUG)
else:
logging_off()
# Download:
download_luts(aerosol_type=aerosol_types, dry_run=dry_run)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (c) 2018 Adam.Dybbroe
# Author(s):
# Adam.Dybbroe <adam.dybbroe@smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Script to download the RSR files from internet
"""
import logging
import argparse
from pyspectral.utils import download_rsr
from pyspectral.utils import logging_on, logging_off
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description='Download relative spectral response data in hdf5')
parser.add_argument("-o", "--destination", help=("Destination path where to store the files"),
default=None, type=str)
parser.add_argument(
"-d", '--dry_run', help=("Dry run - no action"), action='store_true',
default=False)
parser.add_argument(
"-v", '--verbose', help=("Turn logging on"), action='store_true')
args = parser.parse_args()
dest_dir = args.destination
verbose = args.verbose
dry_run = args.dry_run
if verbose:
logging_on(logging.DEBUG)
else:
logging_off()
if dest_dir:
download_rsr(dest_dir=dest_dir, dry_run=dry_run)
else:
download_rsr(dry_run=dry_run)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (c) 2015-2018 Adam.Dybbroe@smhi.se
# Author(s):
# Adam.Dybbroe@smhi.se <adam.dybbroe@smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Plot some spectral responses
"""
import argparse
import matplotlib.pyplot as plt
from pyspectral.rsr_reader import RelativeSpectralResponse
from pyspectral.utils import get_bandname_from_wavelength
import numpy as np
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description='Plot relative spectral response for a given sensor band')
parser.add_argument("platform_name", metavar='p',
help="The Platform name",
type=str)
parser.add_argument("sensor", metavar='s',
help="The sensor/instrument name",
type=str)
parser.add_argument("-r", "--minimum_response",
help=("Minimum response: Any response lower than " +
"this will be ignored when plotting"),
default=0.01, type=float)
parser.add_argument("-x", "--xlimits", nargs=2,
help=("x-axis boundaries for plot"),
default=None, type=float)
parser.add_argument("--title", help=("Plot title"),
default=None, type=str)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--wavelength", '-w',
help="The wavelength in micrometers", type=float)
group.add_argument("--bandname", '-b',
help="The sensor band name", type=str)
args = parser.parse_args()
title = args.title
if not title:
title = 'Relative Spectral Responses'
platform = args.platform_name
sensor = args.sensor
minimum_response = args.minimum_response
xlimits = args.xlimits
rsr = RelativeSpectralResponse(platform, sensor)
if args.bandname:
band = args.bandname
else:
wavelength = args.wavelength
band = get_bandname_from_wavelength(sensor, wavelength, rsr.rsr)
detectors = rsr.rsr[band].keys()
for det in detectors:
resp = rsr.rsr[band][det]['response']
wvl = rsr.rsr[band][det]['wavelength']
resp = np.ma.masked_less_equal(resp, minimum_response)
wvl = np.ma.masked_array(wvl, resp.mask)
resp.compressed()
wvl.compressed()
plt.plot(wvl, resp)
if xlimits:
plt.xlim(xlimits[0], xlimits[1])
plt.title(title)
plt.savefig('{}_{}_rsr_band{}.png'.format(platform, sensor, band))
This diff is collapsed.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (c) 2017 Adam.Dybbroe
# Author(s):
# Adam.Dybbroe <a000680@c20671.ad.smhi.se>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Compare the content of two hdf5 files with rsr data
"""
import pdb
import numpy as np
from pyspectral.rsr_reader import RelativeSpectralResponse
#FILE1 = '/home/a000680/data/pyspectral/rsr_viirs_Suomi-NPP.h5'
#FILE2 = '/home/a000680/data/pyspectral/BACKUP_rsr_viirs_Suomi-NPP.h5'
FILE1 = "/home/a000680/data/pyspectral/OLDrsr_ahi_Himawari-8.h5"
FILE2 = "/home/a000680/data/pyspectral/rsr_ahi_Himawari-8.h5"
rsr_a = RelativeSpectralResponse(filename=FILE1)
rsr_b = RelativeSpectralResponse(filename=FILE2)
for band in rsr_a.rsr:
for det in rsr_a.rsr[band]:
wvl1 = rsr_a.rsr[band][det]['wavelength']
wvl2 = rsr_b.rsr[band][det]['wavelength']
try:
if not np.allclose(wvl1, wvl2):
pdb.set_trace()
except ValueError:
pdb.set_trace()
resp1 = rsr_a.rsr[band][det]['response']
resp2 = rsr_b.rsr[band][det]['response']
try:
if not np.allclose(resp1, resp2):
pdb.set_trace()
except ValueError:
pdb.set_trace()
cw1 = rsr_a.rsr[band][det]['central_wavelength']
cw2 = rsr_b.rsr[band][det]['central_wavelength']
if np.abs(cw1 - cw2) > 0.00001:
pdb.set_trace()
This diff is collapsed.
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"