Commit 15469f05 authored by Ole Streicher's avatar Ole Streicher

Upstream version 4.16

parents
This package contains the source code for the fitsverify FITS file
conformance verification program. It is maintained by the HEASARC
at NASA/GSFC (http://heasarc.gsfc.nasa.gov). This is a stand-alone
version of the fverify task in the FTOOLS package.
The CFITSIO library, also available from the HEASARC, is required
to compile and build fitsverify. In particular, the fitsio.h, and
longnam.h include files are needed to compile the program, which
must then be linked with the CFITSIO library itself (i.e., the file
libcfitsio.a on unix systems).
fitsverify can be built on most unix machines with a command similar
to the following:
gcc -o fitsverify ftverify.c fvrf_data.c fvrf_file.c fvrf_head.c fvrf_key.c
fvrf_misc.c -DSTANDALONE -L. -lcfitsio -lm -lnsl -lsocket
(The '-lm', '-lnsl' and '-lsocket' library directives may not be
required on all platforms.)
Optionally, one may add the -DERR2OUT flag to force all the error
messages to go to stdout rather than stderr (usually only useful
if the output of fitsverify is being piped to another file or task).
To build fitsverify with Visual C++ on Windows PCs, first execute the
vcvars32.bat file that is distributed with vcc, unpack the fitverify
.zip file, then excute the following command:
cl /MD ftverify.c fvrf_data.c fvrf_file.c fvrf_head.c fvrf_key.c
fvrf_misc.c -DSTANDALONE cfitsio.lib
A help file describing the features of fitsverify can be produced
by entering the command 'fitsverify -h'. The help file
is reproduced below:
----------------------------------------------------------------
fitsverify -- Verify that the input files conform to the FITS Standard.
USAGE: fitsverify filename ... - verify one or more FITS files
(may use wildcard characters)
or fitsverify @filelist.txt - verify a list of FITS files
Optional flags:
-l list all header keywords
-q quiet; print one-line pass/fail summary per file
-e only test for error conditions (ignore warnings)
fitsverify exits with a status equal to the number of errors + warnings.
EXAMPLES:
fitsverify -l m101.fits - produce a detailed verificaton report of
a single file, including a keyword listing
fitsverify -q *.fits *.fit - verify all files with .fits or .fit
extensions, writing a 1-line pass/fail
message for each file
DESCRIPTION:
This task reads one or more input FITS files and verifies that the
files conform to the specifications of the FITS Standard document
(known as the NASA/Science Office of Standards and Technology 'Definition
of the Current FITS Standard', document number, NOST 100-2.0, available
online at http://fits.gsfc.nasa.gov/). The input filename template may
contain wildcard characters, in which case all matching files will be
tested. Alternatively, the name of an ASCII text file containing a list
of file names, one per line, may be entered preceded by an '@' character.
The following error or warning conditions will be reported:
ERROR CONDITIONS
- Mandatory keyword not present or out of order
- Mandatory keyword has wrong datatype or illegal value
- END header keyword is not present
- Sum of table column widths is inconsistent with NAXIS1 value
- BLANK keyword present in image with floating-point datatype
- TNULLn keyword present for floating-point binary table column
- Bit column has non-zero fill bits or is not left adjusted
- ASCII TABLE column contains illegal value inconsistent with TFORMn
- Address to a variable length array not within the data heap
- Extraneous bytes in the FITS file following the last HDU
- Mandatory keyword values not expressed in fixed format
- Mandatory keyword duplicated elsewhere in the header
- Header contains illegal ASCII character (not ASCII 32 - 126)
- Keyword name contains illegal character
- Keyword value field has illegal format
- Value and comment fields not separated by a slash character
- END keyword not filled with blanks in columns 9 - 80
- Reserved keyword with wrong datatype or illegal value
- EXTEND not present in the primary array if there are extensions
- BLOCKED, if present, not among the first 36 keywords
- XTENSION keyword in the primary array
- Column related keyword (TFIELDS, TTYPEn,TFORMn, etc.) in an image
- SIMPLE, EXTEND, or BLOCKED keyword in any extension
- BSCALE, BZERO, BUNIT, BLANK, DATAMAX, DATAMIN keywords in a table
- Image WCS keywords (CTYPEn, CRPIXn, CRVALn, etc.) in a table
- Table WCS keywords (TCTYPn, TCRPXn, TCRVLn, etc.) in an image
- TDIMn or THEAP keyword in an ASCII table
- TBCOLn keyword in a Binary table
- THEAP keyword in a binary table that has PCOUNT = 0
- XTENSION, TFORMn, TDISPn or TDIMn value contains leading space(s)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of WCSAXES
- Index of any table column descriptor keyword (TTYPEn, TFORMn,
etc.) greater than value of TFIELDS
- TSCALn or TZEROn present for an ASCII, logical, or Bit column
- TDISPn value is inconsistent with the column datatype
- Length of a variable length array greater than the maximum
length as given by the TFORMn keyword
- Logical column contains illegal value not equal to 'T', 'F', or 0
- Character string column contains non-ASCII text character
- Header fill bytes not all blanks
- Data fill bytes not all blanks in ASCII tables or all zeros
in any other type of HDU
- Gaps between defined ASCII table columns contain characters with
ASCII value > 127
WARNING CONDITIONS
- SIMPLE = F
- Presence of deprecated keywords BLOCKED or EPOCH
- 2 HDUs have identical EXTNAME, EXTVER, and EXTLEVEL values
- BSCALE or TSCALn value = 0.
- BLANK OR TNULLn value exceeds the legal range
- TFORMn has 'rAw' format and r is not a multiple of w
- DATE = 'dd/mm/yy' and yy is less than 10 (Y2K problem??)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of NAXIS, if the WCSAXES keyword is not present
- Duplicated keyword (except COMMENT, HISTORY, blank, etc.)
- Column name (TTYPEn) does not exist or contains characters
other than letter, digit and underscore
- Calculated checksum inconsistent with CHECKSUM or DATASUM keyword
This is the stand alone version of the FTOOLS 'fverify' program. It is
maintained by the HEASARC at NASA/GSFC. Any comments about this program
should be submitted to http://heasarc.gsfc.nasa.gov/cgi-bin/ftoolshelp
This diff is collapsed.
This diff is collapsed.
#ifndef _FVERIFY_H
#define _FVERIFY_H
#include <stdio.h>
#include <ctype.h>
#include <string.h>
#include <stdlib.h>
#include <math.h>
#include <limits.h>
#include "fitsio.h"
#define MAXERRORS 200
#define MAXWRNS 200
static char errmes[256];
static char comm[256];
extern int prhead;
extern int testdata;
extern int testfill;
extern int testcsum;
extern int totalhdu; /* total number of hdu */
extern int err_report;
extern int heasarc_conv;
extern int prstat;
/********************************
* *
* Keywords *
* *
********************************/
typedef enum { STR_KEY, /* string key */
LOG_KEY, /* Logical key */
INT_KEY, /* Integer key */
FLT_KEY, /* Float key */
CMI_KEY, /* Complex integer key */
CMF_KEY, /* Complex float key */
COM_KEY, /* history, comment, " ", and end */
UNKNOWN /* Unknown types */
} kwdtyp;
/* error number masks of the keyword test */
#define BAD_STR 0X0001
#define NO_TRAIL_QUOTE 0X0002
#define BAD_NUM 0X0004
#define LOWCASE_EXPO 0X0008
#define NO_TRAIL_PAREN 0X0010
#define NO_COMMA 0X0020
#define TOO_MANY_COMMA 0X0040
#define BAD_REAL 0X0080
#define BAD_IMG 0X0100
#define BAD_LOGICAL 0x0200
#define NO_START_SLASH 0X0400
#define BAD_COMMENT 0x0800
#define UNKNOWN_TYPE 0x1000
/* keyword structure */
typedef struct {
char kname[FLEN_KEYWORD]; /* fits keyword name */
kwdtyp ktype; /* fits keyword type */
char kvalue[FLEN_VALUE]; /* fits keyword name */
int kindex; /* position at the header */
int goodkey; /* good keyword flag (=1 good)*/
}FitsKey;
int fits_parse_card(FILE *out, int pos, char *card, char *kname, kwdtyp *ktype,
char *kvalue, char *kcomm);
void get_str(char **p, char *kvalue, unsigned long *stat);
void get_log(char **p, char *kvalue, unsigned long *stat);
void get_num(char **p, char *kvalue, kwdtyp *ktype, unsigned long *stat);
void get_cmp(char **p, char *kvalue, kwdtyp *ktype, unsigned long *stat);
int check_str(FitsKey* pkey, FILE *out);
int check_int(FitsKey* pkey, FILE *out);
int check_flt(FitsKey* pkey, FILE *out);
int check_cmi(FitsKey* pkey, FILE *out);
int check_cmf(FitsKey* pkey, FILE *out);
int check_log(FitsKey* pkey, FILE *out);
int check_fixed_int(char *card, FILE *out);
int check_fixed_log(char *card, FILE *out);
int check_fixed_str(char *card, FILE *out);
void get_unknown(char **p, char *kvalue, kwdtyp *ktype, unsigned long *stat);
void get_comm(char **p, char *kcomm, unsigned long *stat);
void pr_kval_err(FILE *out, int pos, char *keyname, char *keyval,
unsigned long stat);
/********************************
* *
* Headers *
* *
********************************/
typedef struct {
int hdutype; /* hdutype */
int hdunum; /* hdunum */
int isgroup; /* random group flag */
int istilecompressed; /* tile compressed image */
int gcount; /* gcount */
int pcount; /* pcount */
int bitpix; /* pix number */
int naxis; /* number of the axis,used for image array*/
LONGLONG *naxes; /* dimension of each axis,used for image array*/
int ncols; /* number of the columns, used for image only*/
char extname[FLEN_VALUE]; /* EXTENSION NAME */
int extver; /* extension version */
char **datamax; /* strings for the maximum of the data in a column */
char **datamin; /* strings for the minimum of the data in a column */
char **tnull; /* number of NULL values */
int nkeys; /* number of keys */
int tkeys; /* total of the keys tested*/
int heap; /* heap */
FitsKey **kwds; /* keywords list starting from the
last NAXISn keyword. The array
is sorted in the ascending alphabetical
order of keyword names. The last keyword END
and commentary keywords are excluded.
The total number of element, tkey, is
nkeys - 4 - naxis - ncomm. */
int use_longstr; /* flag indicates that the long string
convention is used */
}FitsHdu;
typedef struct {
char * name;
int index;
}ColName;
void verify_fits(char *infile, FILE *out);
void leave_early (FILE* out);
void close_err(FILE* out);
void init_hdu(fitsfile *infits, FILE *out, int hdunum, int hdutype,
FitsHdu *hduptr);
void test_hdu(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_ext(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_tbl(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_array(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_prm(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_img_ext(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_asc_ext(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_bin_ext(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_header(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void key_match(char **strs, int nstr, char **pattern, int exact,
int *ikey, int *mkey);
void test_colnam(FILE *out, FitsHdu *hduptr);
void parse_vtform(fitsfile *infits, FILE *out, FitsHdu *hduptr,
int colnum, int *datacode, long *maxlen);
void print_title(FILE* out, int hdunum, int hdutype);
void print_header(FILE* out);
void print_summary(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void close_hdu(FitsHdu *hduptr);
/********************************
* *
* Data *
* *
********************************/
void test_data(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_agap(fitsfile *infits, FILE *out, FitsHdu *hduptr);
void test_checksum(fitsfile *infits, FILE *out);
int iterdata(long totaln, long offset, long firstn, long nrows,
int narrays, iteratorCol *iter_col, void *usrdata);
/********************************
* *
* Files *
* *
********************************/
typedef struct {
int hdutype; /* hdutype */
int hdunum; /* hdunum */
char extname[FLEN_VALUE]; /* extension name, used for extension*/
int extver; /* extension version, used for extension */
int errnum; /* number of errors in this hdu */
int wrnno; /* number of warnning in this hdu */
}HduName;
int get_total_warn();
int get_total_err();
void init_hduname();
void set_hduname(int hdunum,int hdutype, char* extname,int extver);
void set_hduerr(int hdunum);
void set_hdubasic(int hdunum,int hdutype);
int test_hduname(int hdunum1, int hdunum2);
void total_errors (int *totalerr, int * totalwrn);
void hdus_summary(FILE *out);
void destroy_hduname();
void test_end(fitsfile *infits, FILE *out);
void init_report(FILE *out, char *rootnam);
void close_report(FILE *out);
void update_parfile(int numerr, int numwrn);
/********************************
* *
* Miscellaneous *
* *
********************************/
void print_fmt(FILE *out, char *temp, int nprompt);
int wrtout (FILE *out,char *comm);
int wrterr (FILE *out,char *comm, int severity);
int wrtwrn (FILE *out,char *comm, int heasarc);
int wrtferr(FILE *out, char* mess, int *status, int severity);
int wrtserr(FILE *out, char* mess, int *status, int severity);
void wrtsep (FILE *out,char fill, char *title, int nchar);
void num_err_wrn(int *num_err, int *num_wrn);
void reset_err_wrn();
int compkey (const void *key1, const void *key2);
int compcol (const void *col1, const void *col2);
int compcol (const void *col1, const void *col2);
int compstrp (const void *str1, const void *str2);
int compstre (const void *str1, const void *str2);
#endif
This diff is collapsed.
#include "fverify.h"
static HduName **hduname;
static int total_err=1; /* initialzed to 1 in case fail to open file */
static int total_warn=0;
int get_total_warn()
{
return (total_warn);
}
int get_total_err()
{
return (total_err);
}
/* Get the total hdu number and allocate the memory for hdu array */
void init_hduname()
{
int i;
/* allocate memories for the hdu structure array */
hduname = (HduName **)malloc(totalhdu*sizeof(HduName *));
for (i=0; i < totalhdu; i++) {
hduname[i] = (HduName *)calloc(1, sizeof(HduName));
hduname[i]->hdutype = -1;
hduname[i]->errnum = 0;
hduname[i]->wrnno = 0;
strcpy(hduname[i]->extname,"");
hduname[i]->extver = 0;
}
return;
}
/* set the hduname memeber hdutype, extname, extver */
void set_hduname( int hdunum, /* hdu number */
int hdutype, /* hdutype */
char* extname, /* extension name */
int extver /* extension version */
)
{
int i;
i = hdunum - 1;
hduname[i]->hdutype = hdutype;
if(extname!=NULL)
strcpy (hduname[i]->extname,extname);
else
strcpy(hduname[i]->extname,"");
hduname[i]->extver = extver;
return;
}
/* get the total errors and total warnings in this hdu */
void set_hduerr(int hdunum /* hdu number */
)
{
int i;
i = hdunum - 1;
num_err_wrn(&(hduname[i]->errnum), &(hduname[i]->wrnno));
reset_err_wrn(); /* reset the error and warning counter */
return;
}
/* set the basic information for hduname structure */
void set_hdubasic(int hdunum,int hdutype)
{
set_hduname(hdunum, hdutype, NULL, 0);
set_hduerr(hdunum);
return;
}
/* test to see whether the two extension having the same name */
/* return 1: identical 0: different */
int test_hduname (int hdunum1, /* index of first hdu */
int hdunum2 /* index of second hdu */
)
{
HduName *p1;
HduName *p2;
p1 = hduname[hdunum1-1];
p2 = hduname[hdunum2-1];
if(!strlen(p1->extname) || !strlen(p2->extname)) return 0;
if(!strcmp(p1->extname,p2->extname) && p1->hdutype == p2->hdutype
&& p2->extver == p1->extver && hdunum1 != hdunum2){
return 1;
}
return 0;
}
/* Added the error numbers */
void total_errors (int *toterr, int * totwrn)
{
int i = 0;
int ierr, iwrn;
*toterr = 0;
*totwrn = 0;
if (totalhdu == 0) { /* this means the file couldn't be opened */
*toterr = 1;
return;
}
for (i = 0; i < totalhdu; i++) {
*toterr += hduname[i]->errnum;
*totwrn += hduname[i]->wrnno;
}
/*check the end of file errors */
num_err_wrn(&ierr, &iwrn);
*toterr +=ierr;
*totwrn +=iwrn;
return;
}
/* print the extname, exttype, extver, errnum and wrnno in a table */
void hdus_summary(FILE *out)
{
HduName *p;
int i;
int ierr, iwrn;
char temp[FLEN_VALUE];
char temp1[FLEN_VALUE];
wrtsep(out,'+'," Error Summary ",60);
wrtout(out," ");
sprintf(comm," HDU# Name (version) Type Warnings Errors");
wrtout(out,comm);
sprintf(comm," 1 Primary Array %-4d %-4d ",
hduname[0]->wrnno,hduname[0]->errnum);
wrtout(out,comm);
for (i=2; i <= totalhdu; i++) {
p = hduname[i-1];
strcpy(temp,p->extname);
if(p->extver && p->extver!= -999) {
sprintf(temp1," (%-d)",p->extver);
strcat(temp,temp1);
}
switch(hduname[i-1]->hdutype){
case IMAGE_HDU:
sprintf(comm," %-5d %-20s Image Array %-4d %-4d ",
i,temp, p->wrnno,p->errnum);
wrtout(out,comm);
break;
case ASCII_TBL:
sprintf(comm," %-5d %-20s ASCII Table %-4d %-4d ",
i,temp, p->wrnno,p->errnum);
wrtout(out,comm);
break;
case BINARY_TBL:
sprintf(comm," %-5d %-20s Binary Table %-4d %-4d ",
i,temp, p->wrnno,p->errnum);
wrtout(out,comm);
break;
default:
sprintf(comm," %-5d %-20s Unknown HDU %-4d %-4d ",
i,temp, p->wrnno,p->errnum);
wrtout(out,comm);
break;
}
}
/* check the end of file */
num_err_wrn(&ierr, &iwrn);
if (iwrn || ierr) {
sprintf(comm," End-of-file %-30s %-4d %-4d ", "", iwrn,ierr);
wrtout(out,comm);
}
wrtout(out," ");
return;
}
void destroy_hduname()
{
int i;
for (i=0; i < totalhdu; i++) free(hduname[i]);
free(hduname);
return;
}
/* Routine to test the extra bytes at the end of file */
void test_end(fitsfile *infits,
FILE *out)
{
int status = 0;
LONGLONG headstart, datastart, dataend;
int hdutype;
/* check whether there are any HDU left */
fits_movrel_hdu(infits,1, &hdutype, &status);
if (!status) {
wrtout(out,"< End-of-File >");
sprintf(errmes,
"There are extraneous HDU(s) beyond the end of last HDU.");
wrterr(out,errmes,2);
wrtout(out," ");
return;
}
if (status != END_OF_FILE) {
wrtserr(out,"Bad HDU? ",&status,2);
return;
}
status = 0;
fits_clear_errmsg();
if(ffghadll(infits, &headstart, &datastart, &dataend, &status))
wrtferr(out, "",&status,1);
/* try to move to the last byte of this extension. */
if (ffmbyt(infits, dataend - 1,0,&status))
{
sprintf(errmes,
"Error trying to read last byte of the file at byte %ld.", (long) dataend);
wrterr(out,errmes,2);
wrtout(out,"< End-of-File >");
wrtout(out," ");
return;
}
/* try to move to what would be the first byte of the next extension.
If successfull, we have a problem... */
ffmbyt(infits, dataend,0,&status);
if(status == 0) {
wrtout(out,"< End-of-File >");
sprintf(errmes,
"File has extra byte(s) after last HDU at byte %ld.", (long) dataend);
wrterr(out,errmes,2);
wrtout(out," ");
}
return;
}
/******************************************************************************
* Function
* init_report
*
*
* DESCRIPTION:
* Initialize the fverify report
*
*******************************************************************************/
void init_report(FILE *out, /* output file */
char *rootnam /* input file name */
)
{
sprintf(comm,"\n%d Header-Data Units in this file.",totalhdu);
wrtout(out,comm);
wrtout(out," ");
reset_err_wrn();
init_hduname();
}
/******************************************************************************
* Function
* close_report
*
*
* DESCRIPTION:
* Close the fverify report
*
*******************************************************************************/
void close_report(FILE *out /* output file */ )
{
int numerrs = 0; /* number of the errors */
int numwrns = 0; /* number of the warnings */
/* print out a summary of all the hdus */
if(prstat)hdus_summary(out);
total_errors (&numerrs, &numwrns);
total_warn = numwrns;
total_err = numerrs;
/* get the total number of errors and warnnings */
sprintf(comm,"**** Verification found %d warning(s) and %d error(s). ****",
numwrns, numerrs);
wrtout(out,comm);
update_parfile(numerrs,numwrns);
/* destroy the hdu name */
destroy_hduname();
return ;
}
This diff is collapsed.
This diff is collapsed.
/******************************************************************************
* Function
* wrtout: print messages in the streams of stdout and out.
* wrterr: print erro messages in the streams of stderr and out.
* wrtferr: print cfitsio erro messages in the streams of stderr and out.
* wrtwrn: print warning messages in the streams of stdout and out.
* wrtsep: print seperators.
* num_err_wrn: Return the number of errors and warnings.
*
*******************************************************************************/
#include "fverify.h"
static int nwrns = 0;
static int nerrs = 0;
static char temp[512];
void num_err_wrn(int *num_err, int *num_wrn)
{
*num_wrn = nwrns;
*num_err = nerrs;
return;
}
void reset_err_wrn()
{
nwrns = 0;
nerrs = 0;
return;
}
int wrtout(FILE *out, char *mess)
{
if(out != NULL )fprintf(out,"%s\n",mess);
if(out == stdout) fflush(stdout);
return 0;
}
int wrtwrn(FILE *out, char *mess, int isheasarc)
{
if(err_report) return 0; /* Don't print the warnings */
if(!heasarc_conv && isheasarc) return 0; /* heasarc warnings but with
heasarc convention turns off */
nwrns++;
strcpy(temp,"*** Warning: ");
strcat(temp,mess);
if(isheasarc) strcat(temp," (HEASARC Convention)");
print_fmt(out,temp,13);
/* if(nwrns > MAXWRNS ) {
fprintf(stderr,"??? Too many Warnings! I give up...\n");
} */
return nwrns;
}
int wrterr(FILE *out, char *mess, int severity )
{
if(severity < err_report) {
fits_clear_errmsg();
return 0;
}
nerrs++;
strcpy(temp,"*** Error: ");
strcat(temp,mess);
if(out != NULL) {
if ((out!=stdout) && (out!=stderr)) print_fmt(out,temp,13);
/*
if ERR2OUT is defined, then error messages will be sent to the
stdout stream rather than to stderr
*/
#ifdef ERR2OUT
print_fmt(stdout,temp,13);
#else
print_fmt(stderr,temp,13);
#endif
}
if(nerrs > MAXERRORS ) {
#ifdef ERR2OUT
fprintf(stdout,"??? Too many Errors! I give up...\n");
#else
fprintf(stderr,"??? Too many Errors! I give up...\n");
#endif
close_report(out);
exit(1);
}
fits_clear_errmsg();
return nerrs;
}
int wrtferr(FILE *out, char* mess, int *status, int severity)
/* construct an error message: mess + cfitsio error */
{
char ttemp[255];
if(severity < err_report) {
fits_clear_errmsg();
return 0;
}
nerrs++;
strcpy(temp,"*** Error: ");
strcat(temp,mess);
fits_get_errstatus(*status