Loading INSTALL_DBdeleted 100755 → 0 +0 −29 Original line number Diff line number Diff line #!/bin/bash # INPUT: path_to_db # Check if the user provided the input if [ -z "$1" ]; then echo "Provide a path to or a name for the database, followed by [ENTER]:" read path else path=$1 fi # Check if the path is valid and unexisting if [ ! -d $path ]; then git clone git@bitbucket.org:genomicepidemiology/resfinder_db.git $path else echo "Error: path exists! Please provide a valid and unused path..." fi echo "Do you want to fix permissions, yes/y or no/n, followed by [ENTER]:" read fix if [[ $fix == "yes" ]] || [[ $fix == "y" ]]; then echo "enter group name followed by [ENTER]:" read gn if [ ! -z "$gn" ]; then chgrp -R $gn $path fi find $path \( -name .git \) -prune -printf '' -o -type d -print -exec chmod 775 {} \; find $path \( -name .git \) -prune -printf '' -o -type f -print -exec chmod 664 {} \; fi Makefiledeleted 100755 → 0 +0 −9 Original line number Diff line number Diff line install: cpanm CJFIELDS/BioPerl-1.6.924.tar.gz --force cpanm Data::Dumper cpanm Getopt::Long cpanm File::Temp clean: find ./ -name "*.DS_Store" -delete find ./ -name "*.log" -delete find ./ -name "*.gz" -delete README.md +73 −126 Original line number Diff line number Diff line =================== ResFinder =================== This project documents ResFinder service Documentation ResFinder documentation ============= ## What is it? The ResFinder service contains one perl script *resfinder.pl* which is the script of the latest version of the ResFinder service. ResFinder identifies acquired antimicrobial resistance genes in total or partial sequenced isolates of bacteria. This repository also contains a python script *resfinder.py* which is a new version of ResFinder, but not yet running on the CGE server. This program was added because it uses a newer version of blastn, which, in contrary from the blastall version that the perl script uses, is avail to download. ## Content of the repository 1. resfinder.pl - the program 2. INSTALL_DB - shell script for downloading the ResFinder database 3. UPDATE_DB - shell script for updating the database to the newest version 4. VALIDATE_DB - python script for verifying the database contains all required files 5. brew.sh - shell script for installing dependencies 6. makefile - make script for installing dependencies 7. test.fsa - test fasta file 2. resfinder.py - (same program using an available blastn version - blastn-2.2.26+) 3. test.fsa - test fasta file ## Installation Setting up ResFinder Setting up ResFinder script and database ```bash # Go to wanted location for resfinder cd /path/to/some/dir # Clone and enter the resfinder directory git clone https://bitbucket.org/genomicepidemiology/resfinder.git git clone https://git@bitbucket.org/genomicepidemiology/resfinder.git cd resfinder ``` Installing up the ResFinder database ```bash cd /path/to/resfinder ./INSTALL_DB database # Check all DB scripts works, and validate the database is correct ./UPDATE_DB database ./VALIDATE_DB database ``` # Installing up the ResFinder database # Go to wanted location for resfinder database cd /path/to/some/dir # Clone and enter the resfinder directory git clone https://git@bitbucket.org/genomicepidemiology/resfinder_db.git cd resfinder_db Installing dependencies: Perlbrew is used to manage isolated perl environments. To install it run: ```bash bash brew.sh ``` This will installed Perl 5.23 in the Home folder, along with CPAN minus as package manager. Blast will also be installed when running brew.sh if BlastAll and FormatDB are not already installed and place in the user's path. After running brew.sh and installing Blast add this command to the end of your ~/bash_profile to add BlastAll and FormatDB to the user's path ### Installing dependencies (for python script): ```bash export PATH=$PATH:blast-2.2.26/bin ``` The BlastAll and FormatDB that the perl script uses are no longer available for downloading through ncbi. Therefor we have provided the resfinder.py scriot that uses Blastn instead. Note, this is not not script that is running on the CGE server. The CGE server is running the perl script using BlastAll If you want to download the two external tools from the Blast package, BlastAll and FormatDB, yourself go to ```url ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST ``` and download the version for your OS with the format: #### Download Blastn and BioPython ```url blast-version-architecture-OS.tar.gz http://biopython.org/DIST/docs/install/Installation.html ``` after unzipping the file, add this command to the end of your ~/bash_profile. ```bash export PATH=$PATH:/path/to/blast-folder/bin ```url ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ ``` where path/to/blast-folder is the folder you unzipped. At last ResFinder has several Perl dependencies. To install them (this requires CPAN minus as package manager): #### Install the cgecore module to python3 ```bash make install ``` The scripts are self contained. You just have to copy them to where they should be used. Remember to add the program to your system path if you want to be able to invoke the program without calling the full path. If you don't do that you have to write the full path to the program when using it. ## Test the scripts and database ```bash cd /path/to/test_dir cp /path/to/resfinder/test.fsa . perl /path/to/resfinder/resfinder.pl -d /path/to/resfinder/database \ -b /path/to/blast/parent/dir -i test.fsa -a aminoglycoside -k 90.00 -l 0.60 pip3 install cgecore ``` ## Usage The program can be invoked with the -h option to get help and more information of the service. You can run resfinder command line using python3 ```bash Usage: perl resfinder.pl [options] Options: -h HELP Prints a message with options and information to the screen -d DATABASE The path to where you have located the database folder -b BLAST The path to the location of blast-2.2.26 if it is not added to the users path (see the install guide in 'README.md') -i INFILE Your input file which needs to be preassembled partial or complete genomes in fasta format -o OUTFOLDER The folder you want to have your output files places. If not specified the program will create a folder named 'Output' in which the result files will be stored -a ANTIMICROBIAL Antimicrobial configuration. The options can be found in the file 'ResFinder_Antimicrobial' -k THRESHOLD The threshold for % identity for example '95.00' for 95 % -l MIN_LENGHT The minimum length of the overlap ex 0.60 for an overlap of minimum 60 % ``` #### Example of use with the *database* folder is loacted in the current #### directory and Blast added to the user's path ```perl perl resfinder.pl -i test.fsa -o OUTFOLDER -a aminoglycoside -k 90.00 \ -l 0.60 ``` #### Example of use with the *database* and *blast-2.2.26* folders loacted in #### other directories ```perl perl resfinder.pl -d path/to/database -b path/to/blast-2.2.26 -i \ test.fsa -o OUTFOLDER -a aminoglycoside -k 90.00 -l 0.60 # Example of running resfinder python3 resfinder.py -i test.fsa -o . -p /path/to/resfinder_db \ -b /path/to/blastn -d aminoglycoside -k 90.00 -l 0.60 # The program can be invoked with the -h option Usage: resfinder.py [-h] [-i INPUTFILE] [-1 FASTQ1] [-2 FASTQ2] [-o OUT_PATH] [-b BLAST_PATH] [-p DB_PATH] [-k KMA_PATH] [-q DB_PATH_KMA] [-d DATABASES] [-l MIN_COV] [-t THRESHOLD] optional arguments: -h, --help show this help message and exit -i INPUTFILE, --inputfile INPUTFILE Input file -1 FASTQ1, --fastq1 FASTQ1 Raw read data file 1. -2 FASTQ2, --fastq2 FASTQ2 Raw read data file 2 (only required if data is paired- end). -o OUT_PATH, --outputPath OUT_PATH Path to blast output -b BLAST_PATH, --blastPath BLAST_PATH Path to blast -p DB_PATH, --databasePath DB_PATH Path to the databases -k KMA_PATH, --kmaPath KMA_PATH Path to KMA -q DB_PATH_KMA, --databasePathKMA DB_PATH_KMA Path to the directories containing the KMA indexed databases. Defaults to the directory 'kma_indexing' inside the databasePath directory. -d DATABASES, --databases DATABASES Databases chosen to search in - if none are specified all are used -l MIN_COV, --min_cov MIN_COV Minimum coverage default 0.6 -t THRESHOLD, --threshold THRESHOLD Blast threshold for identity default minimum 0.9 ``` ## Web-server ### Web-server A webserver implementing the methods is available at the [CGE website](http://www.genomicepidemiology.org/) and can be found here: https://cge.cbs.dtu.dk/services/ResFinder/ ## The Latest Version The latest version can be found at https://bitbucket.org/genomicepidemiology/resfinder/overview ## Documentation ### Documentation The documentation available as of the date of this release can be found at https://bitbucket.org/genomicepidemiology/resfinder/overview. Loading UPDATE_DBdeleted 100755 → 0 +0 −33 Original line number Diff line number Diff line #!/bin/bash # INPUT: path_to_db # Check if the user provided the input if [ -z "$1" ]; then cwd=$(pwd) echo "Current path: "$cwd echo "Type the path to where the database is located, followed by [ENTER]:" read path else path=$1 fi # Check if the path is valid and existing if [ -d $path ]; then cd $path #git stash git pull else echo "Error: path does not exist! Please provide a valid path..." fi echo "Do you want to fix permissions, yes/y or no/n, followed by [ENTER]:" read fix if [[ $fix == "yes" ]] || [[ $fix == "y" ]]; then echo "enter group name followed by [ENTER]:" read gn if [ ! -z "$gn" ]; then chgrp -R $gn $path fi find $path \( -name .git \) -prune -printf '' -o -type d -print -exec chmod 775 {} \; find $path \( -name .git \) -prune -printf '' -o -type f -print -exec chmod 664 {} \; fi VALIDATE_DBdeleted 100755 → 0 +0 −55 Original line number Diff line number Diff line #!/usr/bin/env python ''' Validate Database ''' import sys, os if len(sys.argv) > 1: db_path = sys.argv[1] else: print('Current path: %s'%(os.getcwd())) db_path = input('Please provide the path to the database:') # VALIDATE REQUIRED ARGUMENTS if not os.path.exists(db_path): sys.exit("Error: The specified database directory does not exist!\n") else: # Check existence of config file db_config_file = '%s/config'%(db_path) if not os.path.exists(db_config_file): sys.exit("Error: The database config file could not be found!") dbs = [] extensions = [] with open(db_config_file) as f: for l in f: l = l.strip() if l == '': continue if l[0] == '#': if 'important files are:' in l: files = ["%s/%s"%(db_path, s.strip()) for s in l.split('are:')[-1].split(',')] # Check all files exist for path in files: if not os.path.exists(path): sys.exit('Error: %s not found!'%(path)) if 'extensions:' in l: extensions = [s.strip() for s in l.split('extensions:')[-1].split(',')] continue tmp = l.split('\t') if len(tmp) != 3: sys.exit(("Error: Invalid line in the database config file!\n" "A proper entry requires 3 tab separated columns!\n%s")%(l)) db_prefix = tmp[0].strip() name = tmp[1].split('#')[0] description = tmp[2] # Check if all db files are present for ext in extensions: path = "%s/%s.%s"%(db_path, db_prefix, ext) if not os.path.exists(path): sys.exit(("Error: The database file (%s) could not be found!")%( path)) dbs.append((name, db_prefix)) if len(dbs) == 0: sys.exit("Error: No databases were found in the database config file!") else: print("Validation passed. Database is valid.") No newline at end of file Loading
INSTALL_DBdeleted 100755 → 0 +0 −29 Original line number Diff line number Diff line #!/bin/bash # INPUT: path_to_db # Check if the user provided the input if [ -z "$1" ]; then echo "Provide a path to or a name for the database, followed by [ENTER]:" read path else path=$1 fi # Check if the path is valid and unexisting if [ ! -d $path ]; then git clone git@bitbucket.org:genomicepidemiology/resfinder_db.git $path else echo "Error: path exists! Please provide a valid and unused path..." fi echo "Do you want to fix permissions, yes/y or no/n, followed by [ENTER]:" read fix if [[ $fix == "yes" ]] || [[ $fix == "y" ]]; then echo "enter group name followed by [ENTER]:" read gn if [ ! -z "$gn" ]; then chgrp -R $gn $path fi find $path \( -name .git \) -prune -printf '' -o -type d -print -exec chmod 775 {} \; find $path \( -name .git \) -prune -printf '' -o -type f -print -exec chmod 664 {} \; fi
Makefiledeleted 100755 → 0 +0 −9 Original line number Diff line number Diff line install: cpanm CJFIELDS/BioPerl-1.6.924.tar.gz --force cpanm Data::Dumper cpanm Getopt::Long cpanm File::Temp clean: find ./ -name "*.DS_Store" -delete find ./ -name "*.log" -delete find ./ -name "*.gz" -delete
README.md +73 −126 Original line number Diff line number Diff line =================== ResFinder =================== This project documents ResFinder service Documentation ResFinder documentation ============= ## What is it? The ResFinder service contains one perl script *resfinder.pl* which is the script of the latest version of the ResFinder service. ResFinder identifies acquired antimicrobial resistance genes in total or partial sequenced isolates of bacteria. This repository also contains a python script *resfinder.py* which is a new version of ResFinder, but not yet running on the CGE server. This program was added because it uses a newer version of blastn, which, in contrary from the blastall version that the perl script uses, is avail to download. ## Content of the repository 1. resfinder.pl - the program 2. INSTALL_DB - shell script for downloading the ResFinder database 3. UPDATE_DB - shell script for updating the database to the newest version 4. VALIDATE_DB - python script for verifying the database contains all required files 5. brew.sh - shell script for installing dependencies 6. makefile - make script for installing dependencies 7. test.fsa - test fasta file 2. resfinder.py - (same program using an available blastn version - blastn-2.2.26+) 3. test.fsa - test fasta file ## Installation Setting up ResFinder Setting up ResFinder script and database ```bash # Go to wanted location for resfinder cd /path/to/some/dir # Clone and enter the resfinder directory git clone https://bitbucket.org/genomicepidemiology/resfinder.git git clone https://git@bitbucket.org/genomicepidemiology/resfinder.git cd resfinder ``` Installing up the ResFinder database ```bash cd /path/to/resfinder ./INSTALL_DB database # Check all DB scripts works, and validate the database is correct ./UPDATE_DB database ./VALIDATE_DB database ``` # Installing up the ResFinder database # Go to wanted location for resfinder database cd /path/to/some/dir # Clone and enter the resfinder directory git clone https://git@bitbucket.org/genomicepidemiology/resfinder_db.git cd resfinder_db Installing dependencies: Perlbrew is used to manage isolated perl environments. To install it run: ```bash bash brew.sh ``` This will installed Perl 5.23 in the Home folder, along with CPAN minus as package manager. Blast will also be installed when running brew.sh if BlastAll and FormatDB are not already installed and place in the user's path. After running brew.sh and installing Blast add this command to the end of your ~/bash_profile to add BlastAll and FormatDB to the user's path ### Installing dependencies (for python script): ```bash export PATH=$PATH:blast-2.2.26/bin ``` The BlastAll and FormatDB that the perl script uses are no longer available for downloading through ncbi. Therefor we have provided the resfinder.py scriot that uses Blastn instead. Note, this is not not script that is running on the CGE server. The CGE server is running the perl script using BlastAll If you want to download the two external tools from the Blast package, BlastAll and FormatDB, yourself go to ```url ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST ``` and download the version for your OS with the format: #### Download Blastn and BioPython ```url blast-version-architecture-OS.tar.gz http://biopython.org/DIST/docs/install/Installation.html ``` after unzipping the file, add this command to the end of your ~/bash_profile. ```bash export PATH=$PATH:/path/to/blast-folder/bin ```url ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ ``` where path/to/blast-folder is the folder you unzipped. At last ResFinder has several Perl dependencies. To install them (this requires CPAN minus as package manager): #### Install the cgecore module to python3 ```bash make install ``` The scripts are self contained. You just have to copy them to where they should be used. Remember to add the program to your system path if you want to be able to invoke the program without calling the full path. If you don't do that you have to write the full path to the program when using it. ## Test the scripts and database ```bash cd /path/to/test_dir cp /path/to/resfinder/test.fsa . perl /path/to/resfinder/resfinder.pl -d /path/to/resfinder/database \ -b /path/to/blast/parent/dir -i test.fsa -a aminoglycoside -k 90.00 -l 0.60 pip3 install cgecore ``` ## Usage The program can be invoked with the -h option to get help and more information of the service. You can run resfinder command line using python3 ```bash Usage: perl resfinder.pl [options] Options: -h HELP Prints a message with options and information to the screen -d DATABASE The path to where you have located the database folder -b BLAST The path to the location of blast-2.2.26 if it is not added to the users path (see the install guide in 'README.md') -i INFILE Your input file which needs to be preassembled partial or complete genomes in fasta format -o OUTFOLDER The folder you want to have your output files places. If not specified the program will create a folder named 'Output' in which the result files will be stored -a ANTIMICROBIAL Antimicrobial configuration. The options can be found in the file 'ResFinder_Antimicrobial' -k THRESHOLD The threshold for % identity for example '95.00' for 95 % -l MIN_LENGHT The minimum length of the overlap ex 0.60 for an overlap of minimum 60 % ``` #### Example of use with the *database* folder is loacted in the current #### directory and Blast added to the user's path ```perl perl resfinder.pl -i test.fsa -o OUTFOLDER -a aminoglycoside -k 90.00 \ -l 0.60 ``` #### Example of use with the *database* and *blast-2.2.26* folders loacted in #### other directories ```perl perl resfinder.pl -d path/to/database -b path/to/blast-2.2.26 -i \ test.fsa -o OUTFOLDER -a aminoglycoside -k 90.00 -l 0.60 # Example of running resfinder python3 resfinder.py -i test.fsa -o . -p /path/to/resfinder_db \ -b /path/to/blastn -d aminoglycoside -k 90.00 -l 0.60 # The program can be invoked with the -h option Usage: resfinder.py [-h] [-i INPUTFILE] [-1 FASTQ1] [-2 FASTQ2] [-o OUT_PATH] [-b BLAST_PATH] [-p DB_PATH] [-k KMA_PATH] [-q DB_PATH_KMA] [-d DATABASES] [-l MIN_COV] [-t THRESHOLD] optional arguments: -h, --help show this help message and exit -i INPUTFILE, --inputfile INPUTFILE Input file -1 FASTQ1, --fastq1 FASTQ1 Raw read data file 1. -2 FASTQ2, --fastq2 FASTQ2 Raw read data file 2 (only required if data is paired- end). -o OUT_PATH, --outputPath OUT_PATH Path to blast output -b BLAST_PATH, --blastPath BLAST_PATH Path to blast -p DB_PATH, --databasePath DB_PATH Path to the databases -k KMA_PATH, --kmaPath KMA_PATH Path to KMA -q DB_PATH_KMA, --databasePathKMA DB_PATH_KMA Path to the directories containing the KMA indexed databases. Defaults to the directory 'kma_indexing' inside the databasePath directory. -d DATABASES, --databases DATABASES Databases chosen to search in - if none are specified all are used -l MIN_COV, --min_cov MIN_COV Minimum coverage default 0.6 -t THRESHOLD, --threshold THRESHOLD Blast threshold for identity default minimum 0.9 ``` ## Web-server ### Web-server A webserver implementing the methods is available at the [CGE website](http://www.genomicepidemiology.org/) and can be found here: https://cge.cbs.dtu.dk/services/ResFinder/ ## The Latest Version The latest version can be found at https://bitbucket.org/genomicepidemiology/resfinder/overview ## Documentation ### Documentation The documentation available as of the date of this release can be found at https://bitbucket.org/genomicepidemiology/resfinder/overview. Loading
UPDATE_DBdeleted 100755 → 0 +0 −33 Original line number Diff line number Diff line #!/bin/bash # INPUT: path_to_db # Check if the user provided the input if [ -z "$1" ]; then cwd=$(pwd) echo "Current path: "$cwd echo "Type the path to where the database is located, followed by [ENTER]:" read path else path=$1 fi # Check if the path is valid and existing if [ -d $path ]; then cd $path #git stash git pull else echo "Error: path does not exist! Please provide a valid path..." fi echo "Do you want to fix permissions, yes/y or no/n, followed by [ENTER]:" read fix if [[ $fix == "yes" ]] || [[ $fix == "y" ]]; then echo "enter group name followed by [ENTER]:" read gn if [ ! -z "$gn" ]; then chgrp -R $gn $path fi find $path \( -name .git \) -prune -printf '' -o -type d -print -exec chmod 775 {} \; find $path \( -name .git \) -prune -printf '' -o -type f -print -exec chmod 664 {} \; fi
VALIDATE_DBdeleted 100755 → 0 +0 −55 Original line number Diff line number Diff line #!/usr/bin/env python ''' Validate Database ''' import sys, os if len(sys.argv) > 1: db_path = sys.argv[1] else: print('Current path: %s'%(os.getcwd())) db_path = input('Please provide the path to the database:') # VALIDATE REQUIRED ARGUMENTS if not os.path.exists(db_path): sys.exit("Error: The specified database directory does not exist!\n") else: # Check existence of config file db_config_file = '%s/config'%(db_path) if not os.path.exists(db_config_file): sys.exit("Error: The database config file could not be found!") dbs = [] extensions = [] with open(db_config_file) as f: for l in f: l = l.strip() if l == '': continue if l[0] == '#': if 'important files are:' in l: files = ["%s/%s"%(db_path, s.strip()) for s in l.split('are:')[-1].split(',')] # Check all files exist for path in files: if not os.path.exists(path): sys.exit('Error: %s not found!'%(path)) if 'extensions:' in l: extensions = [s.strip() for s in l.split('extensions:')[-1].split(',')] continue tmp = l.split('\t') if len(tmp) != 3: sys.exit(("Error: Invalid line in the database config file!\n" "A proper entry requires 3 tab separated columns!\n%s")%(l)) db_prefix = tmp[0].strip() name = tmp[1].split('#')[0] description = tmp[2] # Check if all db files are present for ext in extensions: path = "%s/%s.%s"%(db_path, db_prefix, ext) if not os.path.exists(path): sys.exit(("Error: The database file (%s) could not be found!")%( path)) dbs.append((name, db_prefix)) if len(dbs) == 0: sys.exit("Error: No databases were found in the database config file!") else: print("Validation passed. Database is valid.") No newline at end of file
