Main Page: Difference between revisions

From 太極
Jump to navigation Jump to search
No edit summary
 
(227 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Wiki for [http://linus.nci.nih.gov/BRB-ArrayTools.html BRB-ArrayTools] .'''
Shortcuts to some pages that may be useful to you!


== Features ==
{|
|style="width: 5%; border-width: 0px;"|[[File:Dna microarray45.png|link=Arraytools|BRB-ArrayTools]]||style="width: 20%; border-width: 0px;"|[[Arraytools|BRB-ArrayTools]]||style="width: 5%; border-width: 0px;"|[[File:linux48.png|link=Linux|Linux]]||style="width: 20%; border-width: 0px;"|[[Linux|Linux]]||style="width: 5%; border-width: 0px;"|[[File:R_logo48.png|link=R|R]]||style="width: 20%; border-width: 0px;"|[[R|R]]
|-
|| [[File:tai-chi48.png|link=Tai_Chi|Tai Chi]] || [[Tai_Chi|Tai Chi]] || [[File:heart48.png|link=Health|Health]] || [[Health|Health]] || [[File:raspberry48.png|link=Raspberry|Raspberry Pi]]  || [[Raspberry|Raspberry Pi]]
|-
|| [[File:docker48.png|link=Docker|Docker]]
|| [[Docker|Docker]]
|| [[File:recipe48.png|link=Recipes|Recipes]]
|| [[Recipes|Recipes]]
|| [[File:statistics48.png|link=Statistics|Statistics]]
|| [[Statistics|Statistics]]
|-
|| [[File:ubuntulogo48.png|link=Ubuntu|Ubuntu]]
|| [[Ubuntu|Ubuntu]]
|| [[File:vblogo48.png|link=Virtualbox|Virtualbox]]
|| [[Virtualbox|Virtualbox]]
|| [[File:beaglebone48.png|link=Beaglebone|Beaglebone]]
|| [[Beaglebone|Beaglebone]]
|}


=== Import from multiple data types ===
Expression data, Illumina methylation data, Copy number data (CGH-Tools), RNA-Seq count data processed through Galaxy web tool.
=== Sophisticated statistical analysis tools ===
Class comparison for differential expression, class prediction, graphical 2d and 3D interactive plots, gene set analysis, and more.


=== Comprehensive biological annotations ===
MC Li
Gene ontology, pathways, protein domain, broad msigdb, lymphoid signatures, experimentally verified transcription factor targets, computationally predicted microRNA targets.


== Screenshots ==
-----
 
==== [[File:ArrayTools GUI.png|100px]] BRB-ArrayTools graphical user interface ====
 
==== [[File:BRB_HeatmapDendrogram.png|100px]] Heatmap and dendrogram generated from the Pomeroy sample dataset. ====
 
==== [[File:BRB_3Dplot.png|100px]] Interactive MDS plot of samples from running the multidimensional scaling analysis on the Pomeroy dataset. ====
 
==== [[File:BRB_3Dplot2.png|100px]] Interactive 3D scatterplot of genes on the Pomeroy dataset.  ====
X-axis is from array 'Brain_MD_1', y-axis is 'Brain_MD_2' and z-axis is 'Brian_MD_3'.
 
==== [[File:BRB_Scatterplot.png|100px]] Interactive 2D scatterplot of samples with gene annotation from a selected gene using right click menu. ====
The right click menu gives an option to highlight up/down-regulated genes, export gene list, copy plot to clipboard, highlight genes in gene set, link genes among plots and change properties of the plot like title, point size, color of points, fold change threshold for up/down regulated genes.
 
==== [[File:VolcanoPlot.png|100px]] Interactive volcano plot from the output of running a class comparison tool.  ====
When you move mouse over a gene (point), the gene unique ID and/or symbol will be popped up.
 
==== [[File:QHeatmapScreenShot.png|100px]] Interactive zoomable heatmap. ====
The program is multi-platform. The screenshot was taken from running in Ubuntu OS.
 
==== [[File:BRB_ClassComp.png|100px]] HTML output of running the class comparison analysis.  ====
 
==== [[File:BRB_sam.png|100px]] HTML output containing SAM plot from running the significance of microarray analysis.  ====
 
==== [[File:BRB_Prediction.png|100px]] HTML output of running the class prediction analysis.  ====
 
==== [[File:BRB_SRP.png|100px]] HTML output of running the survival risk analysis.  ====
 
==== [[File:BRB_samplesize.png|100px]] Result of sample size analysis.  ====
 
== FAQs  ==
 
=== General ===
 
==== How to install BRB-ArrayTools if you have 64-bit MS-Office? ====
There is no difference in terms of the installation.
 
==== After installation, I did not find the BRB-ArrayTools in Windows > Start > All Programs. ====
Check EXCEL. ArrayTools and CGHTools are under the Excel menu of Addon.
 
==== Can I upgrade R or install multiple versions of R? ====
Better not for upgrading. Installing multiple versions of R is OK provided you know some details described below.
 
Each version of BRB-ArrayTools has been tested with a certain version of R. So there maybe a compatibility problem with certain functions used in the code if you decide to upgrade R.
 
* BRB-ArrayTools (up to v4.2.1) requires StatconnDCOM which means the following conditions have to be satisfied:
** R needs to registered in the Windows's registry (it should be done if you accept all default options when R was installed).
** The R package 'rscproxy' has to be installed under the library folder the registered R. It cannot be installed under user's Document's folder as other R's packages.
 
* BRB-ArrayTools (from v4.3.0) requires Rserve package. That means
** Rserve has to be installed. It does not have to be installed under R\library folder.
** Rserve.exe from R\librar\Rserve\inst\i386 and x64 subfolder has to be copied to R\bin\i386 and R\bin\x64 folder.
 
If you need to use the latest version of R for your own analysis, you can still make it. First, install the latest version of R as usual. Then install again the full-version of BRB-ArrayTools. This will possibly install another version of R and register it in the Windows's registry for BRB-ArrayTools to use. Now you can enjoy both versions of R as you want. The idea is when you install R, it will by default register R, but this behavior can break the setting for R to be used by BRB-ArrayTools. Once you install BRB-ArrayTools, it will install an R it needs and not erase any other versions of R you already have.
 
==== My institute is using a proxy server. So how do I do with BRB-ArrayTools ====
See the [http://65.123.194.80/phpBB3/viewtopic.php?f=3&t=580 message] on BRB-ArrayTools message board. Essentially we need to create a Windows environment variable '''http_proxy''' with a value like
<pre>
http://myproxy:myport
OR
http_proxy=http://username:[email protected]
OR
http_proxy=http://username:[email protected]:81
</pre>
 
==== RExcel-statconnDCOM gave an error ====
 
Please upgrade BRB-ArrayTools to v4.3.x where Rserve has replaced statconnDCOM.
 
==== Rserve ====
 
Since version 4.3.0, BRB-ArrayTools started to use Rserve as a media for the communication between R and Excel. When Rserve is required, an R window will be pop up. This R window has a blue icon on the Windows' taskbar. If you accidentally close it, it will be automatically popped up when it is needed.
 
See my [[Rserve]] wiki page.
 
=== Importing ===
==== GEO (Gene Expression Omnibus) ====
* Use GEO importer to load the gds soft format file (eg GDS1348)
* If there is no gds number,
# If there is raw data, we can try to import them with data import wizard (eg GSE18170). We can use SOURCE to annotation data using 'Symbol' as the lookup key. Select 'Mouse' as the organism.
# If there is no raw data, we can use Serial matrix file with general importer (eg GSE33403). Download GPL10481_family.soft.gz file and Extract it. Open the file in EXCEL. Copy the cells on rows from126 to 45346 and columns from A to I. Paste them to a new Workbook and save it in a tab-delimited format file called <SeriesMatrix.txt>. Download GSE33403_series_matrix.txt.gz file and extract it. Open it in EXCEL. Copy the data on rows from 75 to  45295 and columns from A to AN. Paste them to a new Workbook and save it in a tab-delimited format file called <GeneID.txt>.
 
==== Multiple chips (hgu133a and hgu133b) ====
GSE4922 contains two different array platforms (hgu133a and hgu133b). Starting from ArrayTools v4.1 our software does not support the importing of multi-chips any more. Therefore you cannot directly import all the data into BRB-ArrayTools to create one project with two different chips. However, you could import the data into ArrayTools as two different projects and output the normalized data for each project. Then you can manually combine the two output data files and re-import into BRB-ArrayTools as one project. Here is what you can do,
# Use Data Import Wizard to import the CEL files for hgu133a and hgu133b chips separately to create two projects.
# Open each project and then use the "Export 1-color data to R" plug-in (click on "ArrayTools -> Plugins -> Export 1 color data to R") to output your normalized data file along with the GeneId file.
# Under Excel, combine the normalized output files ("NORMALIZEDLOGINTENSITY.txt") from two projects. Add sample names in the first row and Probe Set Ids in the first column. Save this combined data file. Then combine the two "GENEID.txt" files manually to create one Gene ID file.
# Open Excel, Click on "ArrayTools -> Import data -> General format importer" to import the combined data file. Select your data as "Single-channel", "Affymetrix probeset-summary data". For the chip type, because your data contain probe sets from both hgu133a and hgu133b, almost all of which were included in the hgu133plus2 chip, you can just pick the hgu133plus2 as your chip type. Alternatively, if you do not wish to do this, you can check "I would like to use my own gene identifiers file rather than the one from bioconductor" and use the combined Gene ID file (done in step 3) for annotation.
# At the filter and normalization step, it is VERY IMPORTANT that you need to uncheck all the spot filter and normalization options, because your "raw" data file comes from already normalized data in two different projects. You do not want to re-run normalization. You can keep the options in "Gene filter" tab.
# At the end you can run annotation as your choice.
 
==== PrimeView importing failed on v4.3.0 beta1 (R 2.14.x) ====
# Go to http://www.bioconductor.org/packages/2.11/data/annotation/html/primeviewcdf.html and click on the link for the windows binary package "primeviewcdf_2.11.0.zip" and download the zipped package.
# Unzip the downloaded package and put the entire package folder "primeviewcdf" under your R2.14.2 library folder (default C:\Program Files\R\R-2.14.2\library).
 
However, you cannot run Affymetrix annotation using bioconductor annotation packages at the end of importing, because the annotation package for primeview array is not available at bioconductor. Another commonly use annotation database, SOURCE, has also been down these days. Instead you need to import your own annotation into BRB-ArrayTools. Here is what you can do,
# Download the primeview annotation file in CSV format from Affymetrix website (https://www.affymetrix.com/user/login.jsp?toURL=/support/file_download.affx?onloadforward=/analysis/downloads/na32/ivt/PrimeView.na32.annot.csv.zip).
# Open the CSV format file in Excel and then save it as a tab-delimited .txt file.
# Import your data in CEL file format into BRB-ArrayTools through Data Import Wizard. During importing, you need to check the option "Import your own identifiers file to annotate your data" at the "Probe Set Id options" page, and then Click on "OK". At the next dialog form you need to select the radio button "The identifiers are stored in a separate file" and browse for the tab-delimited file you just saved. The head line # for this file is 25. Then you need to select the corresponding gene identifiers column headers. You need to check the box "Annotate the project with these gene ids, instead of using the data from SOURCE database" before proceeding to the next page.
 
==== miRNA 3.0 ====
For Affymetrix miRNA 2.0 array, you can import the data in CEL file format into BRB-ArrayTools through the CEL file importer (from Data Import Wizard). For miRNA 2.0 and 3.0 arrays, an alternative way is to use the Affymetrix expression console software to pre-process/normalize the CEL file data and obtain a tab-delimited probe-set summary data file to be imported into BRB-ArrayTools using the General format importer. The Affymetrix expression console software can be downloaded at the Affymetrix website (http://www.affymetrix.com/estore/browse/level_seven_software_products_only.jsp?navMode=34000&productId=131414&navAction=jump&aId=productsNav#1_1). Unlike what we do for the regular gene expression chips, BRB-ArrayTools currently does not support the generation of annotation information based on the microRNA ID.
 
For now, the CSV format annotation file provided by Affymetrix can be opened in Excel and then saved as a tab-delimited .txt file, which can be used as a "Gene Identifiers" file for BRB-ArrayTools. When you import your data file into BRB, you can choose to "use your own gene ID file for annotation" and select the "Transcript Id (Array Design)" as the microRNA Id column header. Although we do not directly provide any annotation tools for miRNA data, if you run some particular analyses such as class comparison, there will be a hyperlink associated with the microRNA Id in your html output result file. By clicking on this link, you can be directed to the miRBase website with the annotation information associated with this particular microRNA Id. 
 
==== Gene ST 1.1, 2.0 or any Affymetrix .cel files without cdf packages from Bioconductor ====
 
You will need to install Affymetrix Expression Console. Here is the link of Affymetrix Expression Console download website.
http://www.affymetrix.com/browse/level_seven_software_products_only.jsp?productId=131414&categoryId=35623#1_1
 
Here is the link where you could download Affymetrix Expression Console (64bit).
 
http://www.affymetrix.com/products_services/software/download/expression_console/expression_console_download_terms.affx?v=Release1.2.1.64bit
 
Take the mouse Gene 1.1 ST-array as an example on how to use the Expreesion console to convert .cel files into .txt files and then import .txt files into ArrayTools.
 
# Before downloading the Affymetrix expression console, you need to register at Affymetrix website. After downloading and installing the software, open Affymetrix expression console, name the profile and set a library path by clicking "Edit -> Set library path";
# After you download and install the software, please open it and then download both the '''library file''' and '''annotation file''' for mogene 1.1 ST-array in the software(File->Download Library Files and File->Download Annotation Files). 
# After you download both files, you could click on "File" -> "New study" to open the Affymetrix study window. Click on "Add intensity files" to browse for the CEL files of your interest. Click on "Run analysis".
# When the analysis is done, click on "Export" -> "Export Probe Set Results(pivot table) to TXT" to export a tab-delimited file, which is ready to be imported into BRB-ArrayTools.
# Go to the library path folder and open a file called "xxxx.annot.csv" in Excel. Save this file in tab-delimited .txt format as an annotation file for later use.
# You could start to use BRB-ArrayTools General importer to read the summarization .txt file into ArrayTools. By default, the output expression values from Affymetrix Expression Console is already log 2 transformed. Please turn off all spot filters and normalization during importing in ArrayTools because RMA is already done by Affymetrix Expression Console.
# At the Gene ID dialog form, select "Gene Ids are stored in a separate file" and browse for the annotation file you created in step 5. Check the option "Annotate the project with these gene ids, instead of using the data from SOURCE database".
 
=== Annotation ===
==== SOURCE at stanford is down ====
During the period of time when SOURCE website is down, our users will not be able to import annotations from SOURCE database. The bioconductor annotation packages will still work for users having Affymetrix or Illumina array data. For non-Affymetrix and non-Illumina chip users who are creating new projects, here are a couple of alternative solutions,
 
* If the user has an existing annotated project with an identical chip type to the project he/she is creating, he/she can import the annotations from the existing project. This can be done by clicking on 'ArrayTools -> Utilities -> Annotate the data -> Import annotations from an existing project';
 
* If the user has an annotation file available, during the process of importing, the user can choose to import annotations from this file. At the step of selecting Gene Identifiers, the user can browse for the Gene Identifiers file and check the option 'Annotate the project with these gene ids, instead of using the data from SOURCE database';
 
* If the user does not have an annotation file, for most commonly used commercial chips, the annotation files can be downloaded from GEO database at NCBI. For instance, the annotation file for the Agilent-014850 Whole Human Genome Microarray chip can be downloaded at http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL4133 by clicking on the 'Download full table' button. This downloaded annotation file can then be imported into BRB-ArrayTools following step 2).
 
 
==== Gene ST array annotation using aroma.affymetrix ====
The aroma.affymetrix can handle all single-channel Affymetrix chip types (they have a few multi-channel ones not supported), so my guess is that it is already supported.  What's required for nearly all aroma analysis, is to have a CDF annotation file that defines the units and unit groups, e.g. probe sets of transcripts and exons, SNPs and so on.
When Affymetrix is not providing a CDF, the challenge is to either find a custom CDF (e.g. BrainArray and GeneAnnot) or to create a one from the other types of annotation data they or Bioconductor provide, e.g. http://aroma-project.org/howtos.
 
=== Analysis ===
==== Function analysis using [http://david.abcc.ncifcrf.gov DAVID] ====
See a tutorial from http://nihlibrary.ors.nih.gov/bioinfo/Microarray/Problem1.html
 
==== Best parameter or analysis to choose ====
Ask your local help desk.
 
==== For affy cel files, GCRMA has a memory problem and MAS5 is way too slow ====
It is a known problem for earlier versions of BRB-ArrayTools. The reason is statconnDCOM can only use 32-bit R. Since BRB-ArrayTools v4.3.0, a new engine 'Rserve' was adopted which can make use 64-bit R if the Windows OS is 64-bit. Therefore the memory issue for using GCRMA should be alleviated.
 
BRB-ArrayTools uses justGCRMA() function in 'gcrma' package for GCRMA option and justMAS() function in 'simpleaffy' package for MAS5 option.
 
=== R packages ===
* GCRMA: install cdf, probe, .db packages
* MAS5: install cdf, .db
 
=== Foreign langauge users ===
 
Below are some of the recommendations we typical make to foreign language users:
1: Please, make sure that the regional language settings on your machine to the "English". You can do so by going to the Control Panel ->  Regional and Language Options and choose English.
2: From the Start  Programs  Microsoft office  Microsoft office tools -> ->"Microsoft Office language settings" and please make sure that the "primary editing language" is "English".  If the "Primary editing language" is not "English", please change it to "English" and then re-boot your machine.
 
=== Quirks ===
* Do not place the project in a very deep path.
* Do not include special characters (single/double quote, percent sign, etc), in the project name, output name, column header in the experiment descriptor worksheet.
* Do not sort the experiment descriptor worksheet.
* R's ''impute'' package tends to crash R when the number of genes is small.
* R's ''pamr'' package failed when the number of genes is only one. The error message is
<pre>
Error in rep(1, p) : invalid 'times' argument
</pre>
It is a bug in pamr.train() -> nsc().
* Write the R file used in plugins in a ''conventional'' format.
* Bioconductor package 'affxparser' does not work on Windows XP. The alternative is to use Affymetrix Expression Console to pre-process your ST arrays data and then import the .txt file that are outputted from Affymetrix Expression Console into ArrayTools by using the general format importer.
 
== Citing ==
* [http://www.citeulike.org/search/all?q=arraytools citeulike]
* [http://scholar.google.com/scholar?hl=en&lr=&scoring=r&q=BRB+ArrayTools&as_ylo=1900&btnG=Search google scholar]
 
== Plugins Developers ==
 
* Experiment descriptor is a data frame with numerical or character data type (no factor). So we shall take extra care for cell with NA (numerical data type) or blank (character data type) value.
* Gene identifier is a data frame with factor data type.
 
== Support ==
 
=== Send an email to [email protected] ===
 
Please provide enough information to us so we can understand the problem.
 
* If a bug report file was generated, be sure to send it to us.
* If the question is like 'what method or parameters should be choose to run my analysis', please consult other experienced people near you.
* Since the software depends on a couple factors like Windows operation system, MS-Office, R. Please provide us more detailed information about the software background including BRB-ArrayTools.
* When sending screenshots to us, please provide all error screenshots. If you only provide any random of them, it will create a misleading to us.
 
=== Check BRB-ArrayTools message board [https://secure.emmes.com/brbmessages/index.php] ===
 
== MISC ==
=== [http://www.illumina.com/gsp/genomestudiohelp/default.htm Illumina Genome Studio] Manual ===
=== GEO ===
* [http://www.ncbi.nlm.nih.gov/geo/browse/?view=platforms&search=illumina&display=20&zsort=samples Illumina]. For example, the txt file <GSE13040_nonorm_nobkgd.txt> from [http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE13040 GSE13040] (Illumina MouseRef-8 v2.0 expression beadchip) is close but not the one BRB-ArrayTools requires. We can modify the header to satisfy AT's requirement.
 
=== Affymetrix SNP arrays and cnt file ===
To use the ‘Platform special importer’ for Affymetrix SNP arrays, the user should run the Copy Number Analysis Tool (CNAT) software provided freely by Affymetrix, and output the *.CNT files in a batch process, where one *.CNT file is produced for each SNP array that was performed.  The *.CNT output files should all be placed in one data folder, to be read by CGHTools.  The “ProbeSet”, “Chromosome”, “Position” and “Log2Ratio” data columns will be automatically extracted from each *.CNT file.
 
See also [http://doc.goldenhelix.com/SVS/latest/platform_notes.html#creating-cnt-files-using-the-affymetrix-cnat-batch-analysis-tool goldenhelix.com] for more about creating CNT files using the Affymetrix CNAT Batch Analysis Tool.
 
[http://www.affymetrix.com/support/technical/byproduct.affx?product=cnat Affymetrix] website contains some info about Chromosome Copy Number Analysis Tool (CNAT) software. [http://www.affymetrix.com/estore/partners_programs/programs/developer/whitepapers/automating_workflows.affx This site] provides links to command line tools to process 10K, 100K, and 500K (no SNP 5 or SNP 6) data from CHP file to CNT file. In fact, the tool download page [http://www.affymetrix.com/estore/partners_programs/programs/developer/tools/devnettools.affx here] even provides sample output for download.
 
We may obtain raw files for copy number from [http://www.ncbi.nlm.nih.gov/geo/browse/ GEO] website by searching 'affymetrix' and '10k' under 'Platforms' tab as keywords.
 
== [[ATdeveloper|Developer's page]] ==

Latest revision as of 20:51, 6 July 2022

Shortcuts to some pages that may be useful to you!

BRB-ArrayTools BRB-ArrayTools Linux Linux R R
Tai Chi Tai Chi Health Health Raspberry Pi Raspberry Pi
Docker Docker Recipes Recipes Statistics Statistics
Ubuntu Ubuntu Virtualbox Virtualbox Beaglebone Beaglebone


MC Li

Retrieved from "https://wiki.taichimd.us/index.php?title=Main_Page&oldid=33499"