R packages: Difference between revisions
Line 1,583: | Line 1,583: | ||
* [https://blog.rstudio.com/2019/11/06/renv-project-environments-for-r/ release] 2019-11-6 | * [https://blog.rstudio.com/2019/11/06/renv-project-environments-for-r/ release] 2019-11-6 | ||
* [https://rstudio.github.io/renv/articles/renv.html Introduction to renv] 2021-01-09 | * [https://rstudio.github.io/renv/articles/renv.html Introduction to renv] 2021-01-09 | ||
* [https://www.r-bloggers.com/2023/03/r-renv-how-to-manage-dependencies-in-r-projects-easily/ R renv: How to Manage Dependencies in R Projects Easily] 2023-03-22 | |||
* The [https://rstudio.github.io/renv/reference/migrate.html renv::migrate()] function makes it possible to migrate projects from '''Packrat''' to '''renv'''. | * The [https://rstudio.github.io/renv/reference/migrate.html renv::migrate()] function makes it possible to migrate projects from '''Packrat''' to '''renv'''. | ||
* [https://blog.rstudio.com/2020/08/20/why-package-environment-management-is-critical-for-serious-data-science/ Why Package & Environment Management is Critical for Serious Data Science] and a [https://garciamikep.github.io/useR-webinar/#41 workflow]. | * [https://blog.rstudio.com/2020/08/20/why-package-environment-management-is-critical-for-serious-data-science/ Why Package & Environment Management is Critical for Serious Data Science] and a [https://garciamikep.github.io/useR-webinar/#41 workflow]. |
Revision as of 06:59, 24 March 2023
R package management
Challenges in Package Management
Challenges in Package Management
Packages loaded at startup
getOption("defaultPackages")
How to install a new package
- The Comprehensive Guide to Installing R Packages from CRAN, Bioconductor, GitHub and Co.
- An example from powsimR package
utils::install.packages()
- By default, install.packages() will check versions and install uninstalled packages shown in 'Depends', 'Imports' , and 'LinkingTo' (DIL) fields. See R-exts manual.
- Take advantage of Ncpus parameter in install.packages()
- If we want to install packages listed in 'Suggests' field, we should specify it explicitly by using dependencies argument:
install.packages(XXXX, dependencies = c("Depends", "Imports", "Suggests", "LinkingTo")) # OR install.packages(XXXX, dependencies = TRUE)
For example, if I use a plain install.packages() command to install downloader package, it only installs 'digest' and 'downloader' packages. If I useinstall.packages("downloader", dependencies=TRUE)
it will also install 'testhat' package.
- Even a warning is given when some imports package is not available, it does not stop the installation (trying to install plotROC package on mac R 3.5.3). See also Warning: dependency ‘XXX’ is not available.
> install.packages("plotROC") --- Please select a CRAN mirror for use in this session --- Warning: dependency ‘gridSVG’ is not available trying URL 'https://cloud.r-project.org/bin/macosx/el-capitan/contrib/3.5/plotROC_2.2.1.tgz' Content type 'application/x-gzip' length 1288370 bytes (1.2 MB) ================================================== downloaded 1.2 MB The downloaded binary packages are in /var/folders/2q/slryb0rx4tj97t66v7l6pwvr_z6g3s/T//Rtmptrf2ZS/downloaded_packages > library(gridSVG) Error in library(gridSVG) : there is no package called ‘gridSVG’ > install.packages("gridSVG", type = "source") Warning message: package ‘gridSVG’ is not available (for R version 3.5.3) > ap <- available.packages() > "gridSVG" %in% rownames(ap) # [1] FALSE > dim(ap) # [1] 14220 17 > library(plotROC) Loading required package: ggplot2
- Look at CRAN, we see the latest version (1.7-0) of gridSVG depends on the latest version of R (3.6.0). But why even the source is removed from the old version of R?
- The install.packages function source code can be found in R -> src -> library -> utils -> R -> packages2.R file from Github repository (put 'install.packages' in the search box).
mirror
local({r <- getOption("repos") r["CRAN"] <- "http://cran.r-project.org" options(repos=r) }) options("repos")
RStudio
On my mac,
- install.packages() will install the packages to User Library '/Users/USERNAME/Library/R/3.6/library'
- RStudio GUI method will install the packages to System Library '/Library/Frameworks/R.framework/Versions/3.6/Resources/library'. So if we want to remove a package, we need to specify the lib parameter in remove.packages() or clicking on the adjacent X icon to remove it.
Packages -> Install package(s) from local files...
It works on tarball file.
pacman: Install and load the package at the same time, installing temporarily
p_load() function from the pacman package.
An example of installing the purrr package.
Also used by BBC Visual and Data Journalism cookbook for R graphics
Also used by Biowulf/NIH
For bioconductor, use, for example,
pacman::p_load("BioManager") pacman::p_load("DESeq2",try.bioconductor=TRUE,update.bioconductor=TRUE)
PS. Brute force method
mypackages<-c("ggplot2", "dplyr") for (p in mypackages){ cond <- suppressWarnings(!require(p, character.only = TRUE)) if(cond){ try(install.packages(p), silent = TRUE) library(p, character.only = TRUE) } }
remotes
devtools depends on 92 (non-base) packages and remotes depends on none.
Download and install R packages stored in 'GitHub', 'BitBucket', or plain 'subversion' or 'git' repositories. This package is a lightweight replacement of the 'install_*' functions in 'devtools'.
- https://cran.r-project.org/web/packages/remotes/index.html
- https://www.rdocumentation.org/packages/remotes/versions/2.0.2
To install a package from a local machines with all dependency, run remotes::install_local(path = "MyPackage.tar.gz", dependencies=TRUE) or devtools::install() though the later requires to untar the source first.
remotes::install_github() vs devtools::install_github()
See the answers. remotes is a lighter weight package for those who don't need/want all the other functionality in devtools. The remotes package says it copied most of its functionality from devtools, so the functions are likely the same.
devtools
- devtools::install()
- devtools::install(".", ...) assumes the current location contains the package content; eg Dockerfile from Bioc2020 template.
pak
https://cran.r-project.org/web/packages/pak/index.html. 'pak' supports CRAN, 'Bioconductor' and 'GitHub' packages. An example about the censored package.
BiocManager
https://cran.r-project.org/web/packages/BiocManager/index.html
pkgman
pkgman: A fresh approach to package installation - Gábor Csárdi
ccache: Faster R package installation
Ubuntu
- Finding Binary .deb Files for CRAN Packages
- Add an additional binary source to your source.lists file, see for example, the line in rocker. This is mentioned in Faster R package installation.
- R and CRAN Binaries for Ubuntu by Dirk Eddelbuettel
- r2u: R Binaries for Ubuntu; see The new #CRANapt repo has 19000 .deb binaries (and 200+ from BioConductor) for each with full dependencies and `apt` integration.
- rspm. {rspm}: easy access to RSPM binary packages with automatic management of system requirements
- bspm R package (cf rocker/r-rspm docker image which is not an R package). 5 lines. After that, "install.packages()" will trigger "sudo apt-get install". Note: it seems the apt-add-repository commands will break the apt update command on Debian 11. After a successful installation on Ubuntu OS, I'll see the following messages every time I started R,
Loading required package: utils Tracing function "install.packages" in package "utils"
Also for some reason, I cannot remove packages that were just installed. Not sure if the same problem happened if I use rig (based on rspm).
> install.packages("fgsea") # Works fine > remove.packages("fgsea") Removing package from ‘/home/brb/R/x86_64-pc-linux-gnu-library/4.2’ (as ‘lib’ is unspecified) Error in find.package(pkgs, lib) : there is no package called ‘fgsea’ > packageVersion("fgsea") [1] ‘1.22.0’
Since the bspm::enable() function enables the integration of install_sys() into install.packages(), we have to use remove_sys(pkgs) to uninstall a package. See API.
RStudio Package Manager/RSPM & repository
- Announcing Public Package Manager and v1.1.6
- Welcome RSPM and test-drive with Bionic and Focal
- Serving Package Binaries. These do not include Debian:( So I cannot test on Docker container.
- Package Manager allows linux users to install pre-built binaries of the packages which will make install quicker. See an example here.
# Freeze to Apr 29, 2021 8:00 PM options(repos = c(REPO_NAME = "https://packagemanager.rstudio.com/all/2639103")) # Using Linux Binary Packages https://r-pkgs.example.com/our-cran/__linux__/xenial/latest https://packagemanager.rstudio.com/cran/__linux__/focal/2021-04-23
- Bring Your Own Binary Packages with RSPM
install a package as it existed on a specific date/snapshot: mran repository
- MRAN Time Machine will be retired on July 1 2023
- MRAN is getting shutdown - what else is there for reproducibility with R, or why reproducibility is on a continuum?
- MRAN snapshots, and you May 22, 2019.
Look at https://cran.microsoft.com/snapshot/, it seems the snapshot went back 2014 for R 3.1.1.
For example,
install.packages("xgboost", repos="https://mran.microsoft.com/snapshot/2019-09-20/")
This trick works great when I try to install an R package (glmnet 3.0-2) on an old version of R. The current glmnet requires R 3.6.0. So even the source package is not available for older versions of R. If I install a package that does not require a very new version of R, that it works. The same problem happens on Windows OS. (today is 2020-05-05)
$ docker run --rm -it r-base:3.5.3 ... > install.packages("glmnet") Installing package into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) Warning message: package ‘glmnet’ is not available (for R version 3.5.3) > install.packages("glmnet", repos="https://mran.microsoft.com/snapshot/2019-09-20/") ... * DONE (glmnet) The downloaded source packages are in ‘/tmp/RtmpoWVWFQ/downloaded_packages’ > packageVersion("glmnet") [1] ‘2.0.18’ > options()$repos CRAN "https://cloud.r-project.org" > install.packages('knitr') # OK.
install a package on an old version of R
- Currently R is 4.2.2
- I have build R 4.1.3 and R 4.0.5 from source on Ubuntu 22.04. P.S. We need to include/enable "Source code" URIs Error :: You must put some 'source' URIs in your sources.list.
sudo apt update sudo apt-get build-dep r-base tar -xzvf R-4.1.3.tar.gz cd R-4.1.3 ./configure time make -j2 bin/R
- I'm trying to install DESEq2 on R 4.1.3 and R 4.0.5
- On R 4.1.3, the installation is successful.
install.packages("BiocManager") BiocManager::install("DESeq2") packageVersion("DESeq2") # 1.34.0
- On R 4.0.5, the installation of "locfit" (current version 1.5-9.7) requires R >= 4.1.0. Check on CRAN, we see the locfit version is 1.5-6 for R 4.0.5 (2021-3-31). So I try this version first.
install.packages("BiocManager") install.packages("remotes") remotes::install_version("locfit", "1.5-6") # Error: a 'NAMESPACE' file is required # Try the next version 1.5-7 remotes::install_version("locfit", "1.5-7") # Works BiocManager::install("DESeq2") # Works too packageVersion("DESeq2") # 1.30.1
- Now it is a good opportunity to try renv. So I create a new folder 'testProject' and create a new file 'test.R'. This R file contains only one line: library(DESeq2).
> setwd("testProj") > install.packages("renv") > renv::init() * Initializing project ... * Discovering package dependencies ... Done! * Copying packages into the cache ... [79/79] Done! One or more Bioconductor packages are used in your project, but the BiocManager package is not available. Consider installing BiocManager before snapshot. The following package(s) will be updated in the lockfile: ... The version of R recorded in the lockfile will be updated: - R [*] -> [4.0.5] * Lockfile written to '~/Downloads/testProject/renv.lock'. * Project '~/Downloads/testProject' loaded. [renv 0.16.0] * renv activated -- please restart the R session. > renv::install("BiocManager") # NOT SURE IF THIS IS NECESSARY Retrieving 'https://cloud.r-project.org/src/contrib/BiocManager_1.30.19.tar.gz' ... OK [file is up to date] Installing BiocManager [1.30.19] ... OK [built from source] Moving BiocManager [1.30.19] into the cache ... OK [moved to cache in 20 milliseconds] > q()
Now DESeq2 is in "renv.lock" file!
Note if we accidentally start a different version of R on an renv-ed project based on an older version of R, we shall see some special message informing us The project library is out of sync with the lockfile. At this time, the "renv.lock" is not touched yet.
# Bootstrapping renv 0.16.0 -------------------------------------------------- * Downloading renv 0.16.0 ... OK (downloaded source) * Installing renv 0.16.0 ... Done! * Successfully installed and loaded renv 0.16.0. ! Using R 4.2.2 (lockfile was generated with R 4.0.5) * Project '~/Downloads/testProject' loaded. [renv 0.16.0] * The project library is out of sync with the lockfile. * Use `renv::restore()` to install packages recorded in the lockfile.
- What happened if we start with the same version of R but have a different version of renv in the global environment (vs renv.lock)?
install.packages("remotes") remotes::install_version("renv", "0.15.5") renv::init() q() # Start the same R from another directory and update renv to the latest version 0.16.0 # Now go back to the project directory and start R to see what will happen? # Nothing.... This is because the same project has the old & same version of "renv" library # in the "$PROJECT/renv" directory
If we delete the "renv" subdirectory, ".Rprofile" file but keep "renv.lock" file, and start R (remember the 'renv' package version in the global R is different from the version in "renv.lock".
> renv::restore() This project has not yet been activated. Activating this project will ensure the project library is used during restore. Please see `?renv::activate` for more details. Would you like to activate this project before restore? [Y/n]: y Retrieving 'https://cloud.r-project.org/src/contrib/BiocManager_1.30.19.tar.gz' ... OK [file is up to date] Installing BiocManager [1.30.19] ... OK [built from source] Moving BiocManager [1.30.19] into the cache ... OK [moved to cache in 27 milliseconds] * Project '~/Downloads/testProject' loaded. [renv 0.16.0] * The project may be out of sync -- use `renv::status()` for more details. The following package(s) will be updated: # Bioconductor ======================= - AnnotationDbi [* -> 1.56.2] ... # CRAN =============================== - renv [0.16.0 -> 0.15.5] ... Do you want to proceed? [y/N]: y Retrieving 'https://cloud.r-project.org/src/contrib/Archive/renv/renv_0.15.5.tar.gz' ... OK [downloaded 961.3 Kb in 0.2 secs] Installing renv [0.15.5] ... OK [built from source] Moving renv [0.15.5] into the cache ... OK [moved to cache in 6.8 milliseconds] ... The following package(s) have been updated: renv [installed version 0.15.5 != loaded version 0.16.0] Consider restarting the R session and loading the newly-installed packages.
All the R packages are restored in the $PROJECT/renv directory.
Check installed Bioconductor version
Following this post, use tools:::.BioC_version_associated_with_R_version().
Mind the '.' in front of the 'BioC'. It may be possible for some installed packages to have been sourced from a different BioC version.
tools:::.BioC_version_associated_with_R_version() # `3.6' tools:::.BioC_version_associated_with_R_version() == '3.6' # TRUE
CRAN Package Depends on Bioconductor Package
For example, if I run install.packages("NanoStringNorm") to install the package from CRAN, I may get
ERROR: dependency ‘vsn’ is not available for package ‘NanoStringNorm’
This is because the NanoStringNorm package depends on the vsn package which is on Bioconductor.
Another example is CRAN's tidyHeatmap that imports ComplexHeatmap in Bioconductor.
Another instance is CRAN's biospear (actually plsRcox) that depends on Bioc's 'survcomp' & 'mixOmics'.
One solution is to run a line setRepositories(ind=1:2). See this post or this one. Note that the default repository list can be found at (Ubuntu) /usr/lib/R/etc/repositories file.
options("repos") # display the available repositories (only CRAN) setRepositories(ind=1:2) options("repos") # CRAN and bioc are included # CRAN # "https://cloud.r-project.org" # "https://bioconductor.org/packages/3.6/bioc" install.packages("biospear") # it will prompt to select CRAN install.packages("biospear", repos = "http://cran.rstudio.com") # NOT work since bioc repos is erased
This will also install the BiocInstaller package if it has not been installed before. See also Install Bioconductor Packages.
Bioconductor packages depend on CRAN
For example cowplot shows breakpointR from Bioconductor depends on it.
update.packages()
update.packages(ask="graphics") can open a graphical window to select packages.
Binary packages only for two versions of R
Check What Repos You are Using.
CRAN only provides binaries for one version of R prior to the current one. So when CRAN moves to post-3.6.* R most non version-stuck mirrors will not have 3.5.* binary versions of packages.
Install a tar.gz (e.g. an archived package) from a local directory
R CMD INSTALL <package-name>.tar.gz
Or in R:
# Method 1: cannot install dependencies install.packages(<pathtopackage>, repos = NULL) # These paths can be source directories or archives or binary package # archive files (as created by ‘R CMD build --binary’). # (‘http://’ and ‘file://’ URLs are also accepted and the files # will be downloaded and installed from local copies.) # Method 2: take care of dependencies from CRAN devtools::install(<directory to package>, dependencies = TRUE) # this will use 'R CMD INSTALL' to install the package. # It will try to install dependencies of the package from CRAN, # if they're not already installed.
The installation process can be nasty due to the dependency issue. Consider the 'biospear' package
biospear - plsRcox (archived) - plsRglm (archived) - bipartite - lars - pls - kernlab - mixOmics (CRAN->Bioconductor) - risksetROC - survcomp (Bioconductor) - rms
So in order to install the 'plsRcox' package, we need to do the following steps. Note: plsRcox package is back on 6/2/2018.
# For curl system("apt update") system("apt install curl libcurl4-openssl-dev libssl-dev") # For X11 system("apt install libcgal-dev libglu1-mesa-dev libglu1-mesa-dev") system("apt install libfreetype6-dev") # https://stackoverflow.com/questions/31820865/error-in-installing-rgl-package
source("https://bioconductor.org/biocLite.R") biocLite("survcomp") # this has to be run before the next command of installing a bunch of packages from CRAN install.packages("https://cran.r-project.org/src/contrib/Archive/biospear/biospear_1.0.1.tar.gz", repos = NULL, type="source") # ERROR: dependencies ‘pkgconfig’, ‘cobs’, ‘corpcor’, ‘devtools’, ‘glmnet’, ‘grplasso’, ‘mboost’, ‘plsRcox’, # ‘pROC’, ‘PRROC’, ‘RCurl’, ‘survAUC’ are not available for package ‘biospear’ install.packages(c("pkgconfig", "cobs", "corpcor", "devtools", "glmnet", "grplasso", "mboost", "plsRcox", "pROC", "PRROC", "RCurl", "survAUC")) # optional: install.packages(c("doRNG", "mvnfast")) install.packages("https://cran.r-project.org/src/contrib/Archive/biospear/biospear_1.0.1.tar.gz", repos = NULL, type="source") # OR # devtools::install_github("cran/biospear") library(biospear) # verify
To install the (deprecated, bioc) packages 'inSilicoMerging',
biocLite(c('rjson', 'Biobase', 'RCurl')) # destination directory is required # download.file("http://www.bioconductor.org/packages/3.3/bioc/src/contrib/inSilicoDb_2.7.0.tar.gz", # "~/Downloads/inSilicoDb_2.7.0.tar.gz") # download.file("http://www.bioconductor.org/packages/3.3/bioc/src/contrib/inSilicoMerging_1.15.0.tar.gz", # "~/Downloads/inSilicoMerging_1.15.0.tar.gz") # ~/Downloads or $HOME/Downloads won't work in untar() # untar("~/Downloads/inSilicoDb_2.7.0.tar.gz", exdir="/home/brb/Downloads") # untar("~/Downloads/inSilicoMerging_1.15.0.tar.gz", exdir="/home/brb/Downloads") # install.packages("~/Downloads/inSilicoDb", repos = NULL) # install.packages("~/Downloads/inSilicoMerging", repos = NULL) install.packages("http://www.bioconductor.org/packages/3.3/bioc/src/contrib/inSilicoDb_2.7.0.tar.gz", repos = NULL, type = "source") install.packages("http://www.bioconductor.org/packages/3.3/bioc/src/contrib/inSilicoMerging_1.15.0.tar.gz", repos = NULL, type = "source")
R CMD INSTALL -l LIB/--library=LIB option
Install a package to a custom location
$ R CMD INSTALL -l /usr/me/localR/library myRPackage.tar.gz
Use a package installed in a custom location
R> library("myRPackage", lib.loc="/usr/me/local/R/library") # OR include below in .bashrc file $ export R_LIBS=/usr/me/local/R/library R> .libPaths() # check R> library("myRPackage")
Install a specific version of R/Bioconductor package
For packages from CRAN, use something like remotes::install_version("dplyr", "1.0.2")
For packages from Bioconductor, see the two solutions R how to install a specified version of a bioconductor package?
Install multiple/different versions of the same R package
https://stackoverflow.com/a/2989369
install.packages("~/Downloads/foo_0.1.1.tar.gz", lib = "/tmp", repos = NULL) # a new folder "/tmp/foo" will be created library(foo, lib.loc="/tmp") # Or use 'lib' to be consistent with install.packages() library(foo, lib.loc="~/dev/foo/v1") ## loads v1 # and library(foo, lib.loc="~/dev/foo/v2") ## loads v2 packageVersion("foo", lib.loc = "/tmp") sessionInfo() help(package = "foo", lib.loc = "/tmp") remove.packages("foo", lib = "/tmp")
The same works for install.packages(). help(install.packages)
The install_version() from devtools and remotes will overwrite the existing installation.
Query an R package installed locally
packageDescription("MASS") packageVersion("MASS")
Query an R package (from CRAN) basic information: available.packages()
packageStatus() # Summarize information about installed packages available.packages() # List Available Packages at CRAN-like Repositories # Even I use an old version of R, it still return the latest version of the packages # The 'problem' happens on install.packages() too.
The available.packages() command is useful for understanding package dependency. Use setRepositories() or 'RGUI -> Packages -> select repositories' to select repositories and options()$repos to check or change the repositories.
The return result of available.packages() depends on the R version.
Also the packageStatus() is another useful function for query how many packages are in the repositories, how many have been installed, and individual package status (installed or not, needs to be upgraded or not).
> options()$repos CRAN "https://cran.rstudio.com/" > packageStatus() Number of installed packages: ok upgrade unavailable C:/Program Files/R/R-3.0.1/library 110 0 1 Number of available packages (each package counted only once): installed not installed http://watson.nci.nih.gov/cran_mirror/bin/windows/contrib/3.0 76 4563 http://www.stats.ox.ac.uk/pub/RWin/bin/windows/contrib/3.0 0 5 http://www.bioconductor.org/packages/2.12/bioc/bin/windows/contrib/3.0 16 625 http://www.bioconductor.org/packages/2.12/data/annotation/bin/windows/contrib/3.0 4 686 > tmp <- available.packages() > str(tmp) chr [1:5975, 1:17] "A3" "ABCExtremes" "ABCp2" "ACCLMA" "ACD" "ACNE" "ADGofTest" "ADM3" "AER" ... - attr(*, "dimnames")=List of 2 ..$ : chr [1:5975] "A3" "ABCExtremes" "ABCp2" "ACCLMA" ... ..$ : chr [1:17] "Package" "Version" "Priority" "Depends" ... > tmp[1:3,] Package Version Priority Depends Imports LinkingTo Suggests A3 "A3" "0.9.2" NA "xtable, pbapply" NA NA "randomForest, e1071" ABCExtremes "ABCExtremes" "1.0" NA "SpatialExtremes, combinat" NA NA NA ABCp2 "ABCp2" "1.1" NA "MASS" NA NA NA Enhances License License_is_FOSS License_restricts_use OS_type Archs MD5sum NeedsCompilation File A3 NA "GPL (>= 2)" NA NA NA NA NA NA NA ABCExtremes NA "GPL-2" NA NA NA NA NA NA NA ABCp2 NA "GPL-2" NA NA NA NA NA NA NA Repository A3 "http://watson.nci.nih.gov/cran_mirror/bin/windows/contrib/3.0" ABCExtremes "http://watson.nci.nih.gov/cran_mirror/bin/windows/contrib/3.0" ABCp2 "http://watson.nci.nih.gov/cran_mirror/bin/windows/contrib/3.0"
And the following commands find which package depends on Rcpp and also which are from bioconductor repository.
> pkgName <- "Rcpp" > rownames(tmp)[grep(pkgName, tmp[,"Depends"])] > tmp[grep("Rcpp", tmp[,"Depends"]), "Depends"] > ind <- intersect(grep(pkgName, tmp[,"Depends"]), grep("bioconductor", tmp[, "Repository"])) > rownames(grep)[ind] NULL > rownames(tmp)[ind] [1] "ddgraph" "DESeq2" "GeneNetworkBuilder" "GOSemSim" "GRENITS" [6] "mosaics" "mzR" "pcaMethods" "Rdisop" "Risa" [11] "rTANDEM"
CRAN vs Bioconductor packages
> R.version # 3.4.3 # CRAN > x <- available.packages() > dim(x) [1] 12581 17 # Bioconductor Soft > biocinstallRepos() BioCsoft "https://bioconductor.org/packages/3.6/bioc" BioCann "https://bioconductor.org/packages/3.6/data/annotation" BioCexp "https://bioconductor.org/packages/3.6/data/experiment" CRAN "https://cran.rstudio.com/" > y <- available.packages(repos = biocinstallRepos()[1]) > dim(y) [1] 1477 17 > intersect(x[, "Package"], y[, "Package"]) character(0) # Bioconductor Annotation > dim(available.packages(repos = biocinstallRepos()[2])) [1] 909 17 # Bioconductor Experiment > dim(available.packages(repos = biocinstallRepos()[3])) [1] 326 17 # CRAN + All Bioconductor > z <- available.packages(repos = biocinstallRepos()) > dim(z) [1] 15292 17
Downloading Bioconductor package with an old R
When I try to download the GenomicDataCommons package using R 3.4.4 with Bioc 3.6 (the current R version is 3.5.0), it was found it can only install version 1.2.0 instead the latest version 1.4.1.
It does not work by running biocLite("BiocUpgrade") to upgrade Bioc from 3.6 to 3.7.
source("https://bioconductor.org/biocLite.R") biocLite("BiocUpgrade") # Error: Bioconductor version 3.6 cannot be upgraded with R version 3.4.4
See some instruction on RStudio package manager website.
Analyzing data on CRAN packages
New undocumented function in R 3.4.0: tools::CRAN_package_db()
http://blog.revolutionanalytics.com/2017/05/analyzing-data-on-cran-packages.html
R package location when they are installed by root
/usr/local/lib/R/site-library
Customizing your package/library location
Add a personal directory to .libPaths()
.libPaths( c( .libPaths(), "~/userLibrary") )
No need to use the assignment operator.
Install personal R packages after upgrade R, .libPaths(), Rprofile.site, R_LIBS_USER
File | Example |
---|---|
Rprofile.site/.Rprofile | .libPaths(c("/usr/lib/R/site-library", "/usr/lib/R/library")) |
Renviron.site/.Renviron | R_LIB_SITE="/usr/lib/R/site-library:/usr/lib/R/library" |
Scenario: We already have installed many R packages under R 3.1.X in the user's directory. Now we upgrade R to a new version (3.2.X). We like to have these packages available in R 3.2.X.
For Windows OS, refer to R for Windows FAQ
The follow method works on Linux and Windows.
Make sure only one instance of R is running
# Step 1. update R's built-in packages and install them on my personal directory update.packages(ask=FALSE, checkBuilt = TRUE, repos="http://cran.rstudio.com") # Step 2. update Bioconductor packages .libPaths() # The first one is my personal directory # [1] "/home/brb/R/x86_64-pc-linux-gnu-library/3.2" # [2] "/usr/local/lib/R/site-library" # [3] "/usr/lib/R/site-library" # [4] "/usr/lib/R/library" Sys.getenv("R_LIBS_USER") # may or may not equivalent to .libPaths()[1] ul <- unlist(strsplit(Sys.getenv("R_LIBS_USER"), "/")) src <- file.path(paste(ul[1:(length(ul)-1)], collapse="/"), "3.1") des <- file.path(paste(ul[1:(length(ul)-1)], collapse="/"), "3.2") pkg <- dir(src, full.names = TRUE) if (!file.exists(des)) dir.create(des) # If 3.2 subdirectory does not exist yet! file.copy(pkg, des, overwrite=FALSE, recursive = TRUE) source("http://www.bioconductor.org/biocLite.R") biocLite(ask = FALSE)
From Robert Kabacoff (R in Action)
- If you have a customized Rprofile.site file (see appendix B), save a copy outside of R.
- Launch your current version of R and issue the following statements
oldip <- installed.packages()[,1] save(oldip, file="path/installedPackages.Rdata")
where path is a directory outside of R.
- Download and install the newer version of R.
- If you saved a customized version of the Rprofile.site file in step 1, copy it into the new installation.
- Launch the new version of R, and issue the following statements
load("path/installedPackages.Rdata") newip <- installed.packages()[,1] for(i in setdiff(oldip, newip)) install.packages(i)
where path is the location specified in step 2.
- Delete the old installation (optional).
This approach will install only packages that are available from the CRAN. It won’t find packages obtained from other locations. In fact, the process will display a list of packages that can’t be installed For example for packages obtained from Bioconductor, use the following method to update packages
source(http://bioconductor.org/biocLite.R) biocLite(PKGNAME)
Persistent config and data for R packages with .Rprofile and .Renviron
Persistent config and data for R packages. startup, rappdirs, hoardr, keyring.
Would you like to use a personal library instead?
Some posts from internet
- Setting R_LIBS & avoiding “Would you like to use a personal library instead?”. Note: I try to create ~/.Renviron to add my personal folder in it. But update.packages() still asks me if I like to use a personal library instead (tested on Ubuntu + R 3.4).
- automatically create personal library in R. Using suppressUpdates + specify lib in biocLite() or update.packages(Sys.getenv("R_LIBS_USER"), ask = F)
# create local user library path (not present by default) dir.create(path = Sys.getenv("R_LIBS_USER"), showWarnings = FALSE, recursive = TRUE) # install to local user library path install.packages(p, lib = Sys.getenv("R_LIBS_USER"), repos = "https://cran.rstudio.com/") # Bioconductor version biocLite(p, suppressUpdates = TRUE, lib = Sys.getenv("R_LIBS_USER"))
The problem can happen if the R was installed to the C:\Program Files\R folder by users but then some main packages want to be upgraded. R will always pops a message 'Would you like to use a personal library instead?'.
To suppress the message and use the personal library always,
- Run R as administrator. If you do that, main packages can be upgraded from C:\Program Files\R\R-X.Y.Z\library folder.
- Writable R package directory cannot be found and a this. A solution here is to change the security of the R library folder so the user has a full control on the folder.
- Does R run under Windows Vista/7/8/Server 2008? There are 3 ways to get around the issue.
- I don’t have permission to write to the R-3.3.2\library directory
Actually the following hints will help us to create a convenient function UpdateMainLibrary() which will install updated main packages in the user's Documents directory without a warning dialog.
- .libPaths() only returns 1 string "C:/Program Files/R/R-x.y.z/library" on the machines that does not have this problem
- .libPaths() returns two strings "C:/Users/USERNAME/Documents/R/win-library/x.y" & "C:/Program Files/R/R-x.y.z/library" on machines with the problem.
UpdateMainLibrary <- function() { # Update main/site packages # The function is used to fix the problem 'Would you like to use a personal library instead?' if (length(.libPaths()) == 1) return() ind_mloc <- grep("Program", .libPaths()) # main library e.g. 2 ind_ploc <- grep("Documents", .libPaths()) # personal library e.g. 1 if (length(ind_mloc) > 0L && length(ind_ploc) > 0L) # search outdated main packages old_mloc <- ! old.packages(.libPaths()[ind_mloc])[, "Package"] %in% installed.packages(.libPaths()[ind_ploc])[, "Package"] oldpac <- old.packages(.libPaths()[ind_mloc])[old_mloc, "Package"] if (length(oldpac) > 0L) install.packages(oldpac, .libPaths()[ind_ploc]) }
On Linux,
> update.packages() ... The downloaded source packages are in ‘/tmp/RtmpBrYccd/downloaded_packages’ Warning in install.packages(update[instlib == l, "Package"], l, contriburl = contriburl, : 'lib = "/opt/R/3.5.0/lib/R/library"' is not writable Would you like to use a personal library instead? (yes/No/cancel) yes ... > system("ls -lt /home/brb/R/x86_64-pc-linux-gnu-library/3.5 | head") total 224 drwxrwxr-x 9 brb brb 4096 Oct 3 09:30 survival drwxrwxr-x 9 brb brb 4096 Oct 3 09:29 mgcv drwxrwxr-x 10 brb brb 4096 Oct 3 09:29 MASS drwxrwxr-x 9 brb brb 4096 Oct 3 09:29 foreign # So new versions of survival, mgc, MASS, foreign are installed in the personal directory # The update.packages() will issue warnings if we try to run it again. # It's OK to ignore these warnings. > update.packages() Warning: package 'foreign' in library '/opt/R/3.5.0/lib/R/library' will not be updated Warning: package 'MASS' in library '/opt/R/3.5.0/lib/R/library' will not be updated Warning: package 'mgcv' in library '/opt/R/3.5.0/lib/R/library' will not be updated Warning: package 'survival' in library '/opt/R/3.5.0/lib/R/library' will not be updated
R_LIBS_USER is empty in R 3.4.1
See install.package() error, R_LIBS_USER is empty in R 3.4.1.
List vignettes from a package
vignette(package=PACKAGENAME)
List data from a package
data(package=PACKAGENAME)
sysdata.rda
List all functions of a package
Assume a package is already loaded. Then
ls("package:cowplot")
Getting a list of functions and objects in a package. This also assumes the package in loaded. In addition to functions (separated by primitive and non-primitive), it can show constants and objects.
List installed packages and versions
- http://heuristicandrew.blogspot.com/2015/06/list-of-user-installed-r-packages-and.html
- checkpoint package
ip <- as.data.frame(installed.packages()[,c(1,3:4)]) rownames(ip) <- NULL unique(ip$Priority) # [1] <NA> base recommended # Levels: base recommended ip <- ip[is.na(ip$Priority),1:2,drop=FALSE] print(ip, row.names=FALSE)
Query the names of outdated packages
psi <- packageStatus()$inst subset(psi, Status == "upgrade", drop = FALSE) # Package LibPath Version Priority Depends # RcppArmadillo RcppArmadillo C:/Users/brb/Documents/R/win-library/3.2 0.5.100.1.0 <NA> <NA> # Matrix Matrix C:/Program Files/R/R-3.2.0/library 1.2-0 recommended R (>= 2.15.2), methods # Imports LinkingTo Suggests # RcppArmadillo Rcpp (>= 0.11.0) Rcpp RUnit, Matrix, pkgKitten # Matrix graphics, grid, stats, utils, lattice <NA> expm, MASS # Enhances License License_is_FOSS License_restricts_use OS_type MD5sum # RcppArmadillo <NA> GPL (>= 2) <NA> <NA> <NA> <NA> # Matrix MatrixModels, graph, SparseM, sfsmisc GPL (>= 2) <NA> <NA> <NA> <NA> # NeedsCompilation Built Status # RcppArmadillo yes 3.2.0 upgrade # Matrix yes 3.2.0 upgrade
The above output does not show the package version from the latest packages on CRAN. So the following snippet does that.
psi <- packageStatus()$inst pl <- unname(psi$Package[psi$Status == "upgrade"]) # List package names ap <- as.data.frame(available.packages()[, c(1,2,3)], stringsAsFactors = FALSE) out <- cbind(subset(psi, Status == "upgrade")[, c("Package", "Version")], ap[match(pl, ap$Package), "Version"]) colnames(out)[2:3] <- c("OldVersion", "NewVersion") rownames(out) <- NULL out # Package OldVersion NewVersion # 1 RcppArmadillo 0.5.100.1.0 0.5.200.1.0 # 2 Matrix 1.2-0 1.2-1
To consider also the packages from Bioconductor, we have the following code. Note that "3.1" means the Bioconductor version and "3.2" is the R version. See Bioconductor release versions page.
psic <- packageStatus(repos = c(contrib.url(getOption("repos")), "http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2", "http://www.bioconductor.org/packages/3.1/data/annotation/bin/windows/contrib/3.2"))$inst subset(psic, Status == "upgrade", drop = FALSE) pl <- unname(psic$Package[psic$Status == "upgrade"]) ap <- as.data.frame(available.packages(c(contrib.url(getOption("repos")), "http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2", "http://www.bioconductor.org/packages/3.1/data/annotation/bin/windows/contrib/3.2"))[, c(1:3)], stringAsFactors = FALSE) out <- cbind(subset(psic, Status == "upgrade")[, c("Package", "Version")], ap[match(pl, ap$Package), "Version"]) colnames(out)[2:3] <- c("OldVersion", "NewVersion") rownames(out) <- NULL out # Package OldVersion NewVersion # 1 limma 3.24.5 3.24.9 # 2 RcppArmadillo 0.5.100.1.0 0.5.200.1.0 # 3 Matrix 1.2-0 1.2-1
Searching for packages in CRAN
- pkgsearch package - Search R packages on CRAN
- Fishing for packages in CRAN
- CRAN now has 10,000 R packages. Here's how to find the ones you need
- Searching for R packages, packagefinder package
library(packagefinder) findPackage("survival") # 272 out of 13256 CRAN packages found in 5 seconds findPackage("follic") # No results found. # Actually 'follic' comes from randomForestSRC package # https://www.rdocumentation.org/packages/randomForestSRC/versions/2.7.0
The result is shown in an html format with columns of SCORE, NAME, DESC_SHORT, DOWNL_TOTAL & GO.
METACRAN (www.r-pkg.org) - Search and browse all CRAN/R packages
- Source code on https://github.com/metacran. The 'PACKAGES' file is updated regularly to Github.
- Announcement on R/mailing list
- Author's homepage on http://gaborcsardi.org/.
cranly visualisations and summaries for R packages
Exploring R packages with cranly
Query top downloaded packages, download statistics
- Daily download statistics http://cran-logs.rstudio.com/. Note the page is split into 'package' download and 'R' download. It tracks
- Package: date, time, size, r_version, r_arch, r_os, package, version, country, ip_id.
- R: date, time, size, R version, os (win/src/osx), county, ip_id (reset daily).
- Original methods
- https://www.r-bloggers.com/finally-tracking-cran-packages-downloads/. The code still works.
- My modified code for showing the top download R packages. The original code suffers from memory issue (tested on my 64GB Linux box) when it is calling rbindlist() from data.table package with large data. Still 64GB is required since the matrix is 369M by 4 (12GB).
- Popularity bigdata / large data packages in R and ffbase useR presentation
- pkginfo: Tools for Retrieving R Package Information. It's only in github. Shiny interface.
- Top 100 downloaded packages from METACRAN
cranlogs
cranlogs package - Download Logs from the RStudio CRAN Mirror. Suitable on R console.
- 2018 through {cranlogs}
- Shiny app by Hadley (works for packages on CRAN only). It's like the Google Trend app. An example of collection: survC1, survAUC, TreatmentSelection, biospear , (glmnet)
- cranlogs 2.1.1 is on CRAN! 5/2/2019
- limit on number of packages as argument to cran_downloads
library(cranlogs) out <- cran_top_downloads("last-week", 100) # 100 is the maximum limit out$package
packageRank
packageRank package: Computing and Visualizing CRAN Downloads. Suitable to run on RStudio cloud. Include both CRAN and Bioconductor.
> plot(cranDownloads(packages = c("packageRank", "limma"), when = "last-month")) > plot(cranDownloads(packages = c("shiny", "glmnet"), when = "last-month")) > plot(cranDownloads(packages = c("shiny", "glmnet"), from = "2019", to ="2019")) > plot(cranDownloads(packages = c("shiny", "glmnet"), from = "2019-12", to ="2019-12")) > plot(bioconductorDownloads(packages = c("edgeR", "DESeq2", "Rsubread", "limma"), when = "last-year"))
BiocPkgTools
For Bioconductor packages, try BiocPkgTools. See the paper.
dlstats
dlstats. Monthly download stats of 'CRAN' and 'Bioconductor' packages.
installation path not writeable from running biocLite()
When I ran biocLite() to install a new package, I got a message (the Bioc packages are installed successfully anyway)
... * DONE (curatedOvarianData) The downloaded source packages are in ‘/tmp/RtmpHxnH2K/downloaded_packages’ installation path not writeable, unable to update packages: rgl, rJava, codetools, foreign, lattice, MASS, spatial, survival
However, if I uses install.package() it can update the package
> packageVersion("survival") [1] ‘2.42.3’ > update.packages("survival") # Not working though no error message > packageVersion("survival") [1] ‘2.42.3’ > install.packages("survival") Installing package into ‘/home/brb/R/x86_64-pc-linux-gnu-library/3.4’ ... * DONE (survival) The downloaded source packages are in ‘/tmp/RtmpHxnH2K/downloaded_packages’ > packageVersion("survival") [1] ‘2.42.6’ > library(survival) > sessionInfo() # show survival package 2.42-6 was attached
It makes sense to always use personal directory when we install packages. See .libPaths().
Warning: cannot remove prior installation of package
Instance 1.
# Install the latest hgu133plus2cdf package # Remove/Uninstall hgu133plus2.db package # Put/Install an old version of IRanges (eg version 1.18.2 while currently it is version 1.18.3) # Test on R 3.0.1 library(hgu133plus2cdf) # hgu133pluscdf does not depend or import IRanges source("http://bioconductor.org/biocLite.R") biocLite("hgu133plus2.db", ask=FALSE) # hgu133plus2.db imports IRanges # Warning:cannot remove prior installation of package 'IRanges' # Open Windows Explorer and check IRanges folder. Only see libs subfolder.
Note:
- In the above example, all packages were installed under C:\Program Files\R\R-3.0.1\library\.
- In another instance where I cannot reproduce the problem, new R packages were installed under C:\Users\xxx\Documents\R\win-library\3.0\. The different thing is IRanges package CAN be updated but if I use packageVersion("IRanges") command in R, it still shows the old version.
- The above were tested on a desktop.
Instance 2.
# On a fresh R 3.2.0, I install Bioconductor's depPkgTools & lumi packages. Then I close R, re-open it, # and install depPkgTools package again. > source("http://bioconductor.org/biocLite.R") Bioconductor version 3.1 (BiocInstaller 1.18.2), ?biocLite for help > biocLite("pkgDepTools") BioC_mirror: http://bioconductor.org Using Bioconductor version 3.1 (BiocInstaller 1.18.2), R version 3.2.0. Installing package(s) ‘pkgDepTools’ trying URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/pkgDepTools_1.34.0.zip' Content type 'application/zip' length 390579 bytes (381 KB) downloaded 381 KB package ‘pkgDepTools’ successfully unpacked and MD5 sums checked Warning: cannot remove prior installation of package ‘pkgDepTools’ The downloaded binary packages are in C:\Users\brb\AppData\Local\Temp\RtmpYd2l7i\downloaded_packages > library(pkgDepTools) Error in library(pkgDepTools) : there is no package called ‘pkgDepTools’
The pkgDepTools library folder appears in C:\Users\brb\Documents\R\win-library\3.2, but it is empty. The weird thing is if I try the above steps again, I cannot reproduce the problem.
Warning: dependency ‘XXX’ is not available
How should I deal with “package 'xxx' is not available (for R version x.y.z)” warning?
Error: there is no package called XXX
The error happened when I try to run library() command on a package that was just installed. R 3.6.0. 'biospear' version is 1.0.2. macOS.
> library(biospear) Loading required package: pkgconfig Error: package or namespace load failed for ‘biospear’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vIi): there is no package called ‘mixOmics’
Note
- The package mixOmics was removed from CRAN. It is now available on Bioconductor.
- Tested to install on a docker container: docker run --net=host -it --rm r-base
ERROR: dependency ‘car’ is not available for package ‘plsRglm’ * removing ‘/usr/local/lib/R/site-library/plsRglm’ ERROR: dependencies ‘plsRglm’, ‘mixOmics’, ‘survcomp’ are not available for package ‘plsRcox’ * removing ‘/usr/local/lib/R/site-library/plsRcox’ ERROR: dependencies ‘devtools’, ‘plsRcox’, ‘RCurl’ are not available for package ‘biospear’ * removing ‘/usr/local/lib/R/site-library/biospear’
The car package looks OK on CRAN. survcomp was moved from CRAN to Bioconductor too. - As we can see above, the official r-base image does not contain libraries enough to install RCurl/devtools packages. Consider the tidyverse image (based on RStudio image) from the rocker project.
docker pull rocker/tidyverse:3.6.0 docker run --net=host -it --rm -e PASSWORD=password -p 8787:8787 rocker/tidyverse:3.6.0 # the default username is 'rstudio' # Open a browser, log in. Run 'install.packages("RCurl")'. It works.
- Testing on Mint linux also shows errors about dependencies of mixOmics and survcomp.
- The best practice to install a package that may depend on packages located/moved to Bioconductor: Run setRepositories(ind=1:2) before calling install.packages(). However, it does not remedy the situation that the 1st level imports package (eg plsRcox) was installed before but the 2nd level imports package (eg mixOmics) was not installed.
- dependsOnPkgs(): Find Reverse Dependencies. It seems it only return packages that have been installed locally. For example, tools::dependsOnPkgs("RcppEigen", "LinkingTo")
Warning: Unable to move temporary installation
The problem seems to happen only on virtual machines (Virtualbox).
- Warning: unable to move temporary installation `C:\Users\brb\Documents\R\win-library\3.0\fileed8270978f5\quadprog` to `C:\Users\brb\Documents\R\win-library\3.0\quadprog` when I try to run 'install.packages("forecast").
- Warning: unable to move temporary installation ‘C:\Users\brb\Documents\R\win-library\3.2\file5e0104b5b49\plyr’ to ‘C:\Users\brb\Documents\R\win-library\3.2\plyr’ when I try to run 'biocLite("lumi")'. The other dependency packages look fine although I am not sure if any unknown problem can happen (it does, see below).
Here is a note of my trouble shooting.
- If I try to ignore the warning and load the lumi package. I will get an error.
- If I try to run biocLite("lumi") again, it will only download & install lumi without checking missing 'plyr' package. Therefore, when I try to load the lumi package, it will give me an error again.
- Even I install the plyr package manually, library(lumi) gives another error - missing mclust package.
> biocLite("lumi") trying URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/BiocInstaller_1.18.2.zip' Content type 'application/zip' length 114097 bytes (111 KB) downloaded 111 KB ... package ‘lumi’ successfully unpacked and MD5 sums checked The downloaded binary packages are in C:\Users\brb\AppData\Local\Temp\RtmpyUjsJD\downloaded_packages Old packages: 'BiocParallel', 'Biostrings', 'caret', 'DESeq2', 'gdata', 'GenomicFeatures', 'gplots', 'Hmisc', 'Rcpp', 'RcppArmadillo', 'rgl', 'stringr' Update all/some/none? [a/s/n]: a also installing the dependencies ‘Rsamtools’, ‘GenomicAlignments’, ‘plyr’, ‘rtracklayer’, ‘gridExtra’, ‘stringi’, ‘magrittr’ trying URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/Rsamtools_1.20.1.zip' Content type 'application/zip' length 8138197 bytes (7.8 MB) downloaded 7.8 MB ... package ‘Rsamtools’ successfully unpacked and MD5 sums checked package ‘GenomicAlignments’ successfully unpacked and MD5 sums checked package ‘plyr’ successfully unpacked and MD5 sums checked Warning: unable to move temporary installation ‘C:\Users\brb\Documents\R\win-library\3.2\file5e0104b5b49\plyr’ to ‘C:\Users\brb\Documents\R\win-library\3.2\plyr’ package ‘rtracklayer’ successfully unpacked and MD5 sums checked package ‘gridExtra’ successfully unpacked and MD5 sums checked package ‘stringi’ successfully unpacked and MD5 sums checked package ‘magrittr’ successfully unpacked and MD5 sums checked package ‘BiocParallel’ successfully unpacked and MD5 sums checked package ‘Biostrings’ successfully unpacked and MD5 sums checked Warning: cannot remove prior installation of package ‘Biostrings’ package ‘caret’ successfully unpacked and MD5 sums checked package ‘DESeq2’ successfully unpacked and MD5 sums checked package ‘gdata’ successfully unpacked and MD5 sums checked package ‘GenomicFeatures’ successfully unpacked and MD5 sums checked package ‘gplots’ successfully unpacked and MD5 sums checked package ‘Hmisc’ successfully unpacked and MD5 sums checked package ‘Rcpp’ successfully unpacked and MD5 sums checked package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked package ‘rgl’ successfully unpacked and MD5 sums checked package ‘stringr’ successfully unpacked and MD5 sums checked The downloaded binary packages are in C:\Users\brb\AppData\Local\Temp\RtmpyUjsJD\downloaded_packages > library(lumi) Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vIi) : there is no package called ‘plyr’ Error: package or namespace load failed for ‘lumi’ > search() [1] ".GlobalEnv" "package:BiocInstaller" "package:Biobase" "package:BiocGenerics" "package:parallel" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" "package:datasets" "package:methods" "Autoloads" [13] "package:base" > biocLite("lumi") BioC_mirror: http://bioconductor.org Using Bioconductor version 3.1 (BiocInstaller 1.18.2), R version 3.2.0. Installing package(s) ‘lumi’ trying URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/lumi_2.20.1.zip' Content type 'application/zip' length 18185326 bytes (17.3 MB) downloaded 17.3 MB package ‘lumi’ successfully unpacked and MD5 sums checked The downloaded binary packages are in C:\Users\brb\AppData\Local\Temp\RtmpyUjsJD\downloaded_packages > search() [1] ".GlobalEnv" "package:BiocInstaller" "package:Biobase" "package:BiocGenerics" "package:parallel" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" "package:datasets" "package:methods" "Autoloads" [13] "package:base" > library(lumi) Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vIi) : there is no package called ‘plyr’ Error: package or namespace load failed for ‘lumi’ > biocLite("plyr") BioC_mirror: http://bioconductor.org Using Bioconductor version 3.1 (BiocInstaller 1.18.2), R version 3.2.0. Installing package(s) ‘plyr’ trying URL 'http://cran.rstudio.com/bin/windows/contrib/3.2/plyr_1.8.2.zip' Content type 'application/zip' length 1128621 bytes (1.1 MB) downloaded 1.1 MB package ‘plyr’ successfully unpacked and MD5 sums checked The downloaded binary packages are in C:\Users\brb\AppData\Local\Temp\RtmpyUjsJD\downloaded_packages > library(lumi) Error in loadNamespace(j <- i1L, c(lib.loc, .libPaths()), versionCheck = vIj) : there is no package called ‘mclust’ Error: package or namespace load failed for ‘lumi’ > ?biocLite Warning messages: 1: In read.dcf(file.path(p, "DESCRIPTION"), c("Package", "Version")) : cannot open compressed file 'C:/Users/brb/Documents/R/win-library/3.2/Biostrings/DESCRIPTION', probable reason 'No such file or directory' 2: In find.package(if (is.null(package)) loadedNamespaces() else package, : there is no package called ‘Biostrings’ > library(lumi) Error in loadNamespace(j <- i1L, c(lib.loc, .libPaths()), versionCheck = vIj) : there is no package called ‘mclust’ In addition: Warning messages: 1: In read.dcf(file.path(p, "DESCRIPTION"), c("Package", "Version")) : cannot open compressed file 'C:/Users/brb/Documents/R/win-library/3.2/Biostrings/DESCRIPTION', probable reason 'No such file or directory' 2: In find.package(if (is.null(package)) loadedNamespaces() else package, : there is no package called ‘Biostrings’ Error: package or namespace load failed for ‘lumi’
Other people also have the similar problem. The possible cause is the virus scanner locks the file and R cannot move them.
Some possible solutions:
- Delete ALL folders under R/library (e.g. C:/Progra~1/R/R-3.2.0/library) folder and install the main package again using install.packages() or biocLite().
- For specific package like 'lumi' from Bioconductor, we can find out all dependency packages and then install them one by one.
- Find out and install the top level package which misses dependency packages.
- This is based on the fact that install.packages() or biocLite() sometimes just checks & installs the 'Depends' and 'Imports' packages and won't install all packages recursively
- we can do a small experiment by removing a package which is not directly dependent/imported by another package; e.g. 'iterators' is not dependent/imported by 'glment' directly but indirectly. So if we run remove.packages("iterators"); install.packages("glmnet"), then the 'iterator' package is still missing.
- A real example is if the missing packages are 'Biostrings', 'limma', 'mclust' (these are packages that 'minfi' directly depends/imports although they should be installed when I run biocLite("lumi") command), then I should just run the command remove.packages("minfi"); biocLite("minfi"). If we just run biocLite("lumi") or biocLite("methylumi"), the missing packages won't be installed.
Error in download.file(url, destfile, method, mode = "wb", ...)
HTTP status was '404 Not Found'
Tested on an existing R-3.2.0 session. Note that VariantAnnotation 1.14.4 was just uploaded to Bioc.
> biocLite("COSMIC.67") BioC_mirror: http://bioconductor.org Using Bioconductor version 3.1 (BiocInstaller 1.18.3), R version 3.2.0. Installing package(s) ‘COSMIC.67’ also installing the dependency ‘VariantAnnotation’ trying URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/VariantAnnotation_1.14.3.zip' Error in download.file(url, destfile, method, mode = "wb", ...) : cannot open URL 'http://bioconductor.org/packages/3.1/bioc/bin/windows/contrib/3.2/VariantAnnotation_1.14.3.zip' In addition: Warning message: In download.file(url, destfile, method, mode = "wb", ...) : cannot open: HTTP status was '404 Not Found' Warning in download.packages(pkgs, destdir = tmpd, available = available, : download of package ‘VariantAnnotation’ failed installing the source package ‘COSMIC.67’ trying URL 'http://bioconductor.org/packages/3.1/data/experiment/src/contrib/COSMIC.67_1.4.0.tar.gz' Content type 'application/x-gzip' length 40999037 bytes (39.1 MB)
However, when I tested on a new R-3.2.0 (just installed in VM), the COSMIC package installation is successful. That VariantAnnotation version 1.14.4 was installed (this version was just updated today from Bioconductor).
The cause of the error is the available.package() function will read the rds file first from cache in a tempdir (C:\Users\XXXX\AppData\Local\Temp\RtmpYYYYYY). See lines 51-55 of <packages.R>.
dest <- file.path(tempdir(), paste0("repos_", URLencode(repos, TRUE), ".rds")) if(file.exists(dest)) { res0 <- readRDS(dest) } else { ... }
Since my R was opened 1 week ago, the rds file it reads today contains old information. Note that Bioconductor does not hold the source code or binary code for the old version of packages. This explains why biocLite() function broke. When I restart R, the original problem is gone.
If we look at the source code of available.packages(), we will see we could use cacheOK option in download.file() function.
download.file(url, destfile, method, cacheOK = FALSE, quiet = TRUE, mode ="wb")
Another case: Error in download.file(url, destfile, method, mode = "wb", ...)
> install.packages("quantreg") There is a binary version available but the source version is later: binary source needs_compilation quantreg 5.33 5.34 TRUE Do you want to install from sources the package which needs compilation? y/n: n trying URL 'https://cran.rstudio.com/bin/macosx/el-capitan/contrib/3.4/quantreg_5.33.tgz' Warning in install.packages : cannot open URL 'https://cran.rstudio.com/bin/macosx/el-capitan/contrib/3.4/quantreg_5.33.tgz': HTTP status was '404 Not Found' Error in download.file(url, destfile, method, mode = "wb", ...) : cannot open URL 'https://cran.rstudio.com/bin/macosx/el-capitan/contrib/3.4/quantreg_5.33.tgz' Warning in install.packages : download of package ‘quantreg’ failed
It seems the binary package cannot be found on the mirror. So the solution here is to download the package from the R main server. Note that after I have successfully installed the binary package from the main R server, I remove the package in R and try to install the binary package from rstudio.com server agin and it works this time.
> install.packages("quantreg", repos = "https://cran.r-project.org") trying URL 'https://cran.r-project.org/bin/macosx/el-capitan/contrib/3.4/quantreg_5.34.tgz' Content type 'application/x-gzip' length 1863561 bytes (1.8 MB) ================================================== downloaded 1.8 MB
Another case: Error in download.file() on Windows 7
For some reason, IE 8 cannot interpret https://ftp.ncbi.nlm.nih.gov though it understands ftp://ftp.ncbi.nlm.nih.gov.
This is tested using R 3.4.3.
> download.file("https://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz", "test.soft.gz") trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz' Error in download.file("https://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz", : cannot open URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz' In addition: Warning message: In download.file("https://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz", : InternetOpenUrl failed: 'An error occurred in the secure channel support' > download.file("ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz", "test.soft.gz") trying URL 'ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE7nnn/GSE7848/soft/GSE7848_family.soft.gz' downloaded 9.1 MB
ERROR: failed to lock directory
Follow the suggestion to remove the LOCK file. See the post.
It could happened in calling install.packages(), biocLite() or devtools::install_github(), and so on.
Error in unloadNamespace(package)
> d3heatmap(mtcars, scale = "column", colors = "Blues") Error: 'col_numeric' is not an exported object from 'namespace:scales' > packageVersion("scales") [1] ‘0.2.5’ > library(scales) Error in unloadNamespace(package) : namespace ‘scales’ is imported by ‘ggplot2’ so cannot be unloaded In addition: Warning message: package ‘scales’ was built under R version 3.2.1 Error in library(scales) : Package ‘scales’ version 0.2.4 cannot be unloaded > search() [1] ".GlobalEnv" "package:d3heatmap" "package:ggplot2" [4] "package:microbenchmark" "package:COSMIC.67" "package:BiocInstaller" [7] "package:stats" "package:graphics" "package:grDevices" [10] "package:utils" "package:datasets" "package:methods" [13] "Autoloads" "package:base"
If I open a new R session, the above error will not happen!
The problem occurred because the 'scales' package version required by the d3heatmap package/function is old. See this post. And when I upgraded the 'scales' package, it was locked by the package was imported by the ggplot2 package.
Unload a package
Add unload = TRUE option to unload the namespace. See detach().
require(splines) detach(package:splines, unload=TRUE)
crantastic
https://crantastic.org/. A community site for R packages where you can search for, review and tag CRAN packages.
https://github.com/hadley/crantastic
New R packages as reported by CRANberries
http://blog.revolutionanalytics.com/2015/07/mranspackages-spotlight.html
#---------------------------- # SCRAPE CRANBERRIES FILES TO COUNT NEW PACKAGES AND PLOT # library(ggplot2) # Build a vextor of the directories of interest year <- c("2013","2014","2015") month <- c("01","02","03","04","05","06","07","08","09","10","11","12") span <-c(rep(month,2),month[1:7]) dir <- "http://dirk.eddelbuettel.com/cranberries" url2013 <- file.path(dir,"2013",month) url2014 <- file.path(dir,"2014",month) url2015 <- file.path(dir,"2015",month[1:7]) url <- c(url2013,url2014,url2015) # Read each directory and count the new packages new_p <- vector() for(i in url){ raw.data <- readLines(i) new_p[i] <- length(grep("New package",raw.data,value=TRUE)) } # Plot time <- seq(as.Date("2013-01-01"), as.Date("2015-07-01"), by="months") new_pkgs <- data.frame(time,new_p) ggplot(new_pkgs, aes(time,y=new_p)) + geom_line() + xlab("") + ylab("Number of new packages") + geom_smooth(method='lm') + ggtitle("New R packages as reported by CRANberries")
Top new packages in 2015
- 2015 R packages roundup by CHRISTOPH SAFFERLING
- R trends in 2015 by MAX GORDON
keep.source.pkgs option
State of R packages in your library. The original code formatting and commenting being removed by default, unless one changes some options for installing packages.
- options(keep.source.pkgs = TRUE)
- install.packages("rhub", INSTALL_opts = "--with-keep.source", type = "source")
- R CMD install --with-keep.source
Package installation speed for packages installed with ‘keep.source’ has been improved. 2021-12-2.
Speeding up package installation
- http://blog.jumpingrivers.com/posts/2017/speed_package_installation/
- (Much) Faster Package (Re-)Installation via Caching
- (Much) Faster Package (Re-)Installation via Caching, part 2
An efficient way to install and load R packages
An efficient way to install and load R packages
# Package names packages <- c("ggplot2", "readxl", "dplyr", "tidyr", "ggfortify", "DT", "reshape2", "knitr", "lubridate", "pwr", "psy", "car", "doBy", "imputeMissings", "RcmdrMisc", "questionr", "vcd", "multcomp", "KappaGUI", "rcompanion", "FactoMineR", "factoextra", "corrplot", "ltm", "goeveg", "corrplot", "FSA", "MASS", "scales", "nlme", "psych", "ordinal", "lmtest", "ggpubr", "dslabs", "stringr", "assist", "ggstatsplot", "forcats", "styler", "remedy", "snakecaser", "addinslist", "esquisse", "here", "summarytools", "magrittr", "tidyverse", "funModeling", "pander", "cluster", "abind") # Install packages not yet installed installed_packages <- packages %in% rownames(installed.packages()) if (any(installed_packages == FALSE)) { install.packages(packages[!installed_packages]) } # Packages loading invisible(lapply(packages, library, character.only = TRUE))
Alternatively use the pacman package.
library( , exclude, include.only)
See ?library
library(tidyverse) library(MASS, exclude='select') library(thepackage, include.only="thefunction")
package ‘XXX’ was installed by an R version with different internals
it needs to be reinstalled for use with this R version. The problem seems to be specific to R 3.5.1 in Ubuntu 16.04. I got this message when I try to install the keras and tidyverse packages. The 'XXX' package includes nlme for installing "tidyverse" and Matrix for installing "reticulate". I have already logged in as root. I need to manually install these packages again though it seems I did not see a version change for these packages.
Today it also happened when I tried to install "pec" which broke when it was installing "Hmisc". The error message is "Error : package ‘rpart’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version". I am using R 3.5.2. rpart version is ‘4.1.13’. The solution is I install rpart again (under my account is enough) though rpart does not have a new version. Then I can install "Hmisc".
packrat (reproducible search): project specific package managment
- CRAN & Github
- Bioconductor related issues
- Videos:
- Packrat will not only store all packages, but also all project files.
- Packrat is integrated in RStudio’s user interface. It allows you to share projects along co-workers easily. See Using Packrat with RStudio.
- limitations.
- XML package needs to install some OS library libxml2. So it is not just R package issue.
- Ubuntu goodies
- Git and packrat. The packrat/src directory can be very large. If you don't want them available in your git-repo, you simply add packrat/src/ to the .gitignore. But, this will mean that anyone accessing the git-repo will not have access to the package source code, and the files will be downloaded from CRAN, or from wherever the source line dictates within the packrat.lock file.
- A scenario that we need packrat: suppose we are developing a package in the current R-3.5.X. Our package requires the 'doRNG' package. That package depends the 'rngtools' package. A few months later a new R (3.6.0) was released and a new release (1.3.1.1) of 'rngtools' also requires R-3.6.0. So if we want to install 'doRNG' in R-3.5.x, it will fail with an error: dependency 'rngtools' is not available for package 'doRNG' .
Create a snapshot
- Do we really need to call packrat::snapshot()? The walk through page says it is not needed but the lock file is not updated from my testing.
- I got an error when it is trying to fetch the source code from bioconductor and local repositories: packrat is trying to fetch the source from CRAN in these two packages.
- On normal case, the packrat/packrat.lock file contains two entries in 'Repos' field (line 4).
- The cause of the error is I ran snapshot() after I quitted R and entered again. So the solution is to add bioc and local repositories to options(repos).
- So what is important of running snapshot()?
- Check out the forum.
> dir.create("~/projects/babynames", recu=T) > packrat::init("~/projects/babynames") Initializing packrat project in directory: - "~/projects/babynames" Adding these packages to packrat: _ packrat 0.4.9-3 Fetching sources for packrat (0.4.9-3) ... OK (CRAN current) Snapshot written to '/home/brb/projects/babynames/packrat/packrat.lock' Installing packrat (0.4.9-3) ... OK (built source) Initialization complete! Unloading packages in user library: - packrat Packrat mode on. Using library in directory: - "~/projects/babynames/packrat/lib" > install.packages("reshape2") > packrat::snapshot() > system("tree -L 2 ~/projects/babynames/packrat/") /home/brb/projects/babynames/packrat/ ├── init.R ├── lib │ └── x86_64-pc-linux-gnu ├── lib-ext │ └── x86_64-pc-linux-gnu ├── lib-R # base packages │ └── x86_64-pc-linux-gnu ├── packrat.lock ├── packrat.opts └── src ├── bitops ├── glue ├── magrittr ├── packrat ├── plyr ├── Rcpp ├── reshape2 ├── stringi └── stringr
Restoring snapshots
Suppose a packrat project was created on Ubuntu 16.04 and we now want to repeat the analysis on Ubuntu 18.04. We first copy the whole project directory ('babynames') to Ubuntu 18.04. Then we should delete the library subdirectory ('packrat/lib') which contains binary files (*.so) that do not work on the new OS. After we delete the library subdirectory, start R from the project directory. Now if we run packrat::restore() command, it will re-install all missing libraries. Bingo! NOTE: Maybe I should use packrat::bundle() instead of manually copy the whole project folder.
Note: some OS level libraries (e.g. libXXX-dev) need to be installed manually beforehand in order for the magic to work.
$ rm -rf ~/projects/babynames/packrat/lib $ cd ~/projects/babynames/ $ R > > packrat::status() > remove.packages("plyr") > packrat::status() > packrat::restore()
Workflow
setwd("ProjectDir") packrat::init() packrat::on() # packrat::search_path() install.packages() # For personal packages stored locally packrat::set_opts(local.repos = "~/git/R") packrat::install_local("digest") # dir name of the package library(YourPackageName) # double check all dependent ones have been installed packrat::snapshot() packrat::bundle()
A bundle file (*.tar.gz) will be created under ProjectDir/packrat/src directory. Note this tar.gz file includes the whole project folder.
To unbundle the project in a new R environment/directory:
setwd("NewDirectory") # optional packrat::unbundle(FullPathofBundleTarBall, ".") # this will create 'ProjectDir' # CPU is more important than disk speed # At the end, it will show the project has been unbundled and restored at ... setwd("ProjectDir") packrat::packrat_mode() # on .libPaths() # verify library() # Expect to see packages in our bundle # packrat::on()
Example 1: The above method works for packages from Bioconductor; e.g. S4Vectors which depends on BiocGenerics & BiocVersion only. However, Bioconductor project des not have a snapshot repository like MRAN. So it is difficult to reproduce the environment for an earlier release of Bioconductor.
Example 2: bundle our in-house R package for future reproducibility.
Set Up a Custom CRAN-like Repository
See https://rstudio.github.io/packrat/custom-repos.html. Note the personal repository name ('sushi' in this example) used in "Repository" field of the personal package will be used in <packrat/packrat.lock> file. So as long as we work on the same computer, it is easy to restore a packrat project containing packages coming from personal repository.
- packrat::init()
- packrat::snapshot(), packrat::restore()
- packrat::clean()
- packrat::status()
- packrat::install_local() # http://rstudio.github.io/packrat/limitations.html
- packrat::bundle() # see @28:44 of the video, packrat::unbundle() # see @29:17 of the same video. This will rebuild all packages
- packrat::on(), packrat::off()
- packrat::get_opts()
- packrat::set_opts() # http://rstudio.github.io/packrat/limitations.html
- packrat::opts$local.repos("~/local-cran")
- packrat::opts$external.packages(c("devtools")) # break the isolation
- packrat::extlib()
- packrat::with_extlib()
- packrat::project_dir(), .libPaths()
Warning
- If we download and modify some function definition from a package in CRAN without changing DESCRIPTION file or the package name, the snapshot created using packrat::snapshot() will contain the package source from CRAN instead of local repository. This is because (I guess) the DESCRIPTION file contains a field 'Repository' with the value 'CRAN'.
Docker
- This is a minimal example that installs a single package each from CRAN, bioconductor, and github to a Docker image using packrat.
- All operations are done in the container. So the host OS does not need to have R installed.
- The R script will install packrat in the container. It will also initialize packrat in the working directory and install R packages there. But in the packrat::snapshot() it chooses snapshot.sources = FALSE. The goal is to generate packrat.lock file.
- The first part of generating packrat.lock is not quite right since the file was generated in the container only. We should use -v in the docker run command. The github repository at https://github.com/joelnitta/docker-packrat-example has fixed the problem.
$ git clone https://github.com/joelnitta/docker-packrat-example.git $ cd docker-packrat-example # Step 1: create the 'packrat.lock' file $ nano install_packages.R # note: nano is not available in the rstudio container # need to install additional OS level packages like libcurl # in rocker/rstudio. Probably rocker/tidyverse is better than rstudio # $ docker run -it -e DISABLE_AUTH=true -v $(pwd):/home/rstudio/project rocker/tidyverse:3.6.0 bash # Inside the container now $ cd home/rstudio/project $ time Rscript install_packages.R # generate 'packrat/packrat.lock' $ exit # It took 43 minutes. # Question: is there an easier way to generate packrat.lock without # wasting time to install lots of packages? # Step 2: build the image # Open another terminal/tab $ nano Dockerfile # change rocker image and R version. Make sure these two are the same as # we have used when we created the 'packrat.lock' file $ time docker build . -t mycontainer # It took 45 minutes. $ docker run -it mycontainer R # Step 3: check the packages defined in 'install_packages.R' are installed packageVersion("minimal") packageVersion("biospear")
Questions:
- After running the statement packrat::init(), it will leave a footprint of a hidden file .Rprofile in the current directory. PS: The purpose of .Rprofile file is to direct R to use the private package library (when it is started from the project directory).
#### -- Packrat Autoloader (version 0.5.0) -- #### source("packrat/init.R") #### -- End Packrat Autoloader -- ####
- If the 'packrat' directory was accidentally deleted, next time when you launch R it will show an error message because it cannot find the file.
- The ownership of the 'packrat' directory will be root now. See this Package Management for Reproducible R Code.
- This sophisticated approach does not save the package source code. If a package has been updated and the version we used has been moved to archive in CRAN, what will happen when we try to restore it? So it is probably better to use snapshot.sources = TRUE and run packrat::bundle().
renv: successor to the packrat package
- https://rstudio.github.io/renv/index.html
- release 2019-11-6
- Introduction to renv 2021-01-09
- R renv: How to Manage Dependencies in R Projects Easily 2023-03-22
- The renv::migrate() function makes it possible to migrate projects from Packrat to renv.
- Why Package & Environment Management is Critical for Serious Data Science and a workflow.
- Deploying an R Shiny app on Heroku free tier
- Bioconductor related questions
- Installing packages on a PBS-Pro HPC cluster using renv
- Videos
Compare to packrat:
- Many packages are difficult to build from sources. Your system will need to have a compatible compiler toolchain available. In some cases, R packages may depend on C/C++ features that aren't available in an older system toolchain, especially in some older Linux enterprise environments.
- renv no longer attempts to explicitly download and track R package source tarballs within your project. For packages from local sources, refer this article.
- renv has its discovery machinery to analyze your R code to determine which R packages will be included in the lock file. We can however instead prefer to capture all packages installed into your project library by using renv::settings$snapshot.type("all")
renv package does not have bundle() nor unbundle() function.
# mkdir renvdeseq2 setwd("renvdeseq2") renv::init(bioconductor = TRUE) # attempts to copy and reuse packages # already installed in your R libraries # We'll be asked to restart the R session if we # are not doing this in RStudio. renv::install("BiocManager") # method 1: this will only install packages under the curDir/renv/... folder BiocManager::install("DESeq2") # method 2: this will install packages in ~/.cache/R/renv/renv/... folder # therefore, the library can be reused by other needs. options(repos = BiocManager::repositories()) renv::install("DESeq2") renv::snapshot() # create renv.lock # it seems the lock file "renvdeseq2/renv.lock" does not # save any package info I just installed from Bioconductor # except the renv package. # Read https://rstudio.github.io/renv/articles/faq.html
Find R package dependencies in a project
renv::dependencies()
The following line will make snapshot() to write all packages in renv .cache directory (e.g., ~/.cache/R/renv/cache/v5/R-4.2/x86_64-pc-linux-gnu/) to renv.lock file. Note that the setting is persistent even we restart R!
renv::settings$snapshot.type("all") # default is "implicit" renv::snapshot()
Pass renv.lock to other people and/or clone the project repository
# Make sure the 'renv' package has been installed on the remote computer install.packages("renv") renv::init() # install the packages declared in renv.lock
Use renv::migrate() to port a Packrat project to renv.
Bioconductor
Create an Rmd file and include an R chunk "library(DESeq2)". Then run the following line
renv::init(bioconductor = TRUE)
and it will generate "renv.lock", ".Rprofile" files and "renv" directory.
PS. When we install a fresh R in Ubuntu, we should run "sudo apt install r-base-dev curl libcurl4-openssl-dev libssl-dev libxml2-dev " system packages before we can successfully run "BiocManager::install('DESeq2')".
renv::dependencies()
?dependencies. Find R packages used within a project. dependencies() will crawl files within your project, looking for R files and the packages used within those R files.
df <- renv::dependencies("Some_Dir")
It also search Rmd files from my testing.
renv::restore()
See the output message on here. This is based on renv 0.16.0 (2022-09-29).
install.packages()
If I open a project that loaded an renv environment, then calling "install.packages()" will install new packages into the renv's cache folder (e.g., ~/.cache/R/renv/cache/v5/R-4.2/x86_64-pc-linux-gnu/ in Linux). Note that the version number will be recorded too (e.g., ~/.cache/R/renv/cache/v5/R-4.2/x86_64-pc-linux-gnu/pkgndep/1.2.1 ).
Cache
?path (lined from Installing from Local Sources)
On my Linux system, I see the source packages (*.tar.gz) are stored at
- ~/.local/share/renv/source/bioconductor/ # Store bioconductor packages
- ~/.local/share/renv/source/repository/ # Store CRAN packages
and the binary packages are stored at
- ~/.local/share/renv/cache/ (~/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/)
Note that once I have used renv::init() to restore a project, the related R packages (binary and/or source) will be cached. So next time when we do renv::init(), local R packages can be found.
So how do we manage the packages in cache? For example if we are developing an R package and we made a change but did not change the version number. See Reference.
- renv::purge("MyPackage") # remove binary and source
> root <- renv::paths$root() Welcome to renv! It looks like this is your first time using renv. This is a one-time message, briefly describing some of renv's functionality. renv maintains a local cache of data on the filesystem, located at: - '~/.local/share/renv' This path can be customized: please see the documentation in `?renv::paths`. renv will also write to files within the active project folder, including: - A folder 'renv' in the project directory, and - A lockfile called 'renv.lock' in the project directory. In particular, projects using renv will normally use a private, per-project R library, in which new packages will be installed. This project library is isolated from other R libraries on your system. In addition, renv will update files within your project directory, including: - .gitignore - .Rbuildignore - .Rprofile Please read the introduction vignette with `vignette("renv")` for more information. You can browse the package documentation online at https://rstudio.github.io/renv/. Do you want to proceed? [y/N]:
Private R packages
Local R packages
Deprecated?
- https://rstudio.github.io/renv/articles/local-sources.html
- Since local R packages (no matter it is source or binary) are not part of renv.lock, the original location of these packages are not important when we first install these packages.
- When we try to restore local R packages, we can put these packages' source files into renv/local directory.
# mkdir renvbiotrip setwd("renvbiotrip") renv::init() # we shall restart R according to the instruction # * Initializing project ... # * Discovering package dependencies ... Done! # * Copying packages into the cache ... Done! # The following package(s) will be updated in the lockfile: # CRAN =============================== # - renv [* -> 0.10.0] # * Lockfile written to '/tmp/renvbiotrip/renv.lock'. # * Project '/tmp/renvbiotrip' loaded. [renv 0.10.0] # * renv activated -- please restart the R session. renv::install("~/Downloads/MyPackage_0.1.1.tar.gz") # 1. The above command will take care of the dependence. Cool ! # That is, we don't need to use the remotes package. # 2. The output will show if packages are installed from # 'linked cache' or from source renv::settings$snapshot.type("all") renv::snapshot() # It will give a message some package(s) were installed from an unknown source # renv may be unable to restore these packages in the future.
Since the dependence package versions change from time to time, if we compare the renv.lock file created yesterday it will likely be different from what we created today (package version and hash tag).
Now we are ready to test the restoration.
-
Pass renv.lock and MyPackage_0.1.1.tar.gz to other people (different instruction if we pass the project repository?). Suppose we have copied renv.lock to renvbiotrip/ directory on a new computer.
# mkdir renvbiotrip ## Copy renv.lock to renvbiotrip/ # mkdir renvbiotrip/renv/local ## Copy MyPackage_0.1.1.tar.gz (private packages) to renvbiotrip/renv/local install.packages("renv") renv::restore() # install the packages declared in renv.lock # The output will show if packages are installed from # 'linked cache' or from source library(MyPackage) # verify MyPackage::foo() # test
- We can test renv.lock in a Docker container from another directory to mimic the way of passing the file to other people. For example,
docker run --rm -it -v $(pwd):/home/docker -w /home/docker r-base:4.0.0
- We can create a docker image based on the renv.lock and MyPackage.tar.gz files. See the renvbiotrip repository.
Note that
- If we issue renv::restore() instead of renv::init() on the destination machine, the packages will be installed into the global environment.
- It seems renv::init() is equivalent to renv::activate() AND renv::restore() on the destination machine.
The project library is out of sync with the lockfile
We'll get this message if we start R with a version different from what is in the "renv.lock" file. See install a package on an old version of R.
Docker
- Using renv with Docker. Note that there are two ways for the Docker approach. One way is to include package installation in the Docker file which embeds the packages into the image. A second approach is to add appropriate R packages when the container is run.
- Creating Docker Images with renv (see here for 3 example Registries: Rocker Project/R-Hub/RStudio). Make sure <renv.lock> file and the local R package <MyPackage_0.1.0.tar.gz> are in the current directory.
FROM r-base:4.0.0 RUN R -e 'install.packages("renv")' COPY renv.lock /home/docker RUN mkdir -p /home/docker/renv/local COPY MyPackage_0.1.0.tar.gz /home/renv/local WORKDIR /home/docker RUN R -e 'renv::restore()' CMD ["R"]
- Running Docker Containers with renv
docker build -t renvMyPackage . docker run --rm -it renvMyPackage # OR docker run --rm -it -v $(pwd):/home/docker renvbiotrip
Question: how to update a package within a container? 1. start the container with root and update packages in the container 2. system("su docker") to switch to the user 'docker'. 3. when we run system("su docker"), it will exit R and go to the shell. Run "whoami" to double check the current user and type "R" to enter R again.
Another simple but inferior way to test the docker method is the following: assuming <renv.lock> is saved in the ProjectDir directory and the ProjectDir directory does not have renv nor .Rprofile. The big drawback of this approach is the created renv directory and <.Rprofile> belongs to the user root.
docker run --rm -it -v ProjectDir:/home r-base:4.0.0 install.packages("renv") setwd("/home") renv::init()
- Creating Docker Images with renv (see here for 3 example Registries: Rocker Project/R-Hub/RStudio). Make sure <renv.lock> file and the local R package <MyPackage_0.1.0.tar.gz> are in the current directory.
Github actions
Chapter 5 Testing with a reproducible environment
R package dependencies
Depends, Imports, Suggests, Enhances, LinkingTo
See Writing R Extensions and install.packages().
- Depends: list of package names which this package depends on. Those packages will be attached (so it is better to use Imports instead of Depends as much as you can) before the current package when library or require is called. The ‘Depends’ field can also specify a dependence on a certain version of R.
- Imports: lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached.
- Suggests: lists packages that are not necessarily needed. This includes packages used only in examples, tests or vignettes, and packages loaded in the body of functions.
- Enhances: lists packages “enhanced” by the package at hand, e.g., by providing methods for classes from these packages, or ways to handle objects from these packages.
- LinkingTo: A package that wishes to make use of header files in other packages needs to declare them as a comma-separated list in the field ‘LinkingTo’ in the DESCRIPTION file.
- An example is SingleR that links to the beachmat package in its cpp source code.
- available.packages(); see packageStatus().
- download.packages()
- packageStatus(), update(), upgrade(). packageStatus() will return a list with two components:
- inst - a data frame with columns as the matrix returned by installed.packages plus "Status", a factor with levels c("ok", "upgrade"). Note: the manual does not mention "unavailable" case (but I do get it) in R 3.2.0?
- avail - a data frame with columns as the matrix returned by available.packages plus "Status", a factor with levels c("installed", "not installed", "unavailable"). Note: I don't get the "unavailable" case in R 3.2.0?
> x <- packageStatus() > names(x) [1] "inst" "avail" > dim(x'inst') [1] 225 17 > x'inst'[1:3, ] Package LibPath Version Priority Depends Imports acepack acepack C:/Program Files/R/R-3.1.2/library 1.3-3.3 <NA> <NA> <NA> adabag adabag C:/Program Files/R/R-3.1.2/library 4.0 <NA> rpart, mlbench, caret <NA> affxparser affxparser C:/Program Files/R/R-3.1.2/library 1.38.0 <NA> R (>= 2.6.0) <NA> LinkingTo Suggests Enhances acepack <NA> <NA> <NA> adabag <NA> <NA> <NA> affxparser <NA> R.oo (>= 1.18.0), R.utils (>= 1.32.4),\nAffymetrixDataTestFiles <NA> License License_is_FOSS License_restricts_use OS_type MD5sum NeedsCompilation Built acepack MIT + file LICENSE <NA> <NA> <NA> <NA> yes 3.1.2 adabag GPL (>= 2) <NA> <NA> <NA> <NA> no 3.1.2 affxparser LGPL (>= 2) <NA> <NA> <NA> <NA> <NA> 3.1.1 Status acepack ok adabag ok affxparser unavailable > dim(x'avail') [1] 6538 18 > x'avail'[1:3, ] Package Version Priority Depends Imports LinkingTo A3 A3 0.9.2 <NA> R (>= 2.15.0), xtable, pbapply <NA> <NA> ABCExtremes ABCExtremes 1.0 <NA> SpatialExtremes, combinat <NA> <NA> ABCanalysis ABCanalysis 1.0.1 <NA> R (>= 2.10) Hmisc, plotrix <NA> Suggests Enhances License License_is_FOSS License_restricts_use OS_type Archs A3 randomForest, e1071 <NA> GPL (>= 2) <NA> <NA> <NA> <NA> ABCExtremes <NA> <NA> GPL-2 <NA> <NA> <NA> <NA> ABCanalysis <NA> <NA> GPL-3 <NA> <NA> <NA> <NA> MD5sum NeedsCompilation File Repository Status A3 <NA> <NA> <NA> http://cran.rstudio.com/bin/windows/contrib/3.1 not installed ABCExtremes <NA> <NA> <NA> http://cran.rstudio.com/bin/windows/contrib/3.1 not installed ABCanalysis <NA> <NA> <NA> http://cran.rstudio.com/bin/windows/contrib/3.1 not installed
- packageVersion(), packageDescription()
- install.packages(), remove.packages().
- installed.packages(); see packageStatus().
- update.packages(), old.packages(), new.packages()
- setRepositories()
- contrib.url()
- chooseCRANmirror(), chooseBioCmirror()
- suppressForeignCheck()
tools package
- https://www.rdocumentation.org/packages/tools/versions/3.6.1
- CRAN_package_db() from ?CRANtools. Especially, it gives the Description and Maintainer information not provided by utils::available.packages()
- dependsOnPkgs()
db <- tools::CRAN_package_db() nRcpp <- length(tools::dependsOnPkgs("Rcpp", recursive=FALSE, installed=db) ) nCompiled <- table(db[, "NeedsCompilation"])[["yes"]] propRcpp <- nRcpp / nCompiled * 100
- package.dependencies(), pkgDepends(), etc are deprecated now, mostly in favor of package_dependencies() which is both more flexible and efficient. See R 3.3.0 News. For example, tools::package_dependencies(c("remotes", "devtools"), recursive=TRUE) shows remotes has only a few dependencies while devtools has a lot.
crandep package
https://cran.r-project.org/web/packages/crandep/index.html. Useful to find reverse dependencies. ?get_dep. Consider the abc package:
get_dep("abc", "depends") # abc depends on these packages # note my computer does not have 'abc' installed # from to type reverse # 1 abc abc.data depends FALSE # 2 abc nnet depends FALSE # 3 abc quantreg depends FALSE # 4 abc MASS depends FALSE # 5 abc locfit depends FALSE get_dep("abc", "reverse_depends") # from to type reverse # 1 abc abctools depends TRUE # 2 abc EasyABC depends TRUE x <- get_dep("RcppEigen", c("reverse linking to")) dim(x) # [1] 331 4 head(x, 3) # from to type reverse # 1 RcppEigen abess linking to TRUE # 2 RcppEigen acrt linking to TRUE # 3 RcppEigen ADMMnet linking to TRUE
pkgndep
CRAN & the paper pkgndep: a tool for analyzing dependency heaviness of R packages
remotes
remotes::local_package_deps(dependencies=TRUE) will find and return all dependent packages based on the "DESCRIPTION" file. See an example here.
Bioconductor's pkgDepTools package
The is an example of querying the dependencies of the notorious 'lumi' package which often broke the installation script. I am using R 3.2.0 and Bioconductor 3.1.
The getInstallOrder function is useful to get a list of all (recursive) dependency packages.
source("http://bioconductor.org/biocLite.R") if (!require(pkgDepTools)) { biocLite("pkgDepTools", ask = FALSE) library(pkgDepTools) } MkPlot <- FALSE library(BiocInstaller) biocUrl <- biocinstallRepos()["BioCsoft"] biocDeps <- makeDepGraph(biocUrl, type="source", dosize=FALSE) # pkgDepTools defines its makeDepGraph() PKG <- "lumi" if (MkPlot) { if (!require(Biobase)) { biocLite("Biobase", ask = FALSE) library(Biobase) } if (!require(Rgraphviz)) { biocLite("Rgraphviz", ask = FALSE) library(Rgraphviz) } categoryNodes <- c(PKG, names(acc(biocDeps, PKG)1)) categoryGraph <- subGraph(categoryNodes, biocDeps) nn <- makeNodeAttrs(categoryGraph, shape="ellipse") plot(categoryGraph, nodeAttrs=nn) # Complete but plot is too complicated & font is too small. } system.time(allDeps <- makeDepGraph(biocinstallRepos(), type="source", keep.builtin=TRUE, dosize=FALSE)) # takes a little while # user system elapsed # 175.737 10.994 186.875 # Warning messages: # 1: In .local(from, to, graph) : edges replaced: ‘SNPRelate|gdsfmt’ # 2: In .local(from, to, graph) : # edges replaced: ‘RCurl|methods’, ‘NA|bitops’ # When needed.only=TRUE, only those dependencies not currently installed are included in the list. x1 <- sort(getInstallOrder(PKG, allDeps, needed.only=TRUE)$packages); x1 [1] "affy" "affyio" [3] "annotate" "AnnotationDbi" [5] "base64" "beanplot" [7] "Biobase" "BiocParallel" [9] "biomaRt" "Biostrings" [11] "bitops" "bumphunter" [13] "colorspace" "DBI" [15] "dichromat" "digest" [17] "doRNG" "FDb.InfiniumMethylation.hg19" [19] "foreach" "futile.logger" [21] "futile.options" "genefilter" [23] "GenomeInfoDb" "GenomicAlignments" [25] "GenomicFeatures" "GenomicRanges" [27] "GEOquery" "ggplot2" [29] "gtable" "illuminaio" [31] "IRanges" "iterators" [33] "labeling" "lambda.r" [35] "limma" "locfit" [37] "lumi" "magrittr" [39] "matrixStats" "mclust" [41] "methylumi" "minfi" [43] "multtest" "munsell" [45] "nleqslv" "nor1mix" [47] "org.Hs.eg.db" "pkgmaker" [49] "plyr" "preprocessCore" [51] "proto" "quadprog" [53] "RColorBrewer" "Rcpp" [55] "RCurl" "registry" [57] "reshape" "reshape2" [59] "rngtools" "Rsamtools" [61] "RSQLite" "rtracklayer" [63] "S4Vectors" "scales" [65] "siggenes" "snow" [67] "stringi" "stringr" [69] "TxDb.Hsapiens.UCSC.hg19.knownGene" "XML" [71] "xtable" "XVector" [73] "zlibbioc" # When needed.only=FALSE the complete list of dependencies is given regardless of the set of currently installed packages. x2 <- sort(getInstallOrder(PKG, allDeps, needed.only=FALSE)$packages); x2 [1] "affy" "affyio" "annotate" [4] "AnnotationDbi" "base64" "beanplot" [7] "Biobase" "BiocGenerics" "BiocInstaller" [10] "BiocParallel" "biomaRt" "Biostrings" [13] "bitops" "bumphunter" "codetools" [16] "colorspace" "DBI" "dichromat" [19] "digest" "doRNG" "FDb.InfiniumMethylation.hg19" [22] "foreach" "futile.logger" "futile.options" [25] "genefilter" "GenomeInfoDb" "GenomicAlignments" [28] "GenomicFeatures" "GenomicRanges" "GEOquery" [31] "ggplot2" "graphics" "grDevices" [34] "grid" "gtable" "illuminaio" [37] "IRanges" "iterators" "KernSmooth" [40] "labeling" "lambda.r" "lattice" [43] "limma" "locfit" "lumi" [46] "magrittr" "MASS" "Matrix" [49] "matrixStats" "mclust" "methods" [52] "methylumi" "mgcv" "minfi" [55] "multtest" "munsell" "nleqslv" [58] "nlme" "nor1mix" "org.Hs.eg.db" [61] "parallel" "pkgmaker" "plyr" [64] "preprocessCore" "proto" "quadprog" [67] "RColorBrewer" "Rcpp" "RCurl" [70] "registry" "reshape" "reshape2" [73] "rngtools" "Rsamtools" "RSQLite" [76] "rtracklayer" "S4Vectors" "scales" [79] "siggenes" "snow" "splines" [82] "stats" "stats4" "stringi" [85] "stringr" "survival" "tools" [88] "TxDb.Hsapiens.UCSC.hg19.knownGene" "utils" "XML" [91] "xtable" "XVector" "zlibbioc" > sort(setdiff(x2, x1)) # Not all R's base packages are included; e.g. 'base', 'boot', ... [1] "BiocGenerics" "BiocInstaller" "codetools" "graphics" "grDevices" [6] "grid" "KernSmooth" "lattice" "MASS" "Matrix" [11] "methods" "mgcv" "nlme" "parallel" "splines" [16] "stats" "stats4" "survival" "tools" "utils"
Bioconductor BiocPkgTools
Collection of simple tools for learning about Bioc Packages. Functionality includes access to :
- Download statistics
- General package listing
- Build reports
- Package dependency graphs
- Vignettes
Overview of BiocPkgTools & Dependency graphs
BiocPkgTools: Toolkit for Mining the Bioconductor Package Ecosystem in biorxiv.org.
miniCRAN package
miniCRAN package can be used to identify package dependencies or create a local CRAN repository. It can be used on repositories other than CRAN, such as Bioconductor.
- http://blog.revolutionanalytics.com/2014/07/dependencies-of-popular-r-packages.html
- http://www.r-bloggers.com/introducing-minicran-an-r-package-to-create-a-private-cran-repository/
- http://www.magesblog.com/2014/09/managing-r-package-dependencies.html
- Using miniCRAN in Azure ML
- developing internal CRAN Repositories
Before we go into R, we need to install some packages from Ubuntu terminal. See here.
# Consider glmnet package (today is 4/29/2015) # Version: 2.0-2 # Depends: Matrix (≥ 1.0-6), utils, foreach # Suggests: survival, knitr, lars if (!require("miniCRAN")) { install.packages("miniCRAN", dependencies = TRUE, repos="http://cran.rstudio.com") # include 'igraph' in Suggests. library(miniCRAN) } if (!"igraph" %in% installed.packages()[,1]) install.packages("igraph") tags <- "glmnet" pkgDep(tags, suggests=TRUE, enhances=TRUE) # same as pkgDep(tags) # [1] "glmnet" "Matrix" "foreach" "codetools" "iterators" "lattice" "evaluate" "digest" # [9] "formatR" "highr" "markdown" "stringr" "yaml" "mime" "survival" "knitr" # [17] "lars" dg <- makeDepGraph(tags, suggests=TRUE, enhances=TRUE) # miniCRAN defines its makeDepGraph() plot(dg, legendPosition = c(-1, 1), vertex.size=20)
We can also display the dependence for a package from the Bioconductor repository.
tags <- "DESeq2" # Depends S4Vectors, IRanges, GenomicRanges, Rcpp (>= 0.10.1), RcppArmadillo (>= 0.3.4.4) # Imports BiocGenerics(>= 0.7.5), Biobase, BiocParallel, genefilter, methods, locfit, geneplotter, ggplot2, Hmisc # Suggests RUnit, gplots, knitr, RColorBrewer, BiocStyle, airway,\npasilla (>= 0.2.10), DESeq, vsn # LinkingTo Rcpp, RcppArmadillo index <- function(url, type="source", filters=NULL, head=5, cols=c("Package", "Version")){ contribUrl <- contrib.url(url, type=type) available.packages(contribUrl, type=type, filters=filters) } bioc <- local({ env <- new.env() on.exit(rm(env)) evalq(source("http://bioconductor.org/biocLite.R", local=TRUE), env) biocinstallRepos() # return URLs }) bioc # BioCsoft # "http://bioconductor.org/packages/3.0/bioc" # BioCann # "http://bioconductor.org/packages/3.0/data/annotation" # BioCexp # "http://bioconductor.org/packages/3.0/data/experiment" # BioCextra # "http://bioconductor.org/packages/3.0/extra" # CRAN # "http://cran.fhcrc.org" # CRANextra # "http://www.stats.ox.ac.uk/pub/RWin" str(index(bioc["BioCsoft"])) # similar to cranJuly2014 object system.time(dg <- makeDepGraph(tags, suggests=TRUE, enhances=TRUE, availPkgs = index(bioc["BioCsoft"]))) # Very quick! plot(dg, legendPosition = c(-1, 1), vertex.size=20)
The dependencies of GenomicFeature and GenomicAlignments are more complicated. So we turn the 'suggests' option to FALSE.
tags <- "GenomicAlignments" dg <- makeDepGraph(tags, suggests=FALSE, enhances=FALSE, availPkgs = index(bioc["BioCsoft"])) plot(dg, legendPosition = c(-1, 1), vertex.size=20)
Github repository
cranlike
https://github.com/r-hub/cranlike
cranlike keeps the package data in a SQLite database, in addition to the PACKAGES* files. This database is the canonical source of the package data. It can be updated quickly, to add and remove packages. The PACKAGES* files are generated from the database.
MRAN (CRAN only) & checkpoint package
- http://blog.revolutionanalytics.com/2014/10/explore-r-package-connections-at-mran.html
- Reproducible Work in R
According to the snapsot list here, the oldest version is 2014-08-18 which corresponds to R 3.1.0.
checkpoint package
library(checkpoint) checkpoint("2015-03-31")
Note the Bioconductor packages have no similar solution.
groundhog package
- groundhog: Version-Control for CRAN, GitHub, and GitLab Packages (Bioconductor?)
- It seems groundhog is like checkpoint but groundhog does not depend on MRAN and it will figure out the package dependencies by itself.
- MRAN is getting shutdown - what else is there for reproducibility with R, or why reproducibility is on a continuum? The author provides an example where we can integrate "groundhog" in the Dockerfile for reproducibility. Pay attention to the sentence (appear 2 times) “why use Docker at all? Since it’s easy to install older versions of R on Windows and macOS, wouldn’t an renv.lock file suffice? Or even just {groundhog} which is arguably even easier to use?”
From my simple testing, it seems to work. Note I use the docker's R since that's the easiest way to use an old version of R (e.g. for some old R script). Note that it will create a new folder R_groundhog folder in the working directory (see the message below).
$ docker run --rm -it -v $(pwd):/home/docker \ -w /home/docker -u docker r-base:4.0.2 R > install.packages("groundhog") > library(groundhog") groundhog needs authorization to save files to '/home/docker/R_groundhog' Enter 'OK' to provide authorization OK The groundhog folder path is now: /home/docker/R_groundhog/groundhog_library/ Loaded 'groundhog' (version:2.1.0) using R-4.0.2 Tips and troubleshooting: https://groundhogR.com > groundhog.library(" library(ggplot2)", "2020-10-10") > library() # Packages in library ‘/usr/local/lib/R/site-library’: > packageVersion("ggplot2") [1] ‘3.3.2’ # the latest version is 3.4.0 on R 4.2.2 > library(ggplot2)
rang
rang: make ancient R code run again
cranly
sessioninfo
tmp = session_info("sessioninfo") dim(tmp$packages) # [1] 7 11 tmp = session_info("tidyverse") dim(tmp$packages) # [1] 95 11
Reverse dependence
- http://romainfrancois.blog.free.fr/index.php?post/2011/10/30/Rcpp-reverse-dependency-graph
- Checking reverse dependencies: the tiny way
Install packages offline
http://www.mango-solutions.com/wp/2017/05/installing-packages-without-internet/
Install a packages locally and its dependencies
It's impossible to install the dependencies if you want to install a package locally. See Windows-GUI: "Install Packages from local zip files" and dependencies
A minimal R package (for testing purpose)
https://github.com/joelnitta/minimal. Question: is there a one from CRAN?
An R package that does not require others during install.packages()
Create a new R package, namespace, documentation
- http://cran.r-project.org/doc/contrib/Leisch-CreatingPackages.pdf (highly recommend)
- https://stat.ethz.ch/pipermail/r-devel/2013-July/066975.html
- Benefit of import in a namespace
- This youtube video from Tyler Rinker teaches how to use RStudio to develop an R package and also use Git to do version control. Very useful!
- Developing R packages by Jeff Leek in Johns Hopkins University.
- R packages book by Hadley Wickham.
- R package primer a minimal tutorial from Karl Broman.
- How to make and share an R package in 3 steps (6/14/2017)
- Package Development tutorial for useR! 2019 Toulouse
- rOpenSci Packages: Development, Maintenance, and Peer Review
- Package development from "Modern R with the tidyverse"
- How to write your own R package (video, 10 minutes)
- Building R packages with devtools and usethis | RStudio (video, thomas mock)
- The Package: learning how to build an R package
- How to write your own R package and publish it on CRAN
Package structure
http://r-pkgs.had.co.nz/package.html. On Linux/macOS, use tree -d DIRNAME to show the directories only. At a minimum, we will have R and man directories.
- ChangeLog
- DESCRIPTION
- MD5
- NAMESPACE
- R/
- zzz.R
- build/
- Package.pdf (eg dplyr)
- vignette.rds
- data/
- demo/
- inst/
- extdata/
- doc/
- FileName.R
- FileName.Rmd
- FileName.html
- include/
- othersYouInclude/
- tinytest/
- CITATION
- man/
- figures/
- src/
- tests/
- testthat
- vignettes/
NAMESPACE and DESCRIPTION
Namespace dependencies not required. If you use import or importFrom in your NAMESPACE file, you should have an entry for that package in the Imports section of your DESCRIPTION file (unless there is a reason that you need to use Depends in which case the package should have an entry in Depends, and not Imports).
license
- What kind of license is the best license for an R package?
- https://cran.r-project.org/web/licenses/
- https://r-pkgs.org/description.html#description-license
- Some Notes on GNU Licenses in R Packages
Install software for PDF output
- Windows: Miktex and pandoc (Create HTML or PDF Files with R, Knitr, MiKTeX, and Pandoc)
- Unix: TeX Live (sudo apt-get install texlive-full) and pandoc
- Mac: MacTex
Windows: Rtools
Installing RTools for Compiled Code via Rcpp. Just remember to check the option to include some paths in the PATH environment variable.
R CMD
R Installation and Administration
- R CMD build XXX. Note this will not create pdf files for vignettes. The output is a tarball.
- R CMD INSTALL
- R CMD REMOVE
- R CMD SHLIB files. For example, "Rcmd shlib *.f *.c -o surv.dll" on Windows.
- R CMD make
- R CMD check XXX. Useful if we want to create reference manual (PDF file). See R create reference manual with R CMD check.
- R CMD javareconf
usethis package
- https://usethis.r-lib.org/. usethis is a workflow package: it automates repetitive tasks that arise during project setup and development, both for R packages and non-package projects.
- R Package Development 1: Where to Start by John Muschelli. R Package Development playlist.
- create_package(path) with the package name of your choice.
- Sharing with others and helping yourself (slides, use right click menu to save as an HTML file)
- usethis 1.5.0
- usethis 1.6.0
- usethis 2.0.0
Github Actions
To use the Github Actions for continuous integration/CI checks,
- I first follow this to run usethis::use_github_action_check_release(). Once I commit and push the files to Github, Github Actions are kicked off. The R console also tell me to copy and paste a line to add a workflow's badge to README.md.
- Then I modify the yaml file to become this to run a standard check. This took 3m24s to run.
- Then I further delete the 'devel' line (line 23) to reduce one more platform to check. This took 3m to check. My example is on Github (rtoy).
I also try to follow Github Actions with R and create a pkgdown/package documentation web page.
- usethis::use_github_actions("pkgdown") Now go to Github repo's Settings -> Options and scroll down until you see Github Pages. For Source, the page site should be set to being built from the root folder of the gh-pages.
- I have used Jekyll to create a gh-page. I don't need to delete anything for this new gh-pages. I just need to go to the repository setting and (scroll down until we see Github Pages) change the Source of Github Pages to 'gh-pages branch' from 'master branch'.
- The files on the gh-pages branch are generated by Github Actions; these files are not available on my local machine. My location machine only has .github/workflows/pkgdown.yaml file.
Question:
- The workflow file specifies R version and OS platform.
- Right now the workflow file (like pkgdown) is using "r-lib/actions/setup-r@master" that has an "action.yml" file. The r-version is '3.x' only. What about if R 4.0.0 is released?
Packages, webpages and Github
Tutorials by Lisa DeBruine
biocthis
R package depends vs imports
- http://stackoverflow.com/questions/8637993/better-explanation-of-when-to-use-imports-depends
- http://stackoverflow.com/questions/9893791/imports-and-depends
- https://stat.ethz.ch/pipermail/r-devel/2013-August/067082.html
In the namespace era Depends is never really needed. All modern packages have no technical need for Depends anymore. Loosely speaking the only purpose of Depends today is to expose other package's functions to the user without re-exporting them.
load = functions exported in myPkg are available to interested parties as myPkg::foo or via direct imports - essentially this means the package can now be used
attach = the namespace (and thus all exported functions) is attached to the search path - the only effect is that you have now added the exported functions to the global pool of functions - sort of like dumping them in the workspace (for all practical purposes, not technically)
import a function into a package = make sure that this function works in my package regardless of the search path (so I can write fn1 instead of pkg1::fn1 and still know it will come from pkg1 and not someone's workspace or other package that chose the same name)
The distinction is between "loading" and "attaching" a package. Loading it (which would be done if you had MASS::loglm, or imported it) guarantees that the package is initialized and in memory, but doesn't make it visible to the user without the explicit MASS:: prefix. Attaching it first loads it, then modifies the user's search list so the user can see it.
Loading is less intrusive, so it's preferred over attaching. Both library() and require() would attach it.
import() and importFrom()
If our package depends on a package, we need to made some changes. Below we assume the package glmnet in our new package.
- DESCRIPTION: Imports: glmnet
- NAMESPACE: either import(glmnet) to import all functions from glmnet or importFrom(glmnet, cv.glmnet) to import 'cv.glmnet' only
- hello.R: nothing needs to be added
For more resource, see
R package suggests
stringr has suggested htmlwidgets. An error will come out if the suggested packages are not available.
> library(stringr) > str_view(c("abc", "a.c", "bef"), "a\\.c") Error in loadNamespace(name) : there is no package called ‘htmlwidgets’
Useful functions for accessing files in packages
- system.file()
- path.package() & find.package(). Note that path.package() requires the package to be loaded.
- file.path(). This is not related to packages.
- normalizePath(). This is not related to packages.
> system.file(package = "batr") [1] "f:/batr" > system.file("extdata", "logo.png", package = "cowplot") # Mac [1] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/cowplot/extdata/logo.png" > path.package("batr") [1] "f:\\batr" > path.package("ggplot2") # Mac [1] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/ggplot2" # sometimes it returns the forward slash format for some reason; C:/Program Files/R/R-3.4.0/library/batr # so it is best to add normalizePath(). > normalizePath(path.package("batr")) > file.path("f:", "git", "surveyor") [1] "f:/git/surveyor"
Internal functions
RStudio shortcuts
available package
available. Check if a given package name is available to use. It checks the name's validity. Checks if it is used on 'GitHub', 'CRAN' and 'Bioconductor'.
Create an R package
Your first R package in 1 hour
Using usethis
New R Package 'foo' -- Updated
- setwd()
- usethis::create_package("foo")
- usethis::use_git(); usethis_github()
- usethis::use_mit_license("Your name")
- usethis::use_r("foo_function")
- usethis::use_package("dplyr") # specify import dependency, e.g. dplyr package
- usethis::use_testthat()
- usethis::use_test("firsttest")
- setwd("..")
- roxygen2::roxygenise()
Using devtools and roxygen2
- https://www.r-project.org/nosvn/pandoc/devtools.html
- A useful post by Jacob Montgomery. Watch the youtube video there. The process requires 3 components: RStudio software, devtools and roxygen2 (creating documentation from R code) packages.
- MAKING PACKAGES IN R USING DEVTOOLS
- R code workflow from Hadley Wickham.
- Workflow automation tools for package developers
- RStudio:addins part 2 - roxygen documentation formatting made easy
- Inserting a skeleton - Do this by placing your cursor anywhere in the function you want to document and click Code Tools -> Insert Roxygen Skeleton
- Instructions for Creating Your Own R Package. It includes creating a R package with functions written in C++ via Rcpp helper function.
- devtools cheatsheet (2 pages)
- My first R package Part 1, Part 2, Part 3. It uses 3 packages: usethis, roxygen2 and devtools.
- If you love devtools::load_all(), pkgload is where the magic actually happens. pkgload package.
How to use devtools::load_all("FolderName"). load_all() loads any modified R files, and recompile and reload any modified C or Fortran files.
# Step 1 library(devtools) # Step 2 dir.create(file.path("MyCode", "R"), recursive = TRUE) cat("foo=function(x){x*2}", file = file.path("MyCode", "R", "foo.R")) write.dcf(list(Package = "MyCode", Title = "My Code for this project", Description = "To tackle this problem", Version = "0.0", License = "For my eyes only", Author = "First Last <[email protected]>", Maintainer = "First Last <[email protected]>"), file = file.path("MyCode", "DESCRIPTION")) # OR # create("path/to/package/pkgname") # create() will create R/ directory, DESCRIPTION and NAMESPACE files. # Step 3 (C/Fortran code, optional) dir.create(file.path("MyCode", "src")) cat("void cfoo(double *a, double *b, double *c){*c=*a+*b;}\n", file = file.path("MyCode", "src", "cfoo.c")) cat("useDynLib(MyCode)\n", file = file.path("MyCode", "NAMESPACE")) # Step 4 load_all("MyCode") # Step 5 # Modify R/C/Fortran code and run load_all("MyCode") # Step 6 (Automatically generate the documentation from R source code, # in the man/ folder and export the function in NAMESPACE file. # optional, repeat if any function's help has changed) document("MyCode") # Step 7 (check the package, optional) check("MyCode") # Step 8 (Deployment, create a tarball, # optional, repeat if necessary) build("MyCode") # Step 9 (Install the package, optional) install("MyCode")
Note:
- load_all("FolderName") will make the FolderName to become like a package to be loaded into the current R session so the 2nd item returned from search() will be "package:FolderName". However, the FolderName does not exist under Program Files/R/R-X.Y.Z/library nor Documents/R/win-library/X.Y/ (Windows OS).
- build("FolderName") will create a tarball in the current directory. User can install the new package for example using Packages -> Install packages from local files on Windows OS. This will build/run vignettes so it may take some time. The tarball will contain a build folder containing 'vignette.rds' file. It'll also create a new folder inst/doc containing 3 files (MyPkg.html, MyPkg.Rmd and MyPkg.R). The vignettes may contain a new folder MyPkg_cache if we use chunk = TRUE option in Rmd file. Note install.packages() will not run the R code in vignettes.
- For the simplest R package, the source code only contains a file <DESCRIPTION> and a folder <R> with individual R files in the text format.
Using RStudio
- Building, Testing, and Distributing Packages from RStudio.
- Youtube
- How to write your own R package from IDG TECHtalk
- R Package Development playlist by John Muschelli.
- https://youtu.be/Mwe64ziVQXA which does not use RStudio to create a new package
- How to Create and Distribute an R Package.
- The goal of the article to set up an easily installable R package on Github for others to use via remotes::install_github().
- The main tools required are RStudio along with the packages roxygen2 and usethis.
- How to take care of Bioconductor and Github Dependencies
- How to publish your package to Github
- Building a Corporate R Package for Pleasure and Profit
- Toy example*
- In RStudio, click "File" - "New Project" - "New Directory" - "R package". Package name = "a1" (this naming will guarantee this package will be shown on top of all R packages in RStudio). Press 'Create Project'. This new project folder will have necessary files for an R package including DESCRIPTION, NAMESPACE, R/hello.R, man/hello.Rd. The "hello.R" file will be opened in RStudio automatically.
- Create a new R file under a1/R folder. An example of this R file <add.R> containing roxygen comments can be found under here. Pressing Ctrl/Cmd + Shift + D (or running devtools::document()) will generate a man/add.Rd.
- In R, type usethis::use_vignette("my-vignette") to create a new vignette. The new vignette "my-vignette.Rmd" will be saved under "a1/vignettes" subfolder. We can modify the Rmd file as we need.
- In RStudio, click "Build" - "Build Source Package". You will see some messages on the "Build" tab of the top-right panel. Eventually, a tarball "/home/$USERNAME/a1_0.1.0.tar.gz" is created (I create the project under /home/$USERNAME directory).
- We can install the package in R install.packages("~/a1_0.1.0.tar.gz", repos= NULL, type = "source") if we have already created the tarball. Another method is to use RStudio "Build" - "Install and Restart" (Ctrl + Shift + B).
- In RStudio, type help(package = "a1") or click "Packages" tab on the bottom-right panel and click "a1" package. It will show a line "User guides, package vignettes and other documentation". The vignette we just created will be available in HTML, source and R code format.
Using RStudio.cloud
There is a problem. We can use devtools::create("/cloud/project") to create a new package. When it builds the source package, the package file will be located in the root directory of the package. However, if we use a local RStudio to create a source package, the source package will be located in the upper directory.
Binary packages
- No .R files in the R/ directory. There are 3 files that store the parsed functions in an efficient file format. This is the result of loading all the R code and then saving the functions with save().
- A Meta/ directory contains a number of Rds files. These files contain cached metadata about the package, like what topics the help files cover and parsed version of the DESCRIPTION file.
- An html/ directory.
- libs/ directory if you have any code in the src/' directory
- The contents of inst/ are moved to the top-level directory.
Building the tarball
- No matter we uses devtools::build() or the terminal R CMD build MyPkg it will uses run the R code in vignette. Be cautious on the extra time and storage the process incurred.
- If 'cache = TRUE' is used in vignettes, it will create a new subfolder called MyuPkg_cache under the vignettes folder. This takes a lot of space (eg 1GB in some case).
Building the binary
R CMD INSTALL --build MyPkg.tar.gz # OR R CMD INSTALL --build Full_Path_Of_MyPkg
The binary (on Windows) can be installed by install.packages("Mypkg.zip",repos=NULL)
If the installation is successful, it will overwrite any existing installation of the same package. To prevent changes to the present working installation or to provide an install location with write access, create a suitably located directory with write access and use the -l option to build the package in the chosen location.
R CMD INSTALL -l location --build pkg
R folder
- https://r-pkgs.org/r.html
- zzz.R. .onLoad() function.
- R: How to run some code on load of package?
See an example from DuoClustering2018.
#' @importFrom utils read.csv .onLoad <- function(libname, pkgname) { fl <- system.file("extdata", "metadata.csv", package = "DuoClustering2018") titles <- utils::read.csv(fl, stringsAsFactors = FALSE)$Title ExperimentHub::createHubAccessors(pkgname, titles) }
Note that the environment of a function from the DuoClustering2018 package is not the package name.
environment(clustering_summary_filteredExpr10_TrapnellTCC_v2) <environment: 0x7fe01dbe7dd0>
Q: where is the definition of DuoClustering2018::clustering_summary_filteredExpr10_TrapnellTCC_v2()? A: createHubAccessors() - Creating A Hub Package: ExperimentHub or AnnotationHub.
data
http://r-pkgs.had.co.nz/data.html
Three ways to include data in your package.
- If you want to store binary data and make it available to the user, put it in data/. This is the best place to put example datasets.
- If you want to store parsed data, but not make it available to the user, put it in R/sysdata.rda. This is the best place to put data that your functions need.
- If you want to store raw data, put it in inst/extdata. See External data. An example from tximportData package.
# grep -r extdata /home/brb/R/x86_64-pc-linux-gnu-library/4.0 logo_file <- system.file("extdata", "logo.png", package = "cowplot") orgDBLoc = system.file("extdata", "org.Hs.eg.sqlite", package="org.Hs.eg.db") # grep -r readRDS /home/brb/R/x86_64-pc-linux-gnu-library/4.0 patient.data <- readRDS("assets/coxnet.RDS")
How to distribute data with your R package
Rd file
Vignette
- How to Create Vignettes That Really Pull a Room Together
- See examples from packages like magrittr (Rmd, jpg) or tidyr (Rmd, csv) or rvest (Rmd, png) or DESeq2.
- Many vignette uses output: rmarkdown::html_vignette which will expand the width to the page. This one uses output: BiocStyle::html_document which will have a narrow width.
- Add a static pdf vignette to an R package
- devtools::build_vignettes(). The files are copied in the 'doc' directory and an vignette index is created in 'Meta/vignette.rds', as they would be in a built package.
- build_vignette can't find functions
Long execution for R code in vignette
- Caching chunks in RStudio does not work
- caching doesn't seem to work here (i.e. when used in a document that is a vignette)
- How to Add a Vignette to a Package in RStudio
- Optimal workflows for package vignettes
NEWS
Why and how maintain a NEWS file for your R package?
README.Rmd & README.md files
See Releasing a package from R packages by Hadley Wickham.
How to convert .Rmd into .md in R studio?
Example: ggplot2 repository at Github
It seems RStudio cannot create TOC for *.md files. glmnet package creates TOC of its vignette by itself. Visual Studio Code has an extension to do that.
badge
badger: Badge for R Package
tests folder and testthat package
- testthat package.
- testthat: Get Started with Testing 2011
- Automated testing with 'testthat' in practice
- Lots of packages suggest testthat; for example princurve, glmnetUtils.
- How to run a package's testthat tests
- Building, Testing, and Distributing Packages from RStudio
- Run Tests button. See RStudio release notes from RStudio v1.2 - April 8, 2019.
- Working with the RStudio IDE
- Unit Tests in R 2019
- The result of dput can be used in expect_equal().
- A small gotcha when comparing lists using testthat
.Rbuildignore
Non-standard files/directories, Rbuildignore and inst
URL checker
What is a library?
A library is simply a directory containing installed packages.
You can use .libPaths() to see which libraries are currently active.
.libPaths() lapply(.libPaths(), dir)
Object names
- Variable and function names should be lower case.
- Use an underscore (_) to separate words within a name (reserve . for S3 methods).
- Camel case is a legitimate alternative, but be consistent! For example, preProcess(), twoClassData, createDataPartition(), trainingRows, trainPredictors, testPredictors, trainClasses, testClasses have been used in Applied Predictive Modeling by Kuhn & Johnson.
- Generally, variable names should be nouns and function names should be verb.
Spacing
- Add a space around the operators +, -, \ and *.
- Include a space around the assignment operators, <- and =.
- Add a space around any comparison operators such as == and <.
Indentation
- Use two spaces to indent code.
- Never mix tabs and spaces.
- RStudio can automatically convert the tab character to spaces (see Tools -> Global options -> Code).
\dontrun{}
- Writing R Extensions > Documenting functions and search for dontrun
- How to not run an example using roxygen2?
- What does "not run" mean in R help pages? See ?example .
formatR and lintr package
Use formatR package to clean up poorly formatted code
install.packages("formatR") formatR::tidy_dir("R")
Another way is to use the lintr package (lint).
install.packages("lintr") lintr:::lin_package()
Rcpp
Thirteen Simple Steps for Creating An R Package with an External C++ Library
C library
Using R — Packaging a C library in 15 minutes
Data package
Chapter 12 Create a data package from rstudio4edu
Minimal R package for submission
https://stat.ethz.ch/pipermail/r-devel/2013-August/067257.html and CRAN Repository Policy.
Create R Windows Binary on non-Windows OS
- Create R Windows Binary from .tar.gz linux
- win-builder from r-project.org
r-hub/rhub package: the R package builder service
https://github.com/r-hub/proposal
- rhub 1.1.1 is on CRAN! 2019/4/8
- R package developers, why should you care about R-hub? 2019/3/26
- Local Linux checks with Docker
- https://www.rstudio.com/resources/videos/r-hub-overview/
- http://blog.revolutionanalytics.com/2016/10/r-hub-public-beta.html
- Setting up continuous multi-platform R package building, checking and testing with R-Hub, Docker and GitLab CI/CD for free, with a working example 2019/4/27
- How to keep up with CRAN policies and processes? 2019/5/29
- R-hub usage in a few figures
# Today 1/1/2020 $ git clone https://github.com/arraytools/rtoy.git $ rm -rf rtoy/.git $ rm rtoy/.gitignore rtoy/_config.yml $ R > install.packages("devtools", repos = "https://cran.rstudio.com") > install.packages("rhub") > devtools::install_github("r-hub/sysreqs") #needed before calling local_check_linux() > pkg_path <- "~/Downloads/rtoy" > chk <- local_check_linux(pkg_path, image = "rhub/debian-gcc-release") ─ Building package Container name: 6dca434d-84f9-42e2-ab83-b8c364594476-2 It will _not_ be removed after the check. R-hub Linux builder script v0.10.0 (c) R Consortium, 2018-2019 Package: /tmp/RtmpviOCT1/file470972a3c4ab/rtoy_0.1.0.tar.gz Docker image: rhub/debian-gcc-release Env vars: R CMD check arguments: Unable to find image 'rhub/debian-gcc-release:latest' locally latest: Pulling from rhub/debian-gcc-release Digest: sha256:a9e01ca57bfd44f20eb6719f0bfecdd8cf0f59610984342598a53f11555b515d Status: Downloaded newer image for rhub/debian-gcc-release:latest Sysreqs platform: linux-x86_64-debian-gcc No system requirements >>>>>==================== Installing system requirements 8fa4f66e41954b4ed1112eb72e76683c28345863fcc7e260edac6a9a30387fed >>>>>==================== Starting Docker container 2dfae75c9de95b598ff8d9cd19d3cfc7ffe7f16edbc3d4f81c896e4e6b956ebd ls: cannot access '/opt/R-*': No such file or directory > source('https://bioconductor.org/biocLite.R') Error: With R version 3.5 or greater, install Bioconductor packages using BiocManager; see https://bioconductor.org/install Execution halted Error in run(bash, c(file.path(wd, "rhub-linux.sh"), args), echo = TRUE, : System command error
Moreover, it create a new Docker image and a new Docker container. We need to manually clean them:-(
Running a check on its own (remote) server works
> check_on_linux("rtoy") # will run remotely. # We need to verify the email and enter a token. # We will get a report and a full build log. # The report includes both the linux command and the log from the server.
Now I go back to the original method.
$ R > install.packages("tinytex") > tinytex::install_tinytex() > q() $ exit $ sudo apt-get install texinfo $ R CMD build rtoy $ R CMD check rtoy_0.1.0.tar.gz # Install pandoc $ wget https://github.com/jgm/pandoc/releases/download/2.9.1/pandoc-2.9.1-1-amd64.deb $ sudo dpkg -i pandoc-2.9.1-1-amd64.deb $ R CMD check --as-cran rtoy_0.1.0.tar.gz # Ignore a 'Note' https://stackoverflow.com/a/23831508
rcmdcheck
Run R CMD check from R and Capture Results
The rcmdcheck package was used by Github Actions for R language from r-lib/R infrastructure.
CRAN check API
- Overview of the CRAN checks API 2019/6/10
- update 2019/10/9
Continuous Integration
Travis-CI (Linux, Mac)
- A Beginner's Guide to Travis-CI
- http://johnmuschelli.com/neuroc/getting_ready_for_submission/index.html#61_travis
Continuous Integration: Appveyor (Windows)
- Appveyor is a continuous integration service that builds projects on Windows machines.
- http://johnmuschelli.com/neuroc/getting_ready_for_submission/index.html#62_appveyor
Github Actions
- See usethis package.
- Automating R package checks across platforms with GitHub Actions and Docker in a portable way. No need to use "usethis" package.
precommit
Submit packages to cran
- https://cran.r-project.org/submit.html
- http://f.briatte.org/r/submitting-packages-to-cran
- Submitting your first package to CRAN, my experience
- Get your R package on CRAN in 10 steps June 2018
- Preparing Your Package for for Submission by John Muschelli
- https://builder.r-hub.io/. See here.
- The most annoying warning for CRAN submission
- How to write your own R package and publish it on CRAN devtools::release(), July 2020
- A NOTE on URL checks of your R package
- Releasing a package from the book "R packages" by Hadley Wickham.
- I got my first package onto CRAN, and YOU CAN TOO
- Submitting vroom to CRAN, LIVE!
Windows
Everything you should know about WinBuilder
Other tips/advice
- Top 10 tips to make your R package even more awesome
- GoodPractice Advice on R Package Building.
C/Fortran
Rmath.h
For example pnorm5() was used by survS.h by survival package (old version) .
Packages includes Fortran
- survC1
- nlme
- glmnet
- bsamGP (FORTRAN 90). This is searched by using https://www.r-pkg.org/. I am not able to get what I need using the website.
On mac, gfortran (6.1) can be downloaded from CRAN. It will be installed onto /usr/local/gfortran. Note that the binary will not be present in PATH. So we need to run the following command to make gfortran avaiialbe.
sudo ln -s /usr/local/gfortran/bin/gfortran /usr/local/bin/gfortran
A useful tool to find R packages containing Fortran code is pkgsearch package. Note
- The result is not a comprehensive list of packages containing Fortran code.
- It seems the result is the same as I got from https://www.r-pkg.org
> pkg_search("Fortran") - "Fortran" ------------------------------------ 69 packages in 0.009 seconds - # package version by @ title 1 100 covr 3.3.2 Jim Hester 12d Test Coverage for Packages 2 91 inline 0.3.15 Dirk Eddelbuettel 1y Functions to Inline C, ... 3 43 randomForest 4.6.14 Andy Liaw 2y Breiman and Cutler's Ra... 4 39 deSolve 1.24 Thomas Petzoldt 4M Solvers for Initial Val... 5 27 mnormt 1.5.5 Adelchi Azzalini 3y The Multivariate Normal... 6 26 minqa 1.2.4 Katharine M. Mullen 5y Derivative-free optimiz... 7 24 rgcvpack 0.1.4 Xianhong Xie 6y R Interface for GCVPACK... 8 22 leaps 3.0 Thomas Lumley 3y Regression Subset Selec... 9 21 akima 0.6.2 Albrecht Gebhardt 3y Interpolation of Irregu... 10 20 rootSolve 1.7 Karline Soetaert 3y Nonlinear Root Finding,... > more() - "Fortran" ------------------------------------ 69 packages in 0.009 seconds - # package version by @ title 11 15 BB 2019.10.1 Paul Gilbert 11d Solving and Optimizing L... 12 15 limSolve 1.5.5.3 Karline Soetaert 2y Solving Linear Inverse M... 13 14 insideRODE 2.0 YUZHUO PAN 7y insideRODE includes buil... 14 13 earth 5.1.1 Stephen Milborrow 7M Multivariate Adaptive Re... 15 13 cluster 2.1.0 Martin Maechler 4M "Finding Groups in Data"... 16 13 spam 2.3.0.2 ORPHANED 11h SPArse Matrix 17 12 diptest 0.75.7 Martin Maechler 4y Hartigan's Dip Test Stat... 18 10 pbivnorm 0.6.0 Brenton Kenkel 5y Vectorized Bivariate Nor... 19 7 optmatch 0.9.12 Mark M. Fredrickson 17d Functions for Optimal Ma... 20 7 lsei 1.2.0 Yong Wang 2y Solving Least Squares or...
Another way is to run rsync to download src/contrib directory from CRAN and then use grep to find these packages. Note: the source packages takes about 8GB space (2019-10-28).
mkdir ~/Downloads/cran rsync -avz --delete cran.r-project.org::CRAN/src/contrib/*.tar.gz ~/Downloads/cran/ rsync -avz --delete cran.r-project.org::CRAN/src/contrib/PACKAGES ~/Downloads/cran/ rsync -avz --delete cran.r-project.org::CRAN/src/contrib/PACKAGES.gz ~/Downloads/cran/ cd ~/Downloads/cran find . -xtype l -delete # remove broken symbolic links touch tmp for f in *.gz; do tar -tzvf $f | grep -E "(\.f|\.f90|\.f95)$" |& tee -a tmp done
to check if the tarball contains Fortran 77/90 code.
Now to see all packages names we can process it in R
library(magrittr) library(stringr) x <- read.table("~/Downloads/tmp", stringsAsFactors = F) strsplit(x$V6, "/") %>% sapply(function(x) x[1]) %>% unique() # 415 packages strsplit(x$V6, "/") %>% sapply(function(x) x[1]) %>% table() %>% sort() # how many Fortran files in each package str_subset(x$V6, "\\.f90$") %>% strsplit("/") %>% sapply(function(x) x[1]) %>% unique() # Fortran 90 only packages str_subset(x$V6, "\\.f95$") %>% strsplit("/") %>% sapply(function(x) x[1]) %>% unique() # Fortran 95 only packages str_subset(x$V6, "BayesFM") # f95
Misc
Datasets in R packages
https://vincentarelbundock.github.io/Rdatasets/datasets.html
Turn your analysis into a package
Build R package faster using multicore
http://www.rexamine.com/2015/07/speeding-up-r-package-installation-process/
The idea is edit the /lib64/R/etc/Renviron file (where /lib64/R/etc/ is the result to a call to the R.home() function in R) and set:
MAKE='make -j 8' # submit 8 jobs at once
Then build R package as regular, for example,
$ time R CMD INSTALL ~/R/stringi --preclean --configure-args='--disable-pkg-config'
suppressPackageStartupMessages() and .onAttach()
KernSmooth package example.
It is Time for CRAN to Ban Package Ads
suppressPackageStartupMessages(library("dplyr"))
fusen package
- {fusen} is now available on CRAN!
- (Videos) How To Build A Package With The "Rmd First" Method, fusen::inflate()
Identifying R Functions & Packages Used in GitHub Repos
Identifying R Functions & Packages Used in GitHub Repos: funspotr package
CRANalerts
R-universe
How to create your personal CRAN-like repository on R-universe