Rserve: Difference between revisions

From 太極
Jump to navigation Jump to search
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Basic ==
= Basic =
Rserve uses socket connections between R and other applications.
 
* [http://www.r-project.org/conferences/DSC-2003/Proceedings/Urbanek.pdf DSC-2003 Paper]
* [http://www.r-project.org/conferences/DSC-2003/Proceedings/Urbanek.pdf DSC-2003 Paper]
* [https://www.r-project.org/conferences/useR-2004/abstracts/supplements/Urbanek.pdf Use R in other Applications]


== Source code on github ==
= Packages =
https://github.com/s-u/Rserve/commits/master
== [https://cran.r-project.org/web/packages/Rserve/index.html Rserve] ==
* [https://github.com/s-u/Rserve/commits/master Source code on github]


== Properties ==
== [https://cran.r-project.org/web/packages/RSclient/index.html RSclient] ==


=== Running Rserve under Windows is very limited ===
= Properties =
 
== Running Rserve under Windows is very limited ==
'''Windows lacks important features that make the separation of namespaces possible, therefore Rserve for Windows works in cooperative mode only, that is only one connection at a time is allowed and all subsequent connections share the same namespace.'''
'''Windows lacks important features that make the separation of namespaces possible, therefore Rserve for Windows works in cooperative mode only, that is only one connection at a time is allowed and all subsequent connections share the same namespace.'''


This has an immediate implication that we cannot make use of Rserve.cluster package for high performance computing.
This has an immediate implication that we cannot make use of Rserve.cluster package for high performance computing.


=== Rserve needs to take double space compared to running under R gui ===
== Rserve needs to take double space compared to running under R gui ==
For example,
For example,
x <- rnorm(450000*200)
x <- rnorm(450000*200)
The R gui takes 0.7GB but Rserve uses 1.4GB. See a comprehensive comparison of running Rserve on Windows and Linux.
The R gui takes 0.7GB but Rserve uses 1.4GB. See a comprehensive comparison of running Rserve on Windows and Linux.


=== Rserve has a limit of maximum size of a single REXP ===
== Rserve has a limit of maximum size of a single REXP ==
'''Maximum size of a single REXP: 2GB (on 32-bit platforms), theoretical limit is 2^55 on 64-bit platforms. Packet size is auto-adjusted, configured by maxinbuf and maxsendbuf config entries. (maximum 2GB) The maxinbuf (max. packet from client to Rserve) and maxsendbuf (max. packet from Rserve to client) options in the configuration file allow you to set limits in order to prevent memory overflow on machines that act as servers for multiple connections. The defaults are 16MB and unlimited respectively.'''
'''Maximum size of a single REXP: 2GB (on 32-bit platforms), theoretical limit is 2^55 on 64-bit platforms. Packet size is auto-adjusted, configured by maxinbuf and maxsendbuf config entries. (maximum 2GB) The maxinbuf (max. packet from client to Rserve) and maxsendbuf (max. packet from Rserve to client) options in the configuration file allow you to set limits in order to prevent memory overflow on machines that act as servers for multiple connections. The defaults are 16MB and unlimited respectively.'''


Line 28: Line 34:
But if we try to use memory.limit(4000) to allocate 4GB space on 64-bit Rserve, we still get the same error on R console, but no error message on Rserve.
But if we try to use memory.limit(4000) to allocate 4GB space on 64-bit Rserve, we still get the same error on R console, but no error message on Rserve.


=== Difference in [[Windows]] and [[Linux]] ===
== Difference in [[Windows]] and [[Linux]] ==
Test the statement RSeval(cc, 'x <- rnorm(450000*narray)'). That creating a matrix of 450k genes and various number of arrays.  
Test the statement RSeval(cc, 'x <- rnorm(450000*narray)'). That creating a matrix of 450k genes and various number of arrays.  


Line 65: Line 71:
The Windows box has 12GB physical installed ram and Linux has 16GB.
The Windows box has 12GB physical installed ram and Linux has 16GB.


=== Some advice when running Rserve under Windows ===
== Some advice when running Rserve under Windows ==
Do not use memory hungry script through Rserve. Use R in batch mode to run it.
Do not use memory hungry script through Rserve. Use R in batch mode to run it.


=== Do you need to have Rserve subfolder created under library folder in order to use Rserve? ===
== Do you need to have Rserve subfolder created under library folder in order to use Rserve? ==
No. As long as Rserve.exe has been copied to i386\bin or x64\bin folder, Rserve will work. So don't be surprised if you run library(Rserve) and get an error saying ''there is no package called ‘Rserve’ ''.
No. As long as Rserve.exe has been copied to i386\bin or x64\bin folder, Rserve will work. So don't be surprised if you run library(Rserve) and get an error saying ''there is no package called ‘Rserve’ ''.


=== Can we use parallel/snow package with Rserve ===
== Can we use parallel/snow package with Rserve ==
Yes and no. We cannot directly use parallel package with Rserve. But with Rserve.cluster package, we can run parallel computing with Rserve. See the the package official website [http://www.rforge.net/Rserve.cluster].
Yes and no. We cannot directly use parallel package with Rserve. But with Rserve.cluster package, we can run parallel computing with Rserve. See the the package official website [http://www.rforge.net/Rserve.cluster].


== Applications ==
= Applications =


=== BRB-ArrayTools ===
== BRB-ArrayTools ==
Rserve will be opened in a new command prompt window. So a new application appears on the task bar.
Rserve will be opened in a new command prompt window. So a new application appears on the task bar.


=== [http://bio7.org/ Bio7] - An IDE for Ecological Modeling ===
== [http://bio7.org/ Bio7] - An IDE for Ecological Modeling ==
Rserve is started within Bio7 Console so no new application is shown on the task bar.
Rserve is started within Bio7 Console so no new application is shown on the task bar.


Line 86: Line 92:
[[File:Bio7_1.png|100px]] [[File:Bio7_2.png|100px]] [[File:Bio7_3.png|100px]]
[[File:Bio7_1.png|100px]] [[File:Bio7_2.png|100px]] [[File:Bio7_3.png|100px]]


=== [http://bioconductor.org/packages/2.14/bioc/vignettes/RGalaxy/inst/doc/RGalaxy-vignette.html RGalaxy] ===
== [http://bioconductor.org/packages/2.14/bioc/vignettes/RGalaxy/inst/doc/RGalaxy-vignette.html RGalaxy] ==
Given an R function and its manual page, make the documented function available in Galaxy.
Given an R function and its manual page, make the documented function available in Galaxy.


=== Taverna ===
== Taverna ==
http://www.taverna.org.uk/ and http://dev.mygrid.org.uk/wiki/display/taverna23/Rshell
http://www.taverna.org.uk/ and http://dev.mygrid.org.uk/wiki/display/taverna23/Rshell


=== J interface ===
== J interface ==
http://www.jsoftware.com/jwiki/Interfaces/R/Rserve
http://www.jsoftware.com/jwiki/Interfaces/R/Rserve
= Quirks =
== Reval error in unserialize unknown input format ==
In one instance, we got the above error on R 3.5.0 + Windows 7.
# Start up RServe. Copy '''RServe.exe''' to bin\x64 directory. Open a command prompt <syntaxhighlight lang='dos'>
cd C:\Program Files\R\r-2.14.1\bin\x64
R CMD Rserve
</syntaxhighlight>
# Connect through an R client <syntaxhighlight lang='rsplus'>
library(RSclient)
cc <- RSconnect()
RSeval(cc, quote(x2 <- date()))
RSeval(cc, quote(cat(ls(), file = "C:/Users/brb/Documents/out.txt")))
RSclose(cc)
</syntaxhighlight>
# Shutdown Rserve <syntaxhighlight lang='rsplus'>
library(Rserve)
cc <- RSconnect()
RSshutdown(cc)
</syntaxhighlight>
= Alternatives =
== RestRserve ==
https://restrserve.org/

Latest revision as of 16:54, 7 June 2020

Basic

Rserve uses socket connections between R and other applications.

Packages

Rserve

RSclient

Properties

Running Rserve under Windows is very limited

Windows lacks important features that make the separation of namespaces possible, therefore Rserve for Windows works in cooperative mode only, that is only one connection at a time is allowed and all subsequent connections share the same namespace.

This has an immediate implication that we cannot make use of Rserve.cluster package for high performance computing.

Rserve needs to take double space compared to running under R gui

For example, x <- rnorm(450000*200) The R gui takes 0.7GB but Rserve uses 1.4GB. See a comprehensive comparison of running Rserve on Windows and Linux.

Rserve has a limit of maximum size of a single REXP

Maximum size of a single REXP: 2GB (on 32-bit platforms), theoretical limit is 2^55 on 64-bit platforms. Packet size is auto-adjusted, configured by maxinbuf and maxsendbuf config entries. (maximum 2GB) The maxinbuf (max. packet from client to Rserve) and maxsendbuf (max. packet from Rserve to client) options in the configuration file allow you to set limits in order to prevent memory overflow on machines that act as servers for multiple connections. The defaults are 16MB and unlimited respectively.

If we run a statement like

RSeval(cc, "x <- rnorm(450000*400)")

we will see an error "Error in RSeval(cc, "x <- rnorm(450000*400)") : remote evaluation failed" in the R console. The Rserve window shows the problem

1: In rnorm(450000 * 400) :
  Reached total allocation of 2047Mb: see help(memory.size)

But if we try to use memory.limit(4000) to allocate 4GB space on 64-bit Rserve, we still get the same error on R console, but no error message on Rserve.

Difference in Windows and Linux

Test the statement RSeval(cc, 'x <- rnorm(450000*narray)'). That creating a matrix of 450k genes and various number of arrays.

narray space in R Windows Linux
200 0.7 1.4GB 666MB
300 1.0 Fail 1.9GB
400 1.4 Fail 2.6GB
500 1.7 Fail 3.6GB
Note memory.limit(40000) No memory.limit()

The Windows box has 12GB physical installed ram and Linux has 16GB.

Some advice when running Rserve under Windows

Do not use memory hungry script through Rserve. Use R in batch mode to run it.

Do you need to have Rserve subfolder created under library folder in order to use Rserve?

No. As long as Rserve.exe has been copied to i386\bin or x64\bin folder, Rserve will work. So don't be surprised if you run library(Rserve) and get an error saying there is no package called ‘Rserve’ .

Can we use parallel/snow package with Rserve

Yes and no. We cannot directly use parallel package with Rserve. But with Rserve.cluster package, we can run parallel computing with Rserve. See the the package official website [1].

Applications

BRB-ArrayTools

Rserve will be opened in a new command prompt window. So a new application appears on the task bar.

Bio7 - An IDE for Ecological Modeling

Rserve is started within Bio7 Console so no new application is shown on the task bar.

Programming languages are Java, Groovy and R. User interface is Eclipse, Java SWT and Java Swing.

Bio7 1.png Bio7 2.png Bio7 3.png

RGalaxy

Given an R function and its manual page, make the documented function available in Galaxy.

Taverna

http://www.taverna.org.uk/ and http://dev.mygrid.org.uk/wiki/display/taverna23/Rshell

J interface

http://www.jsoftware.com/jwiki/Interfaces/R/Rserve

Quirks

Reval error in unserialize unknown input format

In one instance, we got the above error on R 3.5.0 + Windows 7.

  1. Start up RServe. Copy RServe.exe to bin\x64 directory. Open a command prompt
    cd C:\Program Files\R\r-2.14.1\bin\x64
    R CMD Rserve
  2. Connect through an R client
    library(RSclient)
    cc <- RSconnect()
    RSeval(cc, quote(x2 <- date()))
    RSeval(cc, quote(cat(ls(), file = "C:/Users/brb/Documents/out.txt")))
    RSclose(cc)
  3. Shutdown Rserve
    library(Rserve)
    cc <- RSconnect()
    RSshutdown(cc)

Alternatives

RestRserve

https://restrserve.org/