AI: Difference between revisions

From 太極
Jump to navigation Jump to search
Line 23: Line 23:
** [https://www.manning.com/books/deep-learning-with-python Deep Learning with Python] by François Chollet, 2017 (available on safaribooksonline)
** [https://www.manning.com/books/deep-learning-with-python Deep Learning with Python] by François Chollet, 2017 (available on safaribooksonline)
** [https://github.com/janishar/mit-deep-learning-book-pdf Deep Learning] by Ian Goodfellow and Yoshua Bengio and Aaron Courville
** [https://github.com/janishar/mit-deep-learning-book-pdf Deep Learning] by Ian Goodfellow and Yoshua Bengio and Aaron Courville
* Tensor operations:
* Deep Learning Glossary
** relu(x) = max(0, x)
** http://www.wildml.com/deep-learning-glossary/
** Each neural layer from our first network example transforms its input data:'''output = relu(dot(W, input) + b)''' where  W and b are the ''weights'' or ''trainable parameters'' of the layer.
** [https://www.quora.com/What-is-an-epoch-in-deep-learning What is an epoch in deep learning?]
* Training process
 
*# Draw a batch of X and Y
== Keras ==
*# Run the network on x (a step called the forward pass) to obtain predictions y_pred.  
*# Compute the loss of the network on the batch
*# Update all weights of the network in a way that slightly reduces the loss on this batch.
* Derivative of a tensor operation: the gradient
* Derivative of a tensor operation: the gradient
** Define loss_value = f(W) = dot(W, x)
** Define loss_value = f(W) = dot(W, x)
** W1 = W0 - step * gradient(f)(W0)  
** W1 = W0 - step * gradient(f)(W0)  
* Stochastic gradient descent  
* Stochastic gradient descent  
* Deep Learning Glossary
* Tensor operations:
** http://www.wildml.com/deep-learning-glossary/
** relu(x) = max(0, x)
** [https://www.quora.com/What-is-an-epoch-in-deep-learning What is an epoch in deep learning?]
** Each neural layer from our first network example transforms its input data:'''output = relu(dot(W, input) + b)''' where  W and b are the ''weights'' or ''trainable parameters'' of the layer.
 
Training process
# Draw a batch of X and Y
# Run the network on x (a step called the forward pass) to obtain predictions y_pred.
# Compute the loss of the network on the batch
# Update all weights of the network in a way that slightly reduces the loss on this batch.
 
Keras (in order to use Keras, you need to install TensorFlow or CNTK or Theano):
# Define your training data: input tensors and target tensors.
# Define a network of layers (or model). Two ways to define a model:
## using the '''keras_model_sequential()''' function (only for linear stacks of layers, which is the most common network architecture by far) or <syntaxhighlight lang='rsplus'>
model <- keras_model_sequential() %>%
  layer_dense(units = 32, input_shape = c(784)) %>%
  layer_dense(units = 10, activation = "softmax")
</syntaxhighlight>
## the functional API (for directed acyclic graphs of layers, which let you build completely arbitrary architectures) <syntaxhighlight lang='rsplus'>
input_tensor <- layer_input(shape = c(784))
 
output_tensor <- input_tensor %>%
  layer_dense(units = 32, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")
 
model <- keras_model(inputs = input_tensor, outputs = output_tensor)
</syntaxhighlight>
# Configure the learning process by choosing a loss function, an optimizer, and some metrics to monitor. <syntaxhighlight lang='rsplus'>
model %>% compile(
  optimizer = optimizer_rmsprop(lr = 0.0001),
  loss = "mse",
  metrics = c("accuracy")
)
</syntaxhighlight>
# Iterate on your training data by calling the fit() method of your model. <syntaxhighlight lang='rsplus'>
model %>% fit(input_tensor, target_tensor, batch_size = 128, epochs = 10)
</syntaxhighlight>
 
The following examples can be found at [https://github.com/jjallaire/deep-learning-with-r-notebooks R Markdown Notebooks for "Deep Learning with R"]
== Some examples ==
* [https://jjallaire.github.io/deep-learning-with-r-notebooks/notebooks/3.4-classifying-movie-reviews.nb.html Binary data]
* [https://jjallaire.github.io/deep-learning-with-r-notebooks/notebooks/3.5-classifying-newswires.nb.html Multi class data]
* [https://jjallaire.github.io/deep-learning-with-r-notebooks/notebooks/3.6-predicting-house-prices.nb.html Regression data]


= PyTorch =
= PyTorch =
[https://longhowlam.wordpress.com/2018/12/17/an-r-shiny-app-to-recognize-flower-species/ An R Shiny app to recognize flower species]
[https://longhowlam.wordpress.com/2018/12/17/an-r-shiny-app-to-recognize-flower-species/ An R Shiny app to recognize flower species]

Revision as of 13:36, 1 January 2019

Applications

TensorFlow

Keras

  • Derivative of a tensor operation: the gradient
    • Define loss_value = f(W) = dot(W, x)
    • W1 = W0 - step * gradient(f)(W0)
  • Stochastic gradient descent
  • Tensor operations:
    • relu(x) = max(0, x)
    • Each neural layer from our first network example transforms its input data:output = relu(dot(W, input) + b) where W and b are the weights or trainable parameters of the layer.

Training process

  1. Draw a batch of X and Y
  2. Run the network on x (a step called the forward pass) to obtain predictions y_pred.
  3. Compute the loss of the network on the batch
  4. Update all weights of the network in a way that slightly reduces the loss on this batch.

Keras (in order to use Keras, you need to install TensorFlow or CNTK or Theano):

  1. Define your training data: input tensors and target tensors.
  2. Define a network of layers (or model). Two ways to define a model:
    1. using the keras_model_sequential() function (only for linear stacks of layers, which is the most common network architecture by far) or 
      model <- keras_model_sequential() %>%
  layer_dense(units = 32, input_shape = c(784)) %>%
  layer_dense(units = 10, activation = "softmax")
    2. the functional API (for directed acyclic graphs of layers, which let you build completely arbitrary architectures) 
      input_tensor <- layer_input(shape = c(784))

output_tensor <- input_tensor %>%
  layer_dense(units = 32, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")

model <- keras_model(inputs = input_tensor, outputs = output_tensor)
  3. Configure the learning process by choosing a loss function, an optimizer, and some metrics to monitor. 
    model %>% compile(
  optimizer = optimizer_rmsprop(lr = 0.0001),
  loss = "mse",
  metrics = c("accuracy")
)
  4. Iterate on your training data by calling the fit() method of your model. 
    model %>% fit(input_tensor, target_tensor, batch_size = 128, epochs = 10)

The following examples can be found at R Markdown Notebooks for "Deep Learning with R"

Some examples

PyTorch

An R Shiny app to recognize flower species

Retrieved from "https://wiki.taichimd.us/index.php?title=AI&oldid=18845"