AI

From 太極
Revision as of 16:44, 8 November 2024 by Brb (talk | contribs) (→‎call from R)
Jump to navigation Jump to search

人類如何勝過AI

人類如何勝過AI？

Applications

Prompts

Research prompts

Learning prompts

  • How to Learn Python FAST with ChatGPT?
    • Can you create a roadmap to learn python for data analysis
    • Can you create a roadmap to learn python for data analysis in 3 months with weekly plan and resources for learning
    • Can you create a roadmap to learn python for data analysis in 3 months with weekly plan, including resources and links for each week and youtube video links
    • Explain while loop in python to a child
  • How to learn to code FAST using ChatGPT
    • Give me a study plan to learn python for data science
    • Give me a study plan to learn python for data science with resources and a timeline
    • Sublime is used
    • (After ask a question and get an answer). Let's take this step by step.
  • Ask generative AI to be that colleague. Ask 'As a physicist, describe how cancer cells interact with their environment', or 'As a chemist..', 'As a developmental biologist..', 'As an economist..' 'As an electrician.' ...

Carbon footprint

Free AI isn’t sustainable — and we’ll be paying for it soon enough.

ChatGPT

https://chat.openai.com, https://openai.com/blog/chatgpt-plus/

Down

Network error

Network recommendations for ChatGPT errors on web and apps

Plugins

How to Enable ChatGPT’s Web Browsing and Plugins

Use

Live voice

7 Interesting Ways You Can Use ChatGPT's Live Voice and Vision

API, Extension tools

Create your GPT

How I Stopped Procrastinating with ChatGPT — 2 Hours Saved Each Day!

call from R

call from Python

Jupyter-ai

A generative AI extension for JupyterLab

Bing chat

It's been 18 days but Bing chat says R 4.3.0 is not yet released. It says the latest version of R is 4.1.3. The default conversion style is balanced (blue). After I changed it to more precise (green), the results are right.

DuckDuckGo

https://duck.ai

Brave AI chatbot: Leo

Everything You Need to Know About Leo: Brave Browser’s AI Chatbot

GPT-4

GPT-4o

Alternatives

PDF

Word

How to Automate Your Document Creation With ChatGPT in Microsoft Word

Research

Content writer

Detect AI text

Youtube summary

Chrome extension YouTube Summary with ChatGPT from 8 AI-Powered Chrome Extensions to Summarize YouTube Videos

AutoGPT

How to Download and Install Auto-GPT Step-by-Step

Other chats

4 AI Search Engines I Use Every Day. Perplexity, Exa, You AI, Andi AI.

Google Bard

perplexity.ai

https://www.perplexity.ai/

You.com

You.com’s AI-infused Google rival provides a tantalizing glimpse of the future

Claude

Meta's LLaMA

ollamar R package

Use R to prompt a local LLM with ollamar

Open source chats

Run locally

Jan

LM Studio

Ollama

GPT4ALL

Browser

BERT

LLM

Understanding LLMs from Scratch Using Middle School Math

AI, ML and DL

AI, ML and DL: What’s the Difference?

Coding

Images

Drawing

Describe images

GeoSpy

https://geospy.web.app/

Videos

Music

Text to/from speech

Bioinformatics

AI and statistics

Artificial Intelligence and Statistics: Just the Old Wine in New Wineskins? Faes 2022

What are the most important statistical ideas of the past 50 years

Four Deep Learning Papers to Read in June 2021

Neural network

Types of artificial neural networks

https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks

neuralnet package

nnet package

sauron package

Explaining predictions of Convolutional Neural Networks with 'sauron' package

OneR package

So, what is AI really?

h2o package

https://cran.r-project.org/web/packages/h2o/index.html

shinyML package

shinyML - Compare Supervised Machine Learning Models Using Shiny App

LSBoost

Explainable 'AI' using Gradient Boosted randomized networks Pt2 (the Lasso)

LightGBM/Light Gradient Boosting Machine

Survival data

Simulated neural network

Simulated Neural Network with Bootstrapping Time Series Data

Languages

GitHub: The top 10 programming languages for machine learning

Keras (high level library)

Keras is a model-level library, providing high-level building blocks for developing deep-learning models. It doesn’t handle low-level operations such as tensor manipulation and differentiation. Instead, it relies on a specialized, well-optimized tensor library to do so, serving as the backend engine of Keras.

Currently, the three existing backend implementations are the TensorFlow backend, the Theano backend, and the Microsoft Cognitive Toolkit (CNTK) backend.

On Ubuntu, we can install required packages by 

$ sudo apt-get install build-essential cmake git unzip \
                  pkg-config libopenblas-dev liblapack-dev
$ sudo apt-get install python-numpy python-scipy python- matplotlib python-yaml
$ sudo apt-get install libhdf5-serial-dev python-h5py
$ sudo apt-get install graphviz
$ sudo pip install pydot-ng
$ sudo apt-get install python-opencv

$ sudo pip install tensorflow  # CPU only
$ sudo pip install tensorflow-gpu # GPU support

$ sudo pip install theano

$ sudo pip install keras
$ python -c "import keras; print keras.__version__"
$ sudo pip install --upgrade keras  $ Upgrade Keras

To configure the backend of Keras, see Introduction to Python Deep Learning with Keras.

Example 1: DeepDecon.

  • Model Definition: In train_model.py, model = Sequential() defines a neural network model using the Keras Sequential API. It adds several dense (fully connected) layers with dropout for regularization. The activation function is set to ReLU for hidden layers and sigmoid or softmax for the output layer, depending on the number of output classes.
  • Model Compilation: self.model.compile(loss=self.loss, optimizer=self.optimizer, metrics=[rmse, 'mse', metrics.mae]) compiles the model, specifying the loss function, optimizer, and evaluation metrics. The custom RMSE function is included as one of the metrics.
  • Model Training: history = self.model.fit(X_tr, y_tr, batch_size=self.batch_size, epochs=self.epochs, validation_data=validation_data, callbacks=callbacks, shuffle=True, verbose=verbose) trains the model on the training data (X_tr, y_tr) with specified batch size and number of epochs. It also uses validation data for early stopping if enabled.
  • Early Stopping: if self.early_stopping sets up early stopping to prevent overfitting by monitoring the validation loss and stopping training if it doesn’t improve for a specified number of epochs.

In the eval.py code,

  • Loading Models models = {} This section loads pre-trained models from specified paths and stores them in a dictionary. The custom RMSE function is used during model loading.
  • Calculating Differences def get_difference() This function calculates the differences between the true labels and the predicted labels. It returns the minimum and maximum differences, as well as the difference array.
  • Single Prediction: def get_single_prediction() This function performs a single prediction by iteratively refining the prediction interval until it stabilizes.
  • Batch Prediction: def get_prediction() This function performs predictions for a batch of input data by calling "get_single_prediction" for each input sample.
  • Main Function: This section sets up argument parsing, loads the test data, performs predictions, and saves the results to a specified file.

TensorFlow (backend library)

Basic

Some terms

Machine Learning Glossary from developers.google.com

Tensor

Tensors for Neural Networks, Clearly Explained!!!

Dense layer and dropout layer

In Keras, what is a "dense" and a "dropout" layer?

Fully-connected layer (= dense layer). You can choose "relu" or "sigmoid" or "softmax" activation function.

Activation function

  • Artificial neural network -> Neural networks as functions [math]\displaystyle{ \textstyle f (x) = K \left(\sum_i w_i g_i(x)\right) }[/math] where K (commonly referred to as the activation function) is some predefined function, such as the hyperbolic tangent or sigmoid function or softmax function or rectifier function.
  • Rectifier/ReLU f(x) = max(0, x).
  • Sigmoid. Binary problem. Logistic function and hyperbolic tangent tanh(x) are two examples of sigmoid functions.
  • Softmax. Multiclass classification.

Backpropagation

https://en.wikipedia.org/wiki/Backpropagation

Convolutional network

https://en.wikipedia.org/wiki/Convolutional_neural_network

Deep Learning with Python

Jupyter notebooks for the code samples of the book "Deep Learning with Python" 

sudo apt install python3-pip python3-dev

sudo apt install build-essential cmake git unzip \
   pkg-config libopenblas-dev liblapack-dev
sudo apt-get install python3-numpy python3-scipy python3-matplotlib \
   python3-yaml
sudo apt install libhdf5-serial-dev python3-h5py
sudo apt install graphviz
sudo pip3 install pydot-ng

# sudo apt-get install python-opencv
# https://stackoverflow.com/questions/37188623/ubuntu-how-to-install-opencv-for-python3
# https://askubuntu.com/questions/783956/how-to-install-opencv-3-1-for-python-3-5-on-ubuntu-16-04-lts

sudo pip3 install keras

Colorize black-and-white photos

Colorize black-and-white photos

Keras using R

Training process:

  1. Draw a batch of X and Y
  2. Run the network on x (a step called the forward pass) to obtain predictions y_pred.
    • How many layers to use.
    • How many “hidden units” to chose for each layer.
  3. Compute the loss of the network on the batch
    • loss
    • optimizer: determines how learning proceeds (how the network will be updated based on the loss function). It implements a specific variant of stochastic gradient descent (SGD).
    • metrics
  4. Update all weights of the network in a way that slightly reduces the loss on this batch.
    • batch_size
    • epochs (=iteration over all samples in a batch_size of samples)

Keras (in order to use Keras, you need to install TensorFlow or CNTK or Theano):

  1. Define your training data: input tensors and target tensors.
  2. Define a network of layers (or model). Two ways to define a model:
    1. using the keras_model_sequential() function (only for linear stacks of layers, which is the most common network architecture by far) or 
      model <- keras_model_sequential() %>%
  layer_dense(units = 32, input_shape = c(784)) %>%
  layer_dense(units = 10, activation = "softmax")
    2. the functional API (for directed acyclic graphs of layers, which let you build completely arbitrary architectures) 
      input_tensor <- layer_input(shape = c(784))

output_tensor <- input_tensor %>%
  layer_dense(units = 32, activation = "relu") %>%
  layer_dense(units = 10, activation = "softmax")

model <- keras_model(inputs = input_tensor, outputs = output_tensor)
  3. Compile the learning process by choosing a loss function, an optimizer, and some metrics to monitor. 
    model %>% compile(
  optimizer = optimizer_rmsprop(lr = 0.0001),
  loss = "mse",
  metrics = c("accuracy")
)
  4. Iterate on your training data by calling the fit() method of your model. 
    model %>% fit(input_tensor, target_tensor, batch_size = 128, epochs = 10)

Custom loss function

Custom Loss functions for Deep Learning: Predicting Home Values with Keras for R

Metrics

https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/

Docker RStudio IDE

Assume we are using rocker/rstudio IDE, we need to install some packages first in the OS. 

$ docker run -d -p 8787:8787 -e USER=XXX -e PASSWORD=XXX --name rstudio rocker/rstudio

$ docker exec -it rstudio bash
# apt update
# apt install python-pip python-dev
# pip install virtualenv

And then in R, 

install.packages("keras")
library(keras)
install_keras(tensorflow = "1.5")

Use your own Dockerfile

Data Science for Startups: Containers Building reproducible setups for machine learning

Some examples

See Tensorflow for R from RStudio for several examples.

Binary data (Chapter 3.4)

  • The final layer will use a sigmoid activation so as to output a probability (a score between 0 and 1, indicating how likely the sample is to have the target “1”.
  • A relu (rectified linear unit) is a function meant to zero-out negative values, while a sigmoid “squashes” arbitrary values into the [0, 1] interval, thus outputting something that can be interpreted as a probability.
library(keras)
imdb <- dataset_imdb(num_words = 10000)
c(c(train_data, train_labels), c(test_data, test_labels)) %<-% imdb

# Preparing the data
vectorize_sequences <- function(sequences, dimension = 10000) {...}
x_train <- vectorize_sequences(train_data)
x_test <- vectorize_sequences(test_data)
y_train <- as.numeric(train_labels)
y_test <- as.numeric(test_labels)

# Build the network
## Two intermediate layers with 16 hidden units each
## The final layer will output the scalar prediction
model <- keras_model_sequential() %>% 
  layer_dense(units = 16, activation = "relu", input_shape = c(10000)) %>% 
  layer_dense(units = 16, activation = "relu") %>% 
  layer_dense(units = 1, activation = "sigmoid")
model %>% compile(
  optimizer = "rmsprop",
  loss = "binary_crossentropy",
  metrics = c("accuracy")
)
model %>% fit(x_train, y_train, epochs = 4, batch_size = 512)
## Error in py_call_impl(callable, dots$args, dots$keywords) : MemoryError: 
## 10.3GB memory is necessary on my 16GB machine

# Validation
results <- model %>% evaluate(x_test, y_test)

# Prediction on new data
model %>% predict(x_test[1:10,])

Multi class data (Chapter 3.5)

  • Goal: build a network to classify Reuters newswires into 46 different mutually-exclusive topics.
  • You end the network with a dense layer of size 46. This means for each input sample, the network will output a 46-dimensional vector. Each entry in this vector (each dimension) will encode a different output class.
  • The last layer uses a softmax activation. You saw this pattern in the MNIST example. It means the network will output a probability distribution over the 46 different output classes: that is, for every input sample, the network will produce a 46-dimensional output vector, where outputi is the probability that the sample belongs to class i. The 46 scores will sum to 1.
library(keras)
reuters <- dataset_reuters(num_words = 10000)
c(c(train_data, train_labels), c(test_data, test_labels)) %<-% reuters

model <- keras_model_sequential() %>% 
  layer_dense(units = 64, activation = "relu", input_shape = c(10000)) %>% 
  layer_dense(units = 64, activation = "relu") %>% 
  layer_dense(units = 46, activation = "softmax")
model %>% compile(
  optimizer = "rmsprop",
  loss = "categorical_crossentropy",
  metrics = c("accuracy")
)
history <- model %>% fit(
  partial_x_train,
  partial_y_train,
  epochs = 9,
  batch_size = 512,
  validation_data = list(x_val, y_val)
)
results <- model %>% evaluate(x_test, one_hot_test_labels)
# Prediction on new data
predictions <- model %>% predict(x_test)

Regression data (Chapter 3.6)

  • Because so few samples are available, we will be using a very small network with two hidden layers. In general, the less training data you have, the worse overfitting will be, and using a small network is one way to mitigate overfitting.
  • Our network ends with a single unit, and no activation (i.e. it will be linear layer). This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). Applying an activation function would constrain the range that the output can take. Here, because the last layer is purely linear, the network is free to learn to predict values in any range.
  • We are also monitoring a new metric during training: mae. This stands for Mean Absolute Error.
library(keras)
dataset <- dataset_boston_housing()
c(c(train_data, train_targets), c(test_data, test_targets)) %<-% dataset

build_model <- function() {
  model <- keras_model_sequential() %>% 
    layer_dense(units = 64, activation = "relu", 
                input_shape = dim(train_data)[[2]]) %>% 
    layer_dense(units = 64, activation = "relu") %>% 
    layer_dense(units = 1) 
    
  model %>% compile(
    optimizer = "rmsprop", 
    loss = "mse", 
    metrics = c("mae")
  )
}
# K-fold CV
k <- 4
indices <- sample(1:nrow(train_data))
folds <- cut(1:length(indices), breaks = k, labels = FALSE) 
num_epochs <- 100
all_scores <- c()
for (i in 1:k) {
  cat("processing fold #", i, "\n")
  # Prepare the validation data: data from partition # k
  val_indices <- which(folds == i, arr.ind = TRUE) 
  val_data <- train_data[val_indices,]
  val_targets <- train_targets[val_indices]
  
  # Prepare the training data: data from all other partitions
  partial_train_data <- train_data[-val_indices,]
  partial_train_targets <- train_targets[-val_indices]
  
  # Build the Keras model (already compiled)
  model <- build_model()
  
  # Train the model (in silent mode, verbose=0)
  model %>% fit(partial_train_data, partial_train_targets,
                epochs = num_epochs, batch_size = 1, verbose = 0)
                
  # Evaluate the model on the validation data
  results <- model %>% evaluate(val_data, val_targets, verbose = 0)
  all_scores <- c(all_scores, results$mean_absolute_error)
}

PyTorch

An R Shiny app to recognize flower species

Google Cloud Platform

Amazon

Amazon's Machine Learning University is making its online courses available to the public

Workshops

Notebooks from the Practical AI Workshop 2019

OpenML.org

R interface to OpenML.org

Biology

Retrieved from "https://wiki.taichimd.us/index.php?title=AI&oldid=40556"