1 What is deep learning?

1.1 Artificial intelligence, machine learning, and deep learning

1.2 Artificial intelligence

1.3 Machine learning

1.4 Learning rules and representations from data

1.5 The “deep” in “deep learning”

1.6 Understanding how deep learning works, in three figures

1.7 What makes deep learning different

1.8 The age of generative AI

1.9 What deep learning has achieved so far

1.10 Beware of the short-term hype

1.11 Summer can turn to winter

1.12 The promise of AI

1.13 How R fits in

2 The mathematical building blocks of neural networks

2.1 A first look at a neural network

2.2 Data representations for neural networks

Scalars (rank-0 tensors)

Vectors (rank-1 tensors)

Matrices (rank-2 tensors)

Rank-3 tensors and higher-rank tensors

Key attributes

Manipulating NumPy tensors in R

The notion of data batches

Real-world examples of data tensors

2.3 The gears of neural networks: Tensor operations

Element-wise operations

Broadcasting

Tensor products

Tensor reshaping

Geometric interpretation of tensor operations

A geometric interpretation of deep learning

2.4 The engine of neural networks: Gradient-based optimization

What’s a derivative?

Derivative of a tensor operation: The gradient

Stochastic gradient descent

Chaining derivatives: The Backpropagation algorithm

2.5 Looking back at our first example

Reimplementing our first example from scratch

Running one training step

The full training loop

Evaluating the model

3 Introduction to TensorFlow, PyTorch, JAX, and Keras

3.1 A brief history of deep learning frameworks

3.2 How these frameworks relate to each other

3.3 Introduction to TensorFlow

First steps with TensorFlow

An end-to-end example: A linear classifier in pure TensorFlow

What makes the TensorFlow approach unique

3.4 Introduction to PyTorch

First steps with PyTorch

An end-to-end example: A linear classifier in pure PyTorch

What makes the PyTorch approach unique

3.5 Introduction to JAX

First steps with JAX

Tensors in JAX

Random number generation in JAX

An end-to-end example: A linear classifier in pure JAX

What makes the JAX approach unique

3.6 Introduction to Keras

First steps with Keras

Layers: The building blocks of deep learning

From layers to models

The “compile” step: Configuring the learning process

Picking a loss function

Understanding the fit() method

Monitoring loss and metrics on validation data

Inference: Using a model after training

4 Classification and regression

4.1 Classifying movie reviews: A binary classification example

The IMDb dataset

Preparing the data

Building the model

Validating the approach

Using a trained model to generate predictions on new data

Further experiments

Wrapping up

4.2 Classifying newswires: A multiclass classification example

The Reuters dataset

Preparing the data

Building the model

Validating the approach

Generating predictions on new data

A different way to handle the labels and the loss

The importance of having sufficiently large intermediate layers

Further experiments

Wrapping up

4.3 Predicting house prices: A regression example

The California housing dataset

Preparing the data

Building the model

Validating the approach using K-fold validation

Generating predictions on new data

Wrapping up

5 Fundamentals of machine learning

5.1 Generalization: The goal of machine learning

Underfitting and overfitting

The nature of generalization in deep learning

5.2 Evaluating machine learning models

Training, validation, and test sets

Beating a commonsense baseline

Things to keep in mind about model evaluation

5.3 Improving model fit

Tuning key gradient descent parameters

Using better architecture priors

Increasing model capacity

5.4 Improving generalization

Dataset curation

Feature engineering

Using early stopping

Regularizing a model

6 The universal workflow of machine learning

6.1 Defining the task

Framing the problem

Collecting a dataset

Understanding your data

Choosing a measure of success

6.2 Developing a model

Preparing the data

Choosing an evaluation protocol

Beating a baseline

Scaling up: Developing a model that overfits

Regularizing and tuning your model

6.3 Deploying your model

Explaining your work to stakeholders and setting expectations

Shipping an inference model

Monitoring your model in the wild

Maintaining your model

7 A deep dive into Keras

7.1 A spectrum of workflows

7.2 Different ways to build Keras models

The Sequential model

The Functional API

Subclassing the Model class

Mixing and matching different components

Remember: Use the right tool for the job

7.3 Using built-in training and evaluation loops

• Writing our own metrics
• Using callbacks
• The EarlyStopping and ModelCheckpoint callbacks
• Writing our own callbacks
• Monitoring and visualization with TensorBoard

7.4 Writing training and evaluation loops

• Training vs. inference
• Writing custom training step functions
• Low-level usage of metrics
• Using fit() with a custom training loop
• Handling metrics in a custom train_step()