--- title: "*1 What is deep learning? 1*: Deep Learning With R" id: "13172" type: "page" slug: "01-deep-learning-with-r" published_at: "2026-07-19T18:29:47+00:00" modified_at: "2026-07-20T00:41:40+00:00" url: "https://preppers-paradise.com/library/deeplearningwithrsecondedition/01-deep-learning-with-r/" markdown_url: "https://preppers-paradise.com/library/deeplearningwithrsecondedition/01-deep-learning-with-r.md" excerpt: "This guide provides a comprehensive introduction to deep learning, covering its fundamental concepts, historical evolution, and practical applications. It delves into the mathematical underpinnings of" taxonomy_category: - "AI & Machine Learning" - "Books" - "Free Teaser" taxonomy_post_tag: - "artificial intelligence" - "backpropagation" - "data science" - "deep learning" - "gradient descent" - "machine learning" - "neural networks" - "r programming" - "tensors" --- # *1 What is deep learning? 1*: Deep Learning With R [← *1 What is deep learning? 1*](/library/deeplearningwithrsecondedition/) Chapter 1 of 80 · Free teaser ## *Deep Learning with R* SECOND EDITION FRANÇOIS CHOLLET WITH TOMASZ KALINOWSKI AND J.J. ALLAIRE ![](https://preppers-paradise.com/wp-content/uploads/x402-books/deeplearningwithrsecondedition/_page_3_Picture_3.jpeg) {4}------------------------------------------------ No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. > *[Artificial intelligence 2](#page-20-1)* ■ *[Machine learning 3](#page-21-0)* ■ *[Learning](#page-22-0) [rules and representations from data 4](#page-22-0)* ■ *[The "deep" in "deep](#page-25-0) [learning" 7](#page-25-0)* ■ *[Understanding how deep learning works, in](#page-26-0) [three figures 8](#page-26-0)* ■ *[What deep learning has achieved so far 10](#page-28-0) [Don't believe the short-term hype 11](#page-29-0)* ■ *[The promise of AI 12](#page-30-0)* [1.2 Before deep learning: A brief history of machine learning 13](#page-31-0) *[Probabilistic modeling 13](#page-31-1)* ■ *[Early neural networks 13](#page-31-2) [Kernel methods 14](#page-32-0)* ■ *[Decision trees, random forests, and](#page-33-0) [gradient-boosting machines 15](#page-33-0)* ■ *[Back to neural networks 16](#page-34-0) [What makes deep learning different? 17](#page-35-0)* ■ *[The modern machine](#page-35-1) [learning landscape 17](#page-35-1)* {6}------------------------------------------------ **vi** CONTENTS [1.3 Why deep learning? Why now? 20](#page-38-0) | | Hardware
20
■ Data
21
■ Algorithms
22
■ A new wave of
investment
22
■ The democratization of deep learning
23
Will it last?
24 | |-----|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | | 2 The mathematical building blocks of neural networks
26 | | 2.1 | A first look at a neural network
27 | | 2.2 | Data representations for neural networks
31 | | | Scalars (rank 0 tensors)
31
■ Vectors (rank 1 tensors)
31
Matrices (rank 2 tensors)
32
■ Rank 3 and higher-rank
tensors
32
■ Key attributes
33
■ Manipulating tensors
in R
34
■ The notion of data batches
35
■ Real-world examples
of data tensors
35
■ Vector data
35
■ Time-series data or
sequence data
36
■ Image data
36
■ Video data
37 | | 2.3 | The gears of neural networks: Tensor operations
37
Element-wise operations
38
■ Broadcasting
40
■ Tensor
product
41
■ Tensor reshaping
43
■ Geometric interpretation
of tensor operations
44
■ A geometric interpretation of deep
learning
47 | | 2.4 | The engine of neural networks: Gradient-based
optimization
48 | | | What's a derivative?
49
■ Derivative of a tensor operation: The
gradient
50
■ Stochastic gradient descent
51
■ Chaining
derivatives: The backpropagation algorithm
54 | | 2.5 | Looking back at our first example
59 | | | Reimplementing our first example from scratch in TensorFlow
61
Running one training step
63
■ The full training loop
65
Evaluating the model
66 | | | 3 Introduction to Keras and TensorFlow
68 | | 3.1 | What's TensorFlow?
69 | | 3.2 | What's Keras?
69 | | 3.3 | Keras and TensorFlow: A brief history
71 | | 3.4 | Python and R interfaces: A brief history
71 | | 3.5 | Setting up a deep learning workspace
72
Installing Keras and TensorFlow
73 | | 3.6 | First steps with TensorFlow
74
TensorFlow tensors
74 | {7}------------------------------------------------ CONTENTS **vii** [3.7 Tensor attributes 75](#page-93-0) | | Tensor shape and reshaping
77
■ Tensor slicing
78
■ Tensor
broadcasting
79
■ The tf module
80
■ Constant tensors and
variables
81
■ Tensor operations: Doing math in TensorFlow
82
A second look at the GradientTape API
83
■ An end-to-end
example: A linear classifier in pure TensorFlow
84 | | |-----|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----| | 3.8 | Anatomy of a neural network: Understanding
core Keras APIs
89 | | | | Layers: The building blocks of deep learning
89
■ From layers
to models
94
■ The "compile" step: Configuring the learning
process
95
■ Picking a loss function
98
■ Understanding the
fit() method
99
■ Monitoring loss and metrics on validation
data
99
■ Inference: Using a model after training
101 | | | | 4 Getting started with neural networks: Classification | | | | and regression
103 | | | 4.1 | Classifying movie reviews: A binary classification example | 105 | | | The IMDB dataset
105
■ Preparing the data
107
■ Building
your model
108
■ Validating your approach
110
■ Using a
trained model to generate predictions on new data
113
■ Further
experiments
113
■ Wrapping up
113 | | | 4.2 | Classifying newswires: A multiclass classification example | 114 | | | The Reuters dataset
114
■ Preparing the data
116
■ Building
your model
116
■ Validating your approach
117
■ Generating
predictions on new data
119
■ A different way to handle the labels
and the loss
120
■ The importance of having sufficiently large
intermediate layers
120
■ Further experiments
121
Wrapping up
121 | | | 4.3 | Predicting house prices: A regression example
122 | | | | The Boston housing price dataset
122
■ Preparing the data
123
Building your model
123
■ Validating your approach using K-fold
validation
124
■ Generating predictions on new data
128
Wrapping up
128 | | | | 130 | | | 5.1 | 5 Fundamentals of machine learning
Generalization: The goal of machine learning
130 | | | | Underfitting and overfitting
131
■ The nature of generalization in
deep learning
136 | | | 5.2 | Evaluating machine learning models
142
Training, validation, and test sets
142
■ Beating a common-sense | | *[baseline 145](#page-163-0)* ■ *[Things to keep in mind about model evaluation 146](#page-164-0)* {8}------------------------------------------------ **viii** CONTENTS [5.3 Improving model fit 146](#page-164-1) *[Tuning key gradient descent parameters 147](#page-165-0)* ■ *[Leveraging better](#page-167-0) [architecture priors 149](#page-167-0)* ■ *[Increasing model capacity 150](#page-168-0)* | 5.4 | Improving generalization
152 | |-----|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | | Dataset curation
152
■ Feature engineering
153
■ Using early
stopping
154
■ Regularizing your model
155 | | | 166 | | 6.1 | 6 The universal workflow of machine learning
Define the task
168 | | | Frame the problem
168
■ Collect a dataset
169
■ Understand
your data
173
■ Choose a measure of success
173 | | 6.2 | Develop a model
174 | | | Prepare the data
174
■ Choose an evaluation protocol
175
Beat a baseline
176
■ Scale up: Develop a model that
overfits
177
■ Regularize and tune your model
177 | | 6.3 | Deploy the model
178 | | | Explain your work to stakeholders and set expectations
178
■ Ship
an inference model
179
■ Monitor your model in the wild
182
Maintain your model
183 | | | 185 | | 7.1 | 7 Working with Keras: A deep dive
A spectrum of workflows
186 | | 7.2 | Different ways to build Keras models
186 | | | The Sequential model
187
■ The Functional API
189
Subclassing the Model class
196
■ Mixing and matching different
components
199
■ Remember: Use the right tool for the job
200 | | 7.3 | Using built-in training and evaluation loops
201 | | | Writing your own metrics
202
■ Using callbacks
204
■ Writing
your own callbacks
205
■ Monitoring and visualization with
TensorBoard
208 | | 7.4 | Writing your own training and evaluation loops
210 | | | Training vs. inference
210
■ Low-level usage of metrics
211
A complete training and evaluation loop
212
■ Make it fast with
tf_function()
215
■ Leveraging fit() with a custom training | | | loop
216 | | | 8 Introduction to deep learning for computer vision
220 | *[operation 228](#page-246-0)* *[The convolution operation 223](#page-241-0)* ■ *[The max-pooling](#page-246-0)* {9}------------------------------------------------ CONTENTS **ix** | 8.2 | Training a convnet from scratch on a small dataset
230
The relevance of deep learning for small data problems
230
Downloading the data
231
■ Building the model
234
■ Data
preprocessing
235
■ Using data augmentation
241 | |------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 8.3 | Leveraging a pretrained model
245
Feature extraction with a pretrained model
246
■ Fine-tuning a
pretrained model
254 | | | 258 | | 9.1 | 9 Advanced deep learning for computer vision
Three essential computer vision tasks
259 | | 9.2 | An image segmentation example
260 | | 9.3 | Modern convnet architecture patterns
269
Modularity, hierarchy, and reuse
269
■ Residual
connections
272
■ Batch normalization
275
■ Depthwise
separable convolutions
278
■ Putting it together: A mini
Xception-like model
280 | | 9.4 | Interpreting what convnets learn
282 | | | Visualizing intermediate activations
283
■ Visualizing convnet
filters
289
■ Visualizing heatmaps of class activation
294 | | | 301 | | 10.1 | 10 Deep learning for time series
Different kinds of time-series tasks
301 | | 10.2 | A temperature-forecasting example
302
Preparing the data
306
■ A common-sense, non–machine learning
baseline
310
■ Let's try a basic machine learning model
311
Let's try a 1D convolutional model
314
■ A first recurrent
baseline
316 | | 10.3 | Understanding recurrent neural networks
317 | | | A recurrent layer in Keras
320 | | 10.4 | Advanced use of recurrent neural networks
324
Using recurrent dropout to fight overfitting
324
■ Stacking
recurrent layers
327
■ Using bidirectional RNNs
329
Going even further
332 | | | 334 | | 11.1 | 11 Deep learning for text
Natural language processing: The bird's-eye view
334 | | 11.2 | Preparing text data
336 | | | Text standardization
337
■ Text splitting (tokenization)
338
Vocabulary indexing
339
■ Using layer_text_vectorization
340 | | | | {10}------------------------------------------------ **x** CONTENTS | 11.3 | Two approaches for representing groups of words:
Sets and sequences
344 | |------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | | Preparing the IMDB movie reviews data
345
■ Processing words
as a set: The bag-of-words approach
347
■ Processing words as a
sequence: The sequence model approach
355 | | 11.4 | The Transformer architecture
366
Understanding self-attention
366
■ Multi-head attention
371
The Transformer encoder
372
■ When to use sequence models over
bag-of-words models
381 | | 11.5 | Beyond text classification: Sequence-to-sequence
learning
382 | | | A machine translation example
383
■ Sequence-to-sequence
learning with RNNs
387
■ Sequence-to-sequence learning with
Transformer
392 | | | 399 | | 12.1 | 12 Generative deep learning
Text generation
401 | | | A brief history of generative deep learning for sequence
generation
401
■ How do you generate sequence data?
402
The importance of the sampling strategy
402
■ Implementing text
generation with Keras
404
■ A text-generation callback with
variable-temperature sampling
408
■ Wrapping up
413 | | 12.2 | DeepDream
414 | | | Implementing DeepDream in Keras
415
■ Wrapping up
421 | | 12.3 | Neural style transfer
422 | | | The content loss
423
■ The style loss
424
■ Neural style transfer
in Keras
424
■ Wrapping up
431 | | 12.4 | Generating images with variational autoencoders
432
Sampling from latent spaces of images
432
■ Concept vectors
for image editing
433
■ Variational autoencoders
434
Implementing a VAE with Keras
436
■ Wrapping up
442 | | 12.5 | Introduction to generative adversarial networks
442 | | | A schematic GAN implementation
443
■ A bag
of tricks
444
■ Getting our hands on the CelebA dataset
445
The discriminator
447
■ The generator
447
■ The adversarial
network
448
■ Wrapping up
452 | | | 454 | | 13.1 | 13 Best practices for the real world
Getting the most out of your models
455 | *[Hyperparameter optimization 455](#page-473-1)* ■ *[Model ensembling 462](#page-480-0)* {11}------------------------------------------------ CONTENTS **xi** ## [13.2 Scaling-up model training 464](#page-482-0) *[Speeding up training on GPU with mixed precision 465](#page-483-0) [Multi-GPU training 467](#page-486-0)* ■ *[TPU training 471](#page-489-0)* # *14 Conclusions 473* ### [14.1 Key concepts in review 474](#page-492-0) *[Various approaches to AI 474](#page-493-1)* ■ *[What makes deep learning special](#page-493-2) [within the field of machine learning 474](#page-493-2)* ■ *[How to think about](#page-494-0) [deep learning 475](#page-494-0)* ■ *[Key enabling technologies 476](#page-495-0)* ■ *[The](#page-496-0) [universal machine learning workflow 477](#page-496-0)* ■ *[Key network](#page-497-0) [architectures 478](#page-497-0)* ■ *[The space of possibilities 482](#page-501-0)* ## [14.2 The limitations of deep learning 484](#page-503-0) *[The risk of anthropomorphizing machine learning models 485](#page-504-0) [Automatons vs. intelligent agents 487](#page-506-0)* ■ *[Local generalization](#page-507-0) [vs. extreme generalization 488](#page-507-0)* ■ *[The purpose of intelligence 490](#page-509-0) [Climbing the spectrum of generalization 491](#page-510-0)* - [14.3 Setting the course toward greater generality in AI 492](#page-511-0) *[On the importance of setting the right objective: The shortcut](#page-511-1) [rule 492](#page-511-1)* ■ *[A new target 494](#page-513-0)* - [14.4 Implementing intelligence: The missing ingredients 495](#page-514-0) *[Intelligence as sensitivity to abstract analogies 496](#page-515-0)* ■ *[The two poles](#page-516-0) [of abstraction 497](#page-516-0)* ■ *[The two poles of abstraction 500](#page-519-0)* ■ *[The](#page-519-1) [missing half of the picture 500](#page-519-1)* ### [14.5 The future of deep learning 501](#page-520-0) *[Models as programs 502](#page-521-0)* ■ *[Machine learning vs. program](#page-522-0) [synthesis 503](#page-522-0)* ■ *[Blending together deep learning and program](#page-522-1) [synthesis 503](#page-522-1)* ■ *[Lifelong learning and modular subroutine](#page-524-0) [reuse 505](#page-524-0)* ■ *[The long-term vision 506](#page-525-0)*