---
title: "2.4 The engine of neural networks: Gradient-based optimization"
id: "12669"
type: "page"
slug: "deeplearningwithpythonthirdedition"
published_at: "2026-07-19T17:35:33+00:00"
modified_at: "2026-07-19T17:53:38+00:00"
url: "https://preppers-paradise.com/library/deeplearningwithpythonthirdedition/"
markdown_url: "https://preppers-paradise.com/library/deeplearningwithpythonthirdedition.md"
excerpt: "This content guides readers through the practical process of constructing and training a basic neural network for classification, covering essential steps like defining the model's layers, setting up the compilation process with an optimizer and loss function, preparing data, and..."
taxonomy_category:
  - "AI & Machine Learning"
  - "Books"
  - "Free Teaser"
taxonomy_post_tag:
  - "classification"
  - "data preprocessing"
  - "deep learning"
  - "keras"
  - "machine learning"
  - "model training"
  - "neural networks"
  - "overfitting"
  - "python"
  - "tensors"
---

# 2.4 The engine of neural networks: Gradient-based optimization

This content guides readers through the practical process of constructing and training a basic neural network for classification, covering essential steps like defining the model's layers, setting up the compilation process with an optimizer and loss function, preparing data, and evaluating the trained model. It also introduces the fundamental data structures in machine learning, detailing how data is represented using tensors, including scalars, vectors, and matrices, and explains their importance as the building blocks for neural network operations.

This page is free — overview and chapter list only.
Full book: $2.99 USDC · Per chapter: $0.25 USDC
[Buy / open complete book](/library/complete/deeplearningwithpythonthirdedition/) · [Complete book Markdown](/library/complete/deeplearningwithpythonthirdedition.md)

## Chapters

1. [Listing 22 The Network Architecture Import Keras From Keras](/library/deeplearningwithpythonthirdedition/01-listing-22-the-network-architecture-import-keras-from-keras/) (Free teaser)
1. [227 The Notion Of Data Batches](/library/deeplearningwithpythonthirdedition/02-227-the-notion-of-data-batches/) ($0.25)
1. [231 Element Wise Operations](/library/deeplearningwithpythonthirdedition/03-231-element-wise-operations/) ($0.25)
1. [235 Geometric Interpretation Of Tensor Operations](/library/deeplearningwithpythonthirdedition/04-235-geometric-interpretation-of-tensor-operations/) ($0.25)
1. [243 Stochastic Gradient Descent](/library/deeplearningwithpythonthirdedition/05-243-stochastic-gradient-descent/) ($0.25)
1. [25 Looking Back At Our First Example](/library/deeplearningwithpythonthirdedition/06-25-looking-back-at-our-first-example/) ($0.25)
1. [252 Running One Training Step](/library/deeplearningwithpythonthirdedition/07-252-running-one-training-step/) ($0.25)
1. [This Chapter Covers](/library/deeplearningwithpythonthirdedition/08-this-chapter-covers/) ($0.25)
1. [Gradients In Tensorflow A Second Look At The Gradienttape Ap](/library/deeplearningwithpythonthirdedition/09-gradients-in-tensorflow-a-second-look-at-the-gradienttape-ap/) ($0.25)
1. [Listing 316 Plotting The Two Point Classes](/library/deeplearningwithpythonthirdedition/10-listing-316-plotting-the-two-point-classes/) ($0.25)
1. [342 An End To End Example A Linear Classifier In Pure Pytorc](/library/deeplearningwithpythonthirdedition/11-342-an-end-to-end-example-a-linear-classifier-in-pure-pytorc/) ($0.25)
1. [353 Random Number Generation In Jax](/library/deeplearningwithpythonthirdedition/12-353-random-number-generation-in-jax/) ($0.25)
1. [354 An End To End Example A Linear Classifier In Pure Jax](/library/deeplearningwithpythonthirdedition/13-354-an-end-to-end-example-a-linear-classifier-in-pure-jax/) ($0.25)
1. [Automatic Shape Inference Building Layers On The Fly](/library/deeplearningwithpythonthirdedition/14-automatic-shape-inference-building-layers-on-the-fly/) ($0.25)
1. [364 The Compile Step Configuring The Learning Process](/library/deeplearningwithpythonthirdedition/15-364-the-compile-step-configuring-the-learning-process/) ($0.25)
1. [41 Classifying Movie Reviews A Binary Classification Example](/library/deeplearningwithpythonthirdedition/16-41-classifying-movie-reviews-a-binary-classification-example/) ($0.25)
1. [42 Classifying Newswires A Multiclass Classification Example](/library/deeplearningwithpythonthirdedition/17-42-classifying-newswires-a-multiclass-classification-example/) ($0.25)
1. [Listing 431 Plotting Validation Scores Epochs Range1 Lenaver](/library/deeplearningwithpythonthirdedition/18-listing-431-plotting-validation-scores-epochs-range1-lenaver/) ($0.25)
1. [Rare Features And Spurious Correlations](/library/deeplearningwithpythonthirdedition/19-rare-features-and-spurious-correlations/) ($0.25)
1. [The Manifold Hypothesis](/library/deeplearningwithpythonthirdedition/20-the-manifold-hypothesis/) ($0.25)
1. [521 Training Validation And Test Sets](/library/deeplearningwithpythonthirdedition/21-521-training-validation-and-test-sets/) ($0.25)
1. [K Fold Validation](/library/deeplearningwithpythonthirdedition/22-k-fold-validation/) ($0.25)
1. [523 Things To Keep In Mind About Model Evaluation](/library/deeplearningwithpythonthirdedition/23-523-things-to-keep-in-mind-about-model-evaluation/) ($0.25)
1. [533 Increasing Model Capacity](/library/deeplearningwithpythonthirdedition/24-533-increasing-model-capacity/) ($0.25)
1. [542 Feature Engineering](/library/deeplearningwithpythonthirdedition/25-542-feature-engineering/) ($0.25)
1. [Reducing The Networks Size](/library/deeplearningwithpythonthirdedition/26-reducing-the-networks-size/) ($0.25)
1. [Adding Dropout](/library/deeplearningwithpythonthirdedition/27-adding-dropout/) ($0.25)
1. [611 Framing The Problem](/library/deeplearningwithpythonthirdedition/28-611-framing-the-problem/) ($0.25)
1. [Beware Of Nonrepresentative Data](/library/deeplearningwithpythonthirdedition/29-beware-of-nonrepresentative-data/) ($0.25)
1. [621 Preparing The Data](/library/deeplearningwithpythonthirdedition/30-621-preparing-the-data/) ($0.25)
1. [623 Beating A Baseline](/library/deeplearningwithpythonthirdedition/31-623-beating-a-baseline/) ($0.25)
1. [632 Shipping An Inference Model](/library/deeplearningwithpythonthirdedition/32-632-shipping-an-inference-model/) ($0.25)
1. [634 Maintaining Your Model](/library/deeplearningwithpythonthirdedition/33-634-maintaining-your-model/) ($0.25)
1. [721 The Sequential Model](/library/deeplearningwithpythonthirdedition/34-721-the-sequential-model/) ($0.25)
1. [722 The Functional Api](/library/deeplearningwithpythonthirdedition/35-722-the-functional-api/) ($0.25)
1. [723 Subclassing The Model Class](/library/deeplearningwithpythonthirdedition/36-723-subclassing-the-model-class/) ($0.25)
1. [725 Remember Use The Right Tool For The Job](/library/deeplearningwithpythonthirdedition/37-725-remember-use-the-right-tool-for-the-job/) ($0.25)
1. [732 Using Callbacks](/library/deeplearningwithpythonthirdedition/38-732-using-callbacks/) ($0.25)
1. [734 Monitoring And Visualization With Tensorboard](/library/deeplearningwithpythonthirdedition/39-734-monitoring-and-visualization-with-tensorboard/) ($0.25)
1. [741 Training Vs Inference](/library/deeplearningwithpythonthirdedition/40-741-training-vs-inference/) ($0.25)
1. [Listing 82 Displaying The Models Summary](/library/deeplearningwithpythonthirdedition/41-listing-82-displaying-the-models-summary/) ($0.25)
1. [Listing 816 Displaying Curves Of Loss And Accuracy During Tr](/library/deeplearningwithpythonthirdedition/42-listing-816-displaying-curves-of-loss-and-accuracy-during-tr/) ($0.25)
1. [There Are A Few Things To Note Here](/library/deeplearningwithpythonthirdedition/43-there-are-a-few-things-to-note-here/) ($0.25)
1. [1421 Character And Word Tokenization](/library/deeplearningwithpythonthirdedition/44-1421-character-and-word-tokenization/) ($0.25)
1. [1422 Subword Tokenization](/library/deeplearningwithpythonthirdedition/45-1422-subword-tokenization/) ($0.25)
1. [1431 Loading The Imdb Classification Dataset](/library/deeplearningwithpythonthirdedition/46-1431-loading-the-imdb-classification-dataset/) ($0.25)
1. [1441 Training A Bag Of Words Model](/library/deeplearningwithpythonthirdedition/47-1441-training-a-bag-of-words-model/) ($0.25)
1. [Listing 1421 Building An Lstm Sequence Model](/library/deeplearningwithpythonthirdedition/48-listing-1421-building-an-lstm-sequence-model/) ($0.25)
1. [Listing 1424 Building An Lstm Sequence Model With An Embeddi](/library/deeplearningwithpythonthirdedition/49-listing-1424-building-an-lstm-sequence-model-with-an-embeddi/) ($0.25)
1. [Augmenting Text Data](/library/deeplearningwithpythonthirdedition/50-augmenting-text-data/) ($0.25)
1. [1455 Using The Pretrained Embedding For Classification](/library/deeplearningwithpythonthirdedition/51-1455-using-the-pretrained-embedding-for-classification/) ($0.25)
1. [1511 Training A Shakespeare Language Model](/library/deeplearningwithpythonthirdedition/52-1511-training-a-shakespeare-language-model/) ($0.25)
1. [1512 Generating Shakespeare](/library/deeplearningwithpythonthirdedition/53-1512-generating-shakespeare/) ($0.25)
1. [1522 Sequence To Sequence Learning With Rnns](/library/deeplearningwithpythonthirdedition/54-1522-sequence-to-sequence-learning-with-rnns/) ($0.25)
1. [1531 Dot Product Attention](/library/deeplearningwithpythonthirdedition/55-1531-dot-product-attention/) ($0.25)
1. [1532 Transformer Encoder Block](/library/deeplearningwithpythonthirdedition/56-1532-transformer-encoder-block/) ($0.25)
1. [1534 Sequence To Sequence Learning With A Transformer](/library/deeplearningwithpythonthirdedition/57-1534-sequence-to-sequence-learning-with-a-transformer/) ($0.25)
1. [1535 Embedding Positional Information](/library/deeplearningwithpythonthirdedition/58-1535-embedding-positional-information/) ($0.25)
1. [1542 Loading A Pretrained Transformer](/library/deeplearningwithpythonthirdedition/59-1542-loading-a-pretrained-transformer/) ($0.25)
1. [1543 Preprocessing Imdb Movie Reviews](/library/deeplearningwithpythonthirdedition/60-1543-preprocessing-imdb-movie-reviews/) ($0.25)
1. [1623 Generative Decoding](/library/deeplearningwithpythonthirdedition/61-1623-generative-decoding/) ($0.25)
1. [1624 Sampling Strategies](/library/deeplearningwithpythonthirdedition/62-1624-sampling-strategies/) ($0.25)
1. [1631 Text Generation With The Gemma Model](/library/deeplearningwithpythonthirdedition/63-1631-text-generation-with-the-gemma-model/) ($0.25)
1. [1632 Instruction Fine Tuning](/library/deeplearningwithpythonthirdedition/64-1632-instruction-fine-tuning/) ($0.25)
1. [Customizing Lora Training](/library/deeplearningwithpythonthirdedition/65-customizing-lora-training/) ($0.25)
1. [1642 Multimodal Llms](/library/deeplearningwithpythonthirdedition/66-1642-multimodal-llms/) ($0.25)
1. [1712 Variational Autoencoders](/library/deeplearningwithpythonthirdedition/67-1712-variational-autoencoders/) ($0.25)
1. [1713 Implementing A Vae With Keras](/library/deeplearningwithpythonthirdedition/68-1713-implementing-a-vae-with-keras/) ($0.25)
1. [1722 A U Net Denoising Autoencoder](/library/deeplearningwithpythonthirdedition/69-1722-a-u-net-denoising-autoencoder/) ($0.25)
1. [Listing 177 The Diffusion Schedule Def Diffusionschedule Dif](/library/deeplearningwithpythonthirdedition/70-listing-177-the-diffusion-schedule-def-diffusionschedule-dif/) ($0.25)
1. [1727 Its Go Time](/library/deeplearningwithpythonthirdedition/71-1727-its-go-time/) ($0.25)
1. [Visual Artifacts In The Stable Diffusion Output](/library/deeplearningwithpythonthirdedition/72-visual-artifacts-in-the-stable-diffusion-output/) ($0.25)
1. [1821 Multi Gpu Training](/library/deeplearningwithpythonthirdedition/73-1821-multi-gpu-training/) ($0.25)
1. [1822 Distributed Training In Practice](/library/deeplearningwithpythonthirdedition/74-1822-distributed-training-in-practice/) ($0.25)
1. [1832 Float16 Inference](/library/deeplearningwithpythonthirdedition/75-1832-float16-inference/) ($0.25)
1. [1912 Deep Learning Models Are Highly Sensitive To Phrasing A](/library/deeplearningwithpythonthirdedition/76-1912-deep-learning-models-are-highly-sensitive-to-phrasing-a/) ($0.25)
1. [1914 The Risk Of Anthropomorphizing Machine Learning Models](/library/deeplearningwithpythonthirdedition/77-1914-the-risk-of-anthropomorphizing-machine-learning-models/) ($0.25)
1. [1922 Local Generalization Vs Extreme Generalization](/library/deeplearningwithpythonthirdedition/78-1922-local-generalization-vs-extreme-generalization/) ($0.25)
1. [1924 Climbing The Spectrum Of Generalization](/library/deeplearningwithpythonthirdedition/79-1924-climbing-the-spectrum-of-generalization/) ($0.25)
1. [205 Final Words](/library/deeplearningwithpythonthirdedition/80-205-final-words/) ($0.25)