CHAPTER 10: THE SEQUENTIAL API: BUILDING SIMPLE FEEDFORWARD NETWORKS (`tf.keras.Sequential`)

Begin Module 3 by mastering neural network construction with Keras. Learn how to build, compile, and train simple feedforward neural networks (Multi-Layer Perceptrons) using the intuitive `tf.keras.Sequential` API.

🧱 Introduction to Keras and the Sequential API

Welcome to Module 3! Having mastered tensors, automatic differentiation, and data pipelines, we now begin constructing actual neural networks. Keras is TensorFlow's high-level API for building and training deep learning models. The `tf.keras.Sequential` model is the most straightforward way to build neural networks in Kerasβ€”allowing you to stack layers linearly where each layer has exactly one input tensor and one output tensor.

1. Anatomy of a Feedforward Neural Network (MLP)
Understanding the architecture of Multi-Layer Perceptrons (MLPs)β€”comprising input layers, dense hidden layers, and output classification/regression layers.
Feedforward Architecture Matrix
FEEDFORWARD NETWORK STRUCTURE:
[ Input Features ] ──> [ Dense Hidden Layer 1 (ReLU) ] ──> [ Dense Hidden Layer 2 (ReLU) ] ──> [ Output Layer (Softmax/Linear) ]
Architectural Definition: Feedforward neural networks pass information strictly in one directionβ€”from inputs through hidden layers to outputsβ€”without cyclic feedback loops.
2. The Three-Step Keras Workflow (`model.add`, `model.compile`, `model.fit`)
Mastering the core Keras model lifecycle: building architecture, configuring loss and optimizers, and executing training epochs.
Keras Workflow Pattern
KERAS MODEL LIFECYCLE:
1. Construction (`Sequential`): Stack layers linearly using `model.add()` or list initialization.
2. Compilation (`model.compile()`): Configure optimizer (e.g., `'adam'`), loss function (e.g., `'sparse_categorical_crossentropy'`), and evaluation metrics.
3. Training (`model.fit()`): Execute mini-batch gradient descent across training epochs.
Engineering Insight: The `model.compile()` step configures the learning backend and computational graph, while `model.fit()` executes the actual training loop over your `tf.data` dataset.
3. Python Code Example: Building a Sequential Feedforward Network
A production Python script demonstrating how to build, compile, summarize, and train a Sequential feedforward neural network on synthetic classification data.
Python (sequential_mlp_demo.py)
import tensorflow as tf
from tensorflow.keras import layers, Sequential

def build_and_train_sequential_model():
    # 1. Construct Sequential Model
    model = Sequential([
        layers.Input(shape=(20,)),                          # Input feature dimension: 20
        layers.Dense(64, activation='relu', name='hidden_1'), # First hidden layer
        layers.Dense(32, activation='relu', name='hidden_2'), # Second hidden layer
        layers.Dense(2, activation='softmax', name='output')  # Binary classification output
    ], name="sequential_mlp")
    
    # Print model architecture summary
    model.summary()
    
    # 2. Compile Model
    model.compile(
        optimizer='adam',
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy']
    )
    
    # 3. Generate Dummy Training Data (1,000 samples, 20 features)
    X_train = tf.random.normal(shape=(1000, 20))
    y_train = tf.random.uniform(shape=(1000,), minval=0, maxval=2, dtype=tf.int32)
    
    # 4. Train Model
    print("\n[+] Training Sequential Model...")
    history = model.fit(
        X_train, y_train,
        epochs=3,
        batch_size=32,
        verbose=1
    )
    
    print("\n[+] Training complete!")

if __name__ == "__main__":
    build_and_train_sequential_model()
Execution Output: Model: "sequential_mlp" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= hidden_1 (Dense) (None, 64) 1344 hidden_2 (Dense) (None, 32) 2080 output (Dense) (None, 2) 66 ================================================================= Total params: 3,490 Trainable params: 3,490 Non-trainable params: 0 _________________________________________________________________ [+] Training Model... Epoch 1/3 32/32 [==============================] - 1s 5ms/step - loss: 0.7412 - accuracy: 0.5050 Epoch 2/3 32/32 [==============================] - 0s 2ms/step - loss: 0.6875 - accuracy: 0.5480 Epoch 3/3 32/32 [==============================] - 0s 2ms/step - loss: 0.6651 - accuracy: 0.5890 [+] Training complete!
4. Understanding `model.summary()` and Parameter Counts
How to interpret layer output shapes and calculate trainable parameter counts (weights + biases) for Dense layers.
Parameter Calculation Formula
DENSE LAYER PARAMETER CALCULATION:
- Formula: `Param Count = (Input Features Γ— Output Units) + Output Units (Biases)`
- Example (`hidden_1`: Input `20`, Units `64`):
  `(20 Γ— 64) + 64 = 1,280 + 64 = 1,344 parameters`.
Strategic Insight: Inspecting `model.summary()` is critical for verifying tensor shape propagation and ensuring your model capacity matches your dataset size.
5. Best Practices for Sequential Model Design
Software engineering standards for structuring Keras Sequential models and avoiding common architectural mistakes.
Sequential Best Practices Checklist
SEQUENTIAL MODEL CHECKLIST:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ BEST PRACTICE          β”‚ IMPLEMENTATION & OBJECTIVE                                                    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Explicit Input Shape   β”‚ Always define an explicit `layers.Input(shape=(...))` as the first layer so   β”‚
β”‚                        β”‚ Keras initializes weights immediately upon construction.                      β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Use Sequential for     β”‚ Reserve `tf.keras.Sequential` exclusively for linear stack architectures;     β”‚
β”‚ Linear Stacks Only     β”‚ use the Functional API for multi-input/output or branching models.            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Name Your Layers       β”‚ Assign custom names to layers (`name='hidden_1'`) to make debugging and tensorβ”‚
β”‚                        β”‚ inspection straightforward.                                                   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Masterclass Rule: The Sequential API is ideal for simple feedforward networks, but its linear restriction makes it unsuitable for complex architectures with shared layers or multiple branches.
6. Chapter Summary & Next Steps
Reviewing Sequential models and previewing activation functions.
Chapter Review Checklist
CHAPTER REVIEW CHECKLIST:
1. `tf.keras.Sequential` builds linear stack neural network architectures effortlessly.
2. The core Keras workflow follows three steps: construction, compilation (`model.compile`), and training (`model.fit`).
3. Parameter counts in Dense layers scale as `(Inputs Γ— Units) + Biases`.
4. Next Chapter: We cover Chapter 11 ("Activation Functions: ReLU, Sigmoid, Softmax, and Leaky ReLU").
Next Chapter Preview: In Chapter 11, we explore activation functionsβ€”examining how non-linear activations like ReLU, Sigmoid, and Softmax enable neural networks to learn complex decision boundaries.