π§± Introduction to Keras and the Sequential API
Welcome to Module 3! Having mastered tensors, automatic differentiation, and data pipelines, we now begin constructing actual neural networks. Keras is TensorFlow's high-level API for building and training deep learning models. The `tf.keras.Sequential` model is the most straightforward way to build neural networks in Kerasβallowing you to stack layers linearly where each layer has exactly one input tensor and one output tensor.
1. Anatomy of a Feedforward Neural Network (MLP)
Understanding the architecture of Multi-Layer Perceptrons (MLPs)βcomprising input layers, dense hidden layers, and output classification/regression layers.
Feedforward Architecture Matrix
FEEDFORWARD NETWORK STRUCTURE:
[ Input Features ] ββ> [ Dense Hidden Layer 1 (ReLU) ] ββ> [ Dense Hidden Layer 2 (ReLU) ] ββ> [ Output Layer (Softmax/Linear) ]
Architectural Definition: Feedforward neural networks pass information strictly in one directionβfrom inputs through hidden layers to outputsβwithout cyclic feedback loops.
2. The Three-Step Keras Workflow (`model.add`, `model.compile`, `model.fit`)
Mastering the core Keras model lifecycle: building architecture, configuring loss and optimizers, and executing training epochs.
Keras Workflow Pattern
KERAS MODEL LIFECYCLE:
1. Construction (`Sequential`): Stack layers linearly using `model.add()` or list initialization.
2. Compilation (`model.compile()`): Configure optimizer (e.g., `'adam'`), loss function (e.g., `'sparse_categorical_crossentropy'`), and evaluation metrics.
3. Training (`model.fit()`): Execute mini-batch gradient descent across training epochs.
Engineering Insight: The `model.compile()` step configures the learning backend and computational graph, while `model.fit()` executes the actual training loop over your `tf.data` dataset.
3. Python Code Example: Building a Sequential Feedforward Network
A production Python script demonstrating how to build, compile, summarize, and train a Sequential feedforward neural network on synthetic classification data.
Python (sequential_mlp_demo.py)
import tensorflow as tf
from tensorflow.keras import layers, Sequential
def build_and_train_sequential_model():
# 1. Construct Sequential Model
model = Sequential([
layers.Input(shape=(20,)), # Input feature dimension: 20
layers.Dense(64, activation='relu', name='hidden_1'), # First hidden layer
layers.Dense(32, activation='relu', name='hidden_2'), # Second hidden layer
layers.Dense(2, activation='softmax', name='output') # Binary classification output
], name="sequential_mlp")
# Print model architecture summary
model.summary()
# 2. Compile Model
model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy']
)
# 3. Generate Dummy Training Data (1,000 samples, 20 features)
X_train = tf.random.normal(shape=(1000, 20))
y_train = tf.random.uniform(shape=(1000,), minval=0, maxval=2, dtype=tf.int32)
# 4. Train Model
print("\n[+] Training Sequential Model...")
history = model.fit(
X_train, y_train,
epochs=3,
batch_size=32,
verbose=1
)
print("\n[+] Training complete!")
if __name__ == "__main__":
build_and_train_sequential_model()
Execution Output:
Model: "sequential_mlp"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
hidden_1 (Dense) (None, 64) 1344
hidden_2 (Dense) (None, 32) 2080
output (Dense) (None, 2) 66
=================================================================
Total params: 3,490
Trainable params: 3,490
Non-trainable params: 0
_________________________________________________________________
[+] Training Model...
Epoch 1/3
32/32 [==============================] - 1s 5ms/step - loss: 0.7412 - accuracy: 0.5050
Epoch 2/3
32/32 [==============================] - 0s 2ms/step - loss: 0.6875 - accuracy: 0.5480
Epoch 3/3
32/32 [==============================] - 0s 2ms/step - loss: 0.6651 - accuracy: 0.5890
[+] Training complete!
4. Understanding `model.summary()` and Parameter Counts
How to interpret layer output shapes and calculate trainable parameter counts (weights + biases) for Dense layers.
Parameter Calculation Formula
DENSE LAYER PARAMETER CALCULATION:
- Formula: `Param Count = (Input Features Γ Output Units) + Output Units (Biases)`
- Example (`hidden_1`: Input `20`, Units `64`):
`(20 Γ 64) + 64 = 1,280 + 64 = 1,344 parameters`.
Strategic Insight: Inspecting `model.summary()` is critical for verifying tensor shape propagation and ensuring your model capacity matches your dataset size.
5. Best Practices for Sequential Model Design
Software engineering standards for structuring Keras Sequential models and avoiding common architectural mistakes.
Sequential Best Practices Checklist
SEQUENTIAL MODEL CHECKLIST:
ββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β BEST PRACTICE β IMPLEMENTATION & OBJECTIVE β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Explicit Input Shape β Always define an explicit `layers.Input(shape=(...))` as the first layer so β
β β Keras initializes weights immediately upon construction. β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Use Sequential for β Reserve `tf.keras.Sequential` exclusively for linear stack architectures; β
β Linear Stacks Only β use the Functional API for multi-input/output or branching models. β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Name Your Layers β Assign custom names to layers (`name='hidden_1'`) to make debugging and tensorβ
β β inspection straightforward. β
ββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Masterclass Rule: The Sequential API is ideal for simple feedforward networks, but its linear restriction makes it unsuitable for complex architectures with shared layers or multiple branches.
6. Chapter Summary & Next Steps
Reviewing Sequential models and previewing activation functions.
Chapter Review Checklist
CHAPTER REVIEW CHECKLIST:
1. `tf.keras.Sequential` builds linear stack neural network architectures effortlessly.
2. The core Keras workflow follows three steps: construction, compilation (`model.compile`), and training (`model.fit`).
3. Parameter counts in Dense layers scale as `(Inputs Γ Units) + Biases`.
4. Next Chapter: We cover Chapter 11 ("Activation Functions: ReLU, Sigmoid, Softmax, and Leaky ReLU").
Next Chapter Preview: In Chapter 11, we explore activation functionsβexamining how non-linear activations like ReLU, Sigmoid, and Softmax enable neural networks to learn complex decision boundaries.