INTRODUCTION TO DATA ANALYSIS IN PYTHON

Master the data analysis lifecycle, structured tabular workflows, cleaning methodologies, and descriptive statistics using Python.

📊 What is Data Analysis in Python?

Data Analysis is the systematic process of inspecting, cleaning, transforming, and modeling raw data to discover actionable insights, support decision-making, and solve complex business problems.

Python has become the industry-standard language for data science, powered by robust core libraries such as Pandas (tabular manipulation), NumPy (numerical array computing), and Matplotlib (data visualization).

1. The Data Analysis Lifecycle & Environment Setup
Understand the 5 stages of data analysis and import core analysis libraries.
Python
# Importing essential data analysis libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

print("Data Analysis environment initialized successfully.")
print(f"Pandas Version: {pd.__version__}")
print(f"NumPy Version: {np.__version__}")
Output: Data Analysis environment initialized successfully. Pandas Version: 2.2.0 NumPy Version: 1.26.4
Real-Time Example 1: Loading Structured Datasets
Load tabular datasets from CSV files into Pandas DataFrames for immediate analysis.
Python
import pandas as pd

# Simulating dataset loading from CSV (or read_csv('sales_data.csv'))
data = {
    "OrderID": [101, 102, 103, 104],
    "Customer": ["Rahul", "Priya", "Amit", "Sneha"],
    "Product": ["Mouse", "Keyboard", "Monitor", "Webcam"],
    "Amount": [450, 1200, 8500, 1500]
}

df = pd.DataFrame(data)
print("Loaded Dataset Preview:\n", df)
Output: Loaded Dataset Preview: OrderID Customer Product Amount 0 101 Rahul Mouse 450 1 102 Priya Keyboard 1200 2 103 Amit Monitor 8500 3 104 Sneha Webcam 1500
Real-Time Example 2: Initial Dataset Inspection (.head() & .info())
Inspect dataset shape, column data types, and non-null counts using inspection methods.
Python
import pandas as pd

df = pd.DataFrame({
    "Category": ["Electronics", "Clothing", "Electronics", "Furniture"],
    "Sales": [45000, 12000, 68000, 25000]
})

print("Dataset Shape (Rows, Cols):", df.shape)
print("\nFirst 2 Rows (.head(2)):\n", df.head(2))
Output: Dataset Shape (Rows, Cols): (4, 2) First 2 Rows (.head(2)): Category Sales 0 Electronics 45000 1 Clothing 12000
Real-Time Example 3: Exploratory Descriptive Statistics (.describe())
Compute descriptive statistics (mean, median, standard deviation, min, max) across numerical columns instantly.
Python
import pandas as pd

df = pd.DataFrame({
    "Employee": ["Rahul", "Priya", "Amit", "Sneha"],
    "Salary": [65000, 72000, 50000, 85000],
    "ExperienceYears": [3, 5, 2, 7]
})

print("Statistical Summary (.describe()):\n", df.describe())
Output: Statistical Summary (.describe()): Salary ExperienceYears count 4.000000 4.000000 mean 68000.000000 4.250000 std 14719.601456 2.217356 min 50000.000000 2.000000 25% 61250.000000 2.750000 50% 68500.000000 4.000000 75% 75250.000000 5.500000 max 85000.000000 7.000000
Real-Time Example 4: Filtering and Querying Data
Filter rows based on specific analytical conditions using boolean indexing.
Python
import pandas as pd

df = pd.DataFrame({
    "Product": ["Mouse", "Keyboard", "Monitor", "Laptop"],
    "Price": [450, 1200, 8500, 55000],
    "InStock": [True, True, False, True]
})

# Filter products priced over ₹5000 that are in stock
filtered_df = df[(df["Price"] > 5000) & (df["InStock"] == True)]

print("Filtered High-Value In-Stock Products:\n", filtered_df)
Output: Filtered High-Value In-Stock Products: Product Price InStock 3 Laptop 55000 True
Real-Time Example 5: Handling Missing Data (.isnull() and .fillna())
Identify and clean missing (NaN) records in datasets before performing statistical aggregation.
Python
import numpy as np
import pandas as pd

df = pd.DataFrame({
    "City": ["Pune", "Mumbai", None, "Delhi"],
    "Temperature": [32, np.nan, 35, 30]
})

# Fill missing temperature values with column mean
mean_temp = df["Temperature"].mean()
df["Temperature"] = df["Temperature"].fillna(mean_temp)

print("Cleaned Dataset:\n", df)
Output: Cleaned Dataset: City Temperature 0 Pune 32.0 1 Mumbai 32.333333 2 None 35.0 3 Delhi 30.0
Real-Time Example 6: Adding Calculated Columns & Transformations
Create new analytical columns derived from mathematical operations on existing columns.
Python
import pandas as pd

df = pd.DataFrame({
    "Item": ["Mouse", "Keyboard", "Hub"],
    "UnitCost": [450, 1200, 850],
    "QuantitySold": [10, 5, 8]
})

# Calculate total revenue per item
df["TotalRevenue"] = df["UnitCost"] * df["QuantitySold"]

print("Dataset with Calculated Revenue:\n", df)
Output: Dataset with Calculated Revenue: Item UnitCost QuantitySold TotalRevenue 0 Mouse 450 10 4500 1 Keyboard 1200 5 6000 2 Hub 850 8 6800
Real-Time Example 7: Grouping and Aggregating Data (.groupby())
Group datasets by categorical attributes and compute aggregate metrics.
Python
import pandas as pd

df = pd.DataFrame({
    "Department": ["IT", "HR", "IT", "HR", "Engineering"],
    "Salary": [65000, 50000, 70000, 52000, 80000]
})

# Compute total salary expense per department
dept_expense = df.groupby("Department")["Salary"].sum()

print("Total Salary Expense by Department:\n", dept_expense)
Output: Total Salary Expense by Department: Department Engineering 80000 HR 102000 IT 135000 Name: Salary, dtype: int64
Real-Time Example 8: Sorting and Ranking Datasets
Sort records by column values in descending order to identify top performers.
Python
import pandas as pd

df = pd.DataFrame({
    "Salesperson": ["Rahul", "Priya", "Amit", "Sneha"],
    "TotalSales": [45000, 92000, 67000, 81000]
})

# Rank salespersons by total sales descending
ranked_df = df.sort_values(by="TotalSales", ascending=False)

print("Salespersons Ranked by Performance:\n", ranked_df)
Output: Salespersons Ranked by Performance: Salesperson TotalSales 1 Priya 92000 3 Sneha 81000 2 Amit 67000 0 Rahul 45000
Real-Time Example 9: Exporting Analyzed Datasets
Export cleaned and analyzed DataFrames back into CSV or JSON files for reporting.
Python
import pandas as pd

df = pd.DataFrame({
    "Metric": ["Total Users", "Active Sessions", "Conversion Rate"],
    "Value": [12500, 4200, "3.4%"]
})

# Exporting analyzed results
df.to_csv("analytical_report.csv", index=False)
print("Analyzed report exported successfully to 'analytical_report.csv'.")
Output: Analyzed report exported successfully to 'analytical_report.csv'.