📊 What is Data Analysis in Python?
Data Analysis is the systematic process of inspecting, cleaning, transforming, and modeling raw data to discover actionable insights, support decision-making, and solve complex business problems.
Python has become the industry-standard language for data science, powered by robust core libraries such as Pandas (tabular manipulation), NumPy (numerical array computing), and Matplotlib (data visualization).
1. The Data Analysis Lifecycle & Environment Setup
Understand the 5 stages of data analysis and import core analysis libraries.
Python
# Importing essential data analysis libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
print("Data Analysis environment initialized successfully.")
print(f"Pandas Version: {pd.__version__}")
print(f"NumPy Version: {np.__version__}")
Output:
Data Analysis environment initialized successfully.
Pandas Version: 2.2.0
NumPy Version: 1.26.4
Real-Time Example 1: Loading Structured Datasets
Load tabular datasets from CSV files into Pandas DataFrames for immediate analysis.
Python
import pandas as pd
# Simulating dataset loading from CSV (or read_csv('sales_data.csv'))
data = {
"OrderID": [101, 102, 103, 104],
"Customer": ["Rahul", "Priya", "Amit", "Sneha"],
"Product": ["Mouse", "Keyboard", "Monitor", "Webcam"],
"Amount": [450, 1200, 8500, 1500]
}
df = pd.DataFrame(data)
print("Loaded Dataset Preview:\n", df)
Output:
Loaded Dataset Preview:
OrderID Customer Product Amount
0 101 Rahul Mouse 450
1 102 Priya Keyboard 1200
2 103 Amit Monitor 8500
3 104 Sneha Webcam 1500
Real-Time Example 2: Initial Dataset Inspection (.head() & .info())
Inspect dataset shape, column data types, and non-null counts using inspection methods.
Python
import pandas as pd
df = pd.DataFrame({
"Category": ["Electronics", "Clothing", "Electronics", "Furniture"],
"Sales": [45000, 12000, 68000, 25000]
})
print("Dataset Shape (Rows, Cols):", df.shape)
print("\nFirst 2 Rows (.head(2)):\n", df.head(2))
Output:
Dataset Shape (Rows, Cols): (4, 2)
First 2 Rows (.head(2)):
Category Sales
0 Electronics 45000
1 Clothing 12000
Real-Time Example 3: Exploratory Descriptive Statistics (.describe())
Compute descriptive statistics (mean, median, standard deviation, min, max) across numerical columns instantly.
Python
import pandas as pd
df = pd.DataFrame({
"Employee": ["Rahul", "Priya", "Amit", "Sneha"],
"Salary": [65000, 72000, 50000, 85000],
"ExperienceYears": [3, 5, 2, 7]
})
print("Statistical Summary (.describe()):\n", df.describe())
Output:
Statistical Summary (.describe()):
Salary ExperienceYears
count 4.000000 4.000000
mean 68000.000000 4.250000
std 14719.601456 2.217356
min 50000.000000 2.000000
25% 61250.000000 2.750000
50% 68500.000000 4.000000
75% 75250.000000 5.500000
max 85000.000000 7.000000
Real-Time Example 4: Filtering and Querying Data
Filter rows based on specific analytical conditions using boolean indexing.
Python
import pandas as pd
df = pd.DataFrame({
"Product": ["Mouse", "Keyboard", "Monitor", "Laptop"],
"Price": [450, 1200, 8500, 55000],
"InStock": [True, True, False, True]
})
# Filter products priced over ₹5000 that are in stock
filtered_df = df[(df["Price"] > 5000) & (df["InStock"] == True)]
print("Filtered High-Value In-Stock Products:\n", filtered_df)
Output:
Filtered High-Value In-Stock Products:
Product Price InStock
3 Laptop 55000 True
Real-Time Example 5: Handling Missing Data (.isnull() and .fillna())
Identify and clean missing (NaN) records in datasets before performing statistical aggregation.
Python
import numpy as np
import pandas as pd
df = pd.DataFrame({
"City": ["Pune", "Mumbai", None, "Delhi"],
"Temperature": [32, np.nan, 35, 30]
})
# Fill missing temperature values with column mean
mean_temp = df["Temperature"].mean()
df["Temperature"] = df["Temperature"].fillna(mean_temp)
print("Cleaned Dataset:\n", df)
Output:
Cleaned Dataset:
City Temperature
0 Pune 32.0
1 Mumbai 32.333333
2 None 35.0
3 Delhi 30.0
Real-Time Example 6: Adding Calculated Columns & Transformations
Create new analytical columns derived from mathematical operations on existing columns.
Python
import pandas as pd
df = pd.DataFrame({
"Item": ["Mouse", "Keyboard", "Hub"],
"UnitCost": [450, 1200, 850],
"QuantitySold": [10, 5, 8]
})
# Calculate total revenue per item
df["TotalRevenue"] = df["UnitCost"] * df["QuantitySold"]
print("Dataset with Calculated Revenue:\n", df)
Output:
Dataset with Calculated Revenue:
Item UnitCost QuantitySold TotalRevenue
0 Mouse 450 10 4500
1 Keyboard 1200 5 6000
2 Hub 850 8 6800
Real-Time Example 7: Grouping and Aggregating Data (.groupby())
Group datasets by categorical attributes and compute aggregate metrics.
Python
import pandas as pd
df = pd.DataFrame({
"Department": ["IT", "HR", "IT", "HR", "Engineering"],
"Salary": [65000, 50000, 70000, 52000, 80000]
})
# Compute total salary expense per department
dept_expense = df.groupby("Department")["Salary"].sum()
print("Total Salary Expense by Department:\n", dept_expense)
Output:
Total Salary Expense by Department:
Department
Engineering 80000
HR 102000
IT 135000
Name: Salary, dtype: int64
Real-Time Example 8: Sorting and Ranking Datasets
Sort records by column values in descending order to identify top performers.
Python
import pandas as pd
df = pd.DataFrame({
"Salesperson": ["Rahul", "Priya", "Amit", "Sneha"],
"TotalSales": [45000, 92000, 67000, 81000]
})
# Rank salespersons by total sales descending
ranked_df = df.sort_values(by="TotalSales", ascending=False)
print("Salespersons Ranked by Performance:\n", ranked_df)
Output:
Salespersons Ranked by Performance:
Salesperson TotalSales
1 Priya 92000
3 Sneha 81000
2 Amit 67000
0 Rahul 45000
Real-Time Example 9: Exporting Analyzed Datasets
Export cleaned and analyzed DataFrames back into CSV or JSON files for reporting.
Python
import pandas as pd
df = pd.DataFrame({
"Metric": ["Total Users", "Active Sessions", "Conversion Rate"],
"Value": [12500, 4200, "3.4%"]
})
# Exporting analyzed results
df.to_csv("analytical_report.csv", index=False)
print("Analyzed report exported successfully to 'analytical_report.csv'.")
Output:
Analyzed report exported successfully to 'analytical_report.csv'.