Master arithmetic operations on Pandas DataFrames for efficient data analysis and manipulation in Python. Learn scalar & inter-DataFrame calculations.

Arithmetic Operations on Pandas DataFrames

The Pandas DataFrame is a powerful two-dimensional, labeled data structure essential for data analysis and manipulation in Python. A key strength of DataFrames lies in their ability to perform vectorized arithmetic operations efficiently, allowing computations on entire datasets without the need for explicit loops.

This guide covers how to apply arithmetic operations in Pandas, including:

Operations with scalar values
Operations between two DataFrames
Utilizing built-in arithmetic functions

1. Arithmetic Operations with Scalar Values

Pandas allows you to perform arithmetic operations directly on DataFrame elements using scalar (single) values. These operations are applied element-wise across all columns and rows.

Common Operators and Their Usage

Operation	Example	Description
Addition	`df + 2`	Adds 2 to every element.
Subtraction	`df - 2`	Subtracts 2 from every element.
Multiplication	`df * 2`	Multiplies every element by 2.
Division	`df / 2`	Divides every element by 2.
Exponentiation	`df ** 2`	Squares each element.
Modulus	`df % 2`	Returns the remainder when divided by 2.
Floor Division	`df // 2`	Performs integer division (discards remainder).

Example: Scalar Operations on a DataFrame

import pandas as pd

# Creating a DataFrame
data = {'A': [1, 2, 3, 4], 'B': [5, 6, 7, 8]}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Performing arithmetic operations
print("\nAddition (df + 2):")
print(df + 2)

print("\nSubtraction (df - 2):")
print(df - 2)

print("\nMultiplication (df * 2):")
print(df * 2)

print("\nDivision (df / 2):")
print(df / 2)

print("\nExponentiation (df ** 2):")
print(df ** 2)

print("\nModulus (df % 2):")
print(df % 2)

print("\nFloor Division (df // 2):")
print(df // 2)

Output:

Original DataFrame:
   A  B
0  1  5
1  2  6
2  3  7
3  4  8

Addition (df + 2):
   A   B
0  3   7
1  4   8
2  5   9
3  6  10

Subtraction (df - 2):
   A  B
0 -1  3
1  0  4
2  1  5
3  2  6

Multiplication (df * 2):
   A   B
0  2  10
1  4  12
2  6  14
3  8  16

Division (df / 2):
   A    B
0  0.5  2.5
1  1.0  3.0
2  1.5  3.5
3  2.0  4.0

Exponentiation (df ** 2):
   A   B
0   1  25
1   4  36
2   9  49
3  16  64

Modulus (df % 2):
   A  B
0  1  1
1  0  0
2  1  1
3  0  0

Floor Division (df // 2):
   A  B
0  0  2
1  1  3
2  1  3
3  2  4

2. Arithmetic Operations Between Two DataFrames

Pandas facilitates element-wise arithmetic operations between two DataFrames. These operations automatically align based on both row indices and column labels.

Important Notes:

Mismatched Labels: If row indices or column labels do not match between the two DataFrames, the resulting cells for those mismatched labels will contain NaN (Not a Number).
Common Labels Only: Operations are exclusively performed on elements where both row index and column labels are present in both DataFrames.

Example: DataFrame to DataFrame Operations

import pandas as pd

# Creating two DataFrames with different indices
df1 = pd.DataFrame({'A': [1, 2, 3, 4], 'B': [5, 6, 7, 8]})
df2 = pd.DataFrame({'A': [10, 20, 30], 'B': [50, 60, 70]}, index=[1, 2, 3]) # Note the index

print("DataFrame 1:")
print(df1)

print("\nDataFrame 2:")
print(df2)

# Performing arithmetic operations
print("\nAddition (df1 + df2):")
print(df1 + df2)

print("\nSubtraction (df1 - df2):")
print(df1 - df2)

print("\nMultiplication (df1 * df2):")
print(df1 * df2)

print("\nDivision (df1 / df2):")
print(df1 / df2)

Output:

DataFrame 1:
   A  B
0  1  5
1  2  6
2  3  7
3  4  8

DataFrame 2:
    A   B
1  10  50
2  20  60
3  30  70

Addition (df1 + df2):
      A     B
0   NaN   NaN
1  12.0  56.0
2  23.0  67.0
3  34.0  78.0

Subtraction (df1 - df2):
      A     B
0   NaN   NaN
1  -8.0 -44.0
2 -17.0 -53.0
3 -26.0 -62.0

Multiplication (df1 * df2):
      A     B
0   NaN   NaN
1  20.0 300.0
2  60.0 420.0
3 120.0 560.0

Division (df1 / df2):
      A      B
0   NaN    NaN
1  0.20  0.100
2  0.15  0.116
3  0.13  0.125

3. Built-in Arithmetic Functions in Pandas

Pandas provides dedicated arithmetic methods that offer greater flexibility and control. These functions are particularly useful for:

Handling Missing Values: Using the fill_value parameter to substitute missing data during operations.
Specifying Axes: Performing operations along specific rows or columns.
Hierarchical Indexing: Working with multi-level indices using the level parameter.

List of Arithmetic Functions:

Function	Description	Equivalent Operator
`add()`	Element-wise addition	`+`
`sub()`	Element-wise subtraction	`-`
`mul()`	Element-wise multiplication	`*`
`div()`	Element-wise division (true division)	`/`
`floordiv()`	Integer division	`//`
`mod()`	Modulus operation	`%`
`pow()`	Exponentiation	`**`
`dot()`	Matrix multiplication	(N/A for element-wise)
`radd()` to `rpow()`	Reverse operations for each arithmetic type	(e.g., `2 + df`)

Example: Using `add()` with `fill_value`

This example demonstrates how add() can be used to add two DataFrames where one DataFrame has fewer rows, and missing values are handled by filling them with 0.

import pandas as pd

df1 = pd.DataFrame({'A': [1, 2, 3], 'B': [3, 4, 5]})
df2 = pd.DataFrame({'A': [10, 20], 'B': [50, 60]}, index=[0, 1]) # df2 is shorter

print("DataFrame 1:")
print(df1)

print("\nDataFrame 2:")
print(df2)

# Using add() with fill_value=0
# Missing values in df2 (for index 2) will be treated as 0 for the addition
result = df1.add(df2, fill_value=0)

print("\nResult of df1.add(df2, fill_value=0):")
print(result)

Output:

DataFrame 1:
   A  B
0  1  3
1  2  4
2  3  5

DataFrame 2:
    A   B
0  10  50
1  20  60

Result of df1.add(df2, fill_value=0):
     A     B
0  11.0  53.0
1  22.0  64.0
2   3.0   5.0

Conclusion

Arithmetic operations are fundamental to data processing and analysis. Pandas makes these operations efficient, readable, and powerful, whether you're working with scalar values or performing computations between DataFrames.

Key Takeaways:

Scalar Operations: Applied element-wise across the entire DataFrame.
DataFrame-to-DataFrame Operations: Automatically align on row indices and column labels. Mismatched labels result in NaN.
Built-in Methods: Functions like add(), sub(), mul(), etc., offer greater control, especially for handling missing data (fill_value) and specifying operational axes.

By mastering these arithmetic operations, you can significantly streamline your data manipulation tasks and perform advanced analytics efficiently using Pandas.

Pandas DataFrame Arithmetic: Fast Data Operations

Arithmetic Operations on Pandas DataFrames

1. Arithmetic Operations with Scalar Values

Common Operators and Their Usage

Example: Scalar Operations on a DataFrame

2. Arithmetic Operations Between Two DataFrames

Important Notes:

Example: DataFrame to DataFrame Operations

3. Built-in Arithmetic Functions in Pandas

List of Arithmetic Functions:

Example: Using `add()` with `fill_value`

Conclusion

Key Takeaways:

On this page