In Python, a generator is a special kind of function that can give us a sequence of values one by one. It’s like a magic box that creates values when we ask for them. Generators are really useful when we want to make a big list of things, but we don’t want to keep all of them in our memory at the same time.
Normally, when we write a function, it gives us one value using the “return” word. But with generators, we use the word “yield” instead. This lets us create a bunch of values without needing to store them all at once.
In this easy guide, we will explore Python generators. We’ll learn why they are helpful and how to use them with simple examples. By the end, you’ll see how generators can make your code smarter and more efficient.
Creating Python Generators
To create a generator in Python, we use a special syntax that involves the “yield” keyword. Instead of using the “return” keyword, we use “yield” inside a function to produce a value.
def generator_name():
yield 1
yield 2
yield 3
In this, generator_name() is a generator function that yields three values: 1, 2, and 3. Each time the generator is iterated over, it pauses at the “yield” statement and produces the value.
Differences between Python generators functions and regular functions
Generator functions differ from regular functions in several aspects:
- Generators use the “yield” keyword to produce values, while regular functions use “return” to provide a single value and terminate.
- Generator functions can yield multiple values over time, whereas regular functions return a single value and exit.
- Generators maintain their internal state and resume execution from where they left off when the next value is requested. Regular functions start execution from the beginning each time they are called.
- Generators are memory-efficient because they generate values on-the-fly, only storing the current value and state. Regular functions typically compute and store all values in memory before returning them.
Example of Python generator functions
Let’s say we have a large dataset of customer orders, and we want to calculate the total price of all the orders. Instead of loading the entire dataset into memory, we can use a generator expression to lazily calculate the total price as we iterate over the orders.
orders = [
{'product': 'Apple', 'quantity': 3, 'price': 1.50},
{'product': 'Banana', 'quantity': 2, 'price': 0.75},
{'product': 'Orange', 'quantity': 4, 'price': 0.90},
# ... More orders ...
]
total_price = sum(order['quantity'] * order['price'] for order in orders)
print(f"The total price of all orders is: ${total_price}")
Output:
The total price of all orders is: $9.6In this example, we use a generator expression (order['quantity'] * order['price'] for order in orders) to calculate the price for each order. The generator expression iterates over each order in the orders list, multiplies the quantity by the price, and yields the calculated price.
The sum() function then takes the generator expression as input and iterates over it, lazily obtaining each calculated price and accumulating the total.
Processing Sensor Data in Real-Time with Generator Expressions
Imagine you have a system that constantly receives temperature readings from different sensors. You want to work with these readings in real time without storing all of them in memory. Let’s see how generator expressions can help you with that.
Imagine each sensor sends a temperature value, and we want to process and analyze it as soon as it arrives. We can create a generator that continuously generates random temperature values, simulating the readings from sensors. Here’s an example:
Example
import random
def sensor_data_stream():
while True:
temperature = random.uniform(20, 30) # Simulating temperature readings
yield temperature
sensor_stream = sensor_data_stream()
for temperature in sensor_stream:
# Process the temperature reading in real-time
# Analyze, display, or trigger actions based on the temperature
print(f"Current temperature: {temperature}°C")
Output:
Current temperature: 26.446665671395024°C
Current temperature: 22.44313525700551°C
Current temperature: 23.99167237519518°C
Current temperature: 27.351086299603832°C
Current temperature: 29.19548775403927°C
Current temperature: 29.64366869613123°C
Current temperature: 20.974782931814737°C
Current temperature: 24.510497933398057°C
Current temperature: 23.945359574884257°C
Current temperature: 23.7934202709368°C
Current temperature: 27.498892963266286°C
Current temperature: 25.194920087366107°C
Current temperature: 29.523247662098576°C
Current temperature: 21.80252511990193°C
Current temperature: 27.619083698049796°CIn this example, the sensor_data_stream function represents the continuous stream of temperature readings. We generate random temperature values using random.uniform and yield them one by one. The while True loop ensures that the generator keeps producing temperature values endlessly.
By iterating over sensor_stream, which is the generator created from sensor_data_stream(), we can process each temperature reading in real time. Inside the loop, you can perform various actions based on the temperature, such as analyzing the data, displaying it on a screen, or triggering alerts if the temperature goes above or below a certain threshold.
The generator expression in this example allows us to handle real-time temperature readings without storing them all in memory. It generates temperature values on-the-fly, simulating the continuous stream of sensor data, and enables us to process them as soon as they arrive.
Generator Expressions
Python offers another way to create generators called generator expressions. Generator expressions provide a concise and efficient syntax for generating values on-the-fly without the need for writing a separate function. They are a powerful tool for creating generators in a more compact and readable manner.
Syntax of Python generator expressions
my_generator = (x for x in range(1, 10))
In this, the generator expression (x for x in range(1, 10)) creates a generator that yields values from 1 to 9. We can assign this generator to a variable, my_generator, and iterate over it or use it in other parts of our code.
example of generator expressions
even_numbers = (x for x in range(1, 21) if x % 2 == 0)
for number in even_numbers:
print(number)
Output:
2
4
6
8
10
12
14
16
18
20In this example, the generator expression (x for x in range(1, 21) if x % 2 == 0) generates even numbers from 1 to 20.
Here’s how it works:
range(1, 21)generates a sequence of numbers from 1 to 20.- The
if x % 2 == 0part filters out only the numbers that are divisible by 2 and have no remainder, ensuring that only even numbers are included.
The resulting generator expression even_numbers represents a sequence of even numbers. By iterating over even_numbers using a for loop, we can access and process each even number one at a time. In this case, we simply print each even number.
Advantages of Python Generators
1. Memory efficiency
Generators are smart helpers that save memory. They only create and store values when you actually need them. Instead of keeping all the values in memory at once, generators remember the current value and state. This way, they don’t take up too much space, especially when dealing with lots of data.
2. Lazy evaluation
Generators are like laid-back workers. They don’t rush to calculate or produce all the values right away. They do things lazily, meaning they calculate or generate values only when you ask for them. This lazy approach helps save time and resources because they don’t waste energy on unnecessary work.
3. Infinite sequences
Generators can handle never-ending sequences, like counting forever. They don’t get tired or overwhelmed. You can create a generator that keeps producing values endlessly, and it won’t crash your program or run out of values. Generators adapt to these infinite sequences and provide values smoothly whenever you need them.
Generating Fibonacci series using Python generators
def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
fib_generator = fibonacci()
# Generating the first 10 Fibonacci numbers
for _ in range(10):
fib_number = next(fib_generator)
print(fib_number)
Output:
0
1
1
2
3
5
8
13
21
34In this example, we define a generator function called fibonacci that generates Fibonacci numbers. Inside the function, we initialize variables a and b to the first two numbers of the series (0 and 1). Using an infinite loop (while True), we continuously yield the current Fibonacci number (a) and update the variables to calculate the next number in the series (a, b = b, a + b).
We create a generator object fib_generator by calling the fibonacci function. This generator represents an infinite sequence of Fibonacci numbers.
To generate a specific number of Fibonacci numbers, we can use a loop. In this example, we generate the first 10 Fibonacci numbers by iterating 10 times and calling next(fib_generator) to obtain the next Fibonacci number from the generator. Each generated Fibonacci number is printed.
Image Credit : Photo by Jan Kopřiva
