Generators and Iterators

Iterators

An iterator is an object that represents a stream of data. It allows you to traverse through all the elements of a collection (like a list, tuple, string, or custom object) one by one, without needing to know the underlying structure of that collection. In Python, objects that implement the iterator protocol are considered iterators.

The iterator protocol consists of two methods:

1. `__iter__()`: This method returns the iterator object itself. It's called when an iterator is needed, for example, at the beginning of a `for` loop.
2. `__next__()`: This method returns the next item from the sequence. If there are no more items, it must raise the `StopIteration` exception, signaling that the iteration is complete.

Any object that implements these two methods is an iterator. Iterators are fundamental to Python's `for` loops, comprehensions, and many other built-in functions.

Key benefits of iterators include:
- Memory Efficiency: Elements are produced on demand, rather than creating the entire sequence in memory at once. This is crucial for very large or infinite sequences.
- Sequential Access: They provide a standard way to access elements sequentially.

Generators

Generators are a simpler and more elegant way to create iterators in Python. A generator is a function or expression that returns an iterator. While a regular function computes a value and returns it, a generator 'yields' a sequence of values over time.

There are two main ways to create generators:

1. Generator Functions: These are defined like regular functions but use the `yield` keyword instead of `return` to produce a result. When a generator function is called, it doesn't execute its body immediately; instead, it returns a generator object (an iterator). The function's state is paused after each `yield` and resumed from that point the next time `next()` is called on the generator object.
2. Generator Expressions: These are similar to list comprehensions but use parentheses instead of square brackets. They create a generator object directly without needing to define a full function.

Key benefits of generators:
- Syntactic Simplicity: They make it much easier to write iterators, especially for complex sequences, as you don't need to manually manage `__iter__`, `__next__`, and `StopIteration`.
- Memory Efficiency: Like other iterators, they produce values lazily, one at a time, making them ideal for large datasets.
- Readability: The code often looks cleaner and more direct than a custom iterator class.

Relationship: All generators are iterators, but not all iterators are generators. Generators are a specific, convenient type of iterator that uses `yield`.

import sys

 --- Custom Iterator Example ---
class MyRangeIterator:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current < self.end:
            value = self.current
            self.current += 1
            return value
        else:
            raise StopIteration

print("--- Custom Iterator Usage ---")
my_iterator = MyRangeIterator(1, 5)
for num in my_iterator:
    print(num)

 You can also manually call next()
print("\nManually calling next() on a new iterator:")
my_iterator_manual = MyRangeIterator(10, 13)
print(next(my_iterator_manual))
print(next(my_iterator_manual))
print(next(my_iterator_manual))
 print(next(my_iterator_manual))  This would raise StopIteration


 --- Generator Function Example ---
def my_generator_range(start, end):
    current = start
    while current < end:
        yield current   Pauses execution and returns 'current'
        current += 1    Resumes from here on next call

print("\n--- Generator Function Usage ---")
gen = my_generator_range(1, 5)
for num in gen:
    print(num)

 You can also manually call next() on a generator object
print("\nManually calling next() on a new generator:")
gen_manual = my_generator_range(10, 13)
print(next(gen_manual))
print(next(gen_manual))
print(next(gen_manual))
 print(next(gen_manual))  This would raise StopIteration


 --- Generator Expression Example ---
print("\n--- Generator Expression Usage ---")
gen_exp = (x - x for x in range(5))
print(type(gen_exp))  <class 'generator'>
for val in gen_exp:
    print(val)


 --- Memory Efficiency Comparison (Illustrative) ---
def create_list_of_squares(n):
    return [i - i for i in range(n)]

def create_generator_of_squares(n):
    for i in range(n):
        yield i - i

list_obj = create_list_of_squares(1000000)
gen_obj = create_generator_of_squares(1000000)

print(f"\nSize of list (1M elements): {sys.getsizeof(list_obj)} bytes")
print(f"Size of generator object (1M elements): {sys.getsizeof(gen_obj)} bytes")
 The generator object itself is very small, as it doesn't store all elements in memory.
 It generates them on the fly when iterated over.