# Understanding Variance in Python

Python added type hints in version 3.5. Many developers know basic typing, but tricky stuff like variance can be confusing. This can cause problems with complex types. Developers might see weird error messages about variance if they don't understand it.

Learning about variance helps write better Python code. It leads to fewer bugs and follows good coding practices.

## Generic Types

A generic type serves as a template for creating specific types. The key feature that makes a generic type is its ability to accept different type arguments:

# `list` is a generic type so we can define list[int], list[str], list[float], ...
int_list: list[int] = [1, 2, 3]
str_list: list[str] = ["a", "b", "c"]
float_list: list[float] = [1.0, 2.5, 3.7]

In Python, common generic types include list, dict, and tuple. Here's an overview of how these generic types work:

numbers: list[int] = [1, 2, 3, 4, 5]
student_grades: dict[str, float] = {"Alice": 85.5, "Bob": 92.0, "Charlie": 78.5}
person_info: tuple[str, int, bool] = ("Alice", 30, True)

## Variance Definition

Variance describes how generic types are related based on how their type arguments are related (like subtypes or supertypes).

1. **Covariant**: Preserves the original subtyping relationship. If T is a subtype of U, then G[T] is a subtype of G[U].
2. **Contravariant**: Reverses the original subtyping relationship. If T is a subtype of U, then G[U] is a subtype of G[T].
3. **Invariant**: Disregards any subtyping relationship. G[T] and G[U] are unrelated regardless of the relationship between T and U.

Note: In generic types with multiple type parameters, each parameter can have different variance.

## Default Behavior of Python TypeVar: Invariance

In Python, when you create a TypeVar without specifying its variance, it defaults to being invariant. This means that if you have two types A and B where B is a subtype of A, a generic type using this TypeVar will not allow substitution between Container[A] and Container[B] in either direction.

Example:

from typing import TypeVar, Generic

T = TypeVar('T')  # By default, this is invariant

class Container(Generic[T]):
    def __init__(self, item: T):
        self.item = item

class Animal:
    pass

class Dog(Animal):
    pass

def process_animal_container(container: Container[Animal]):
    # Process the container
    pass

animal_container = Container[Animal](Animal())
dog_container = Container[Dog](Dog())

process_animal_container(animal_container)  # This is fine
process_animal_container(dog_container)  # Type error!

In this example, passing a `Container[Dog]` to a function expecting `Container[Animal]` results in a type error, even though `Dog` is a subtype of `Animal`.

## Covariance

Covariance allows a more derived type (subtype) to be used where a less derived type (supertype) is expected. It's often used with immutable containers or read-only operations on generic types.

Example:

from typing import TypeVar, Generic

T = TypeVar('T', covariant=True)

class Animal:
    def make_sound(self):
        return "Some animal sound"

class Dog(Animal):
    def make_sound(self):
        return "Woof!"

class AnimalShelter(Generic[T]):
    def __init__(self, animals: list[T]):
        self.animals = animals

    def release_animal(self) -> T:
        return self.animals.pop()

def release_any_animal(shelter: AnimalShelter[Animal]) -> None:
    animal = shelter.release_animal()
    print(f"Released an animal that says: {animal.make_sound()}")

dog_shelter = AnimalShelter[Dog]([Dog(), Dog()])
release_any_animal(dog_shelter)  # This works fine due to covariance

In this example, we can use `AnimalShelter[Dog]` where `AnimalShelter[Animal]` is expected.

## The Case for Contravariance

Contravariance is less intuitive than covariance but is crucial in certain scenarios, particularly with function arguments. Let's explore this concept using a food and animal example.

### Example: Animals and Food

Consider the following class hierarchy:

class Food:
    pass

class DogFood(Food):
    def dog_nutrients(self):
        return "Special dog nutrients"

class Animal:
    def eat(self, food: Food):
        print("Animal eats food")

class Dog(Animal):
    def eat(self, food: DogFood):  # This is the problematic override
        print(f"Dog eats {food.dog_nutrients()}")

def feed_animal(animal: Animal):
    regular_food = Food()
    animal.eat(regular_food)

dog = Dog()
feed_animal(dog)  # Runtime error: 'Food' object has no attribute 'dog_nutrients'

In this example:
- `DogFood` is a subtype of `Food`
- `Dog` is a subtype of `Animal`

However, using a `Dog` where an `Animal` is expected causes a runtime error. This happens because `Dog.eat()` expects `DogFood`, but receives `Food`.

For proper inheritance, child class methods should work wherever parent methods work. This means child methods must be subtypes of parent methods.

In fact, `Animal.eat` is a subtype of `Dog.eat`. `Callable[[Food], None]` is a subtype of `Callable[[DogFood], None]`. This shows the contravariance of function arguments:

def feed_any_food(food: Food) -> None:
    print("Feeding any food")

def feed_dog_food(food: DogFood) -> None:
    print(f"Feeding dog food with {food.dog_nutrients()}")

# This works:
dog_feeder: Callable[[DogFood], None] = feed_any_food

# This doesn't work:
any_feeder: Callable[[Food], None] = feed_dog_food  # Type error

### The Correct Implementation

To implement a system where Dog only eats DogFood while maintaining a proper class hierarchy, we can use contravariance type variable. Here's a better way to structure the code:

from typing import TypeVar, Generic

class Food:
    pass

class DogFood(Food):
    def dog_nutrients(self):
        return "Special dog nutrients"

F = TypeVar('F', contravariant=True)

class Animal(Generic[F]):
    def eat(self, food: F):
        print("Animal eats food")

class Dog(Animal[DogFood]):
    def eat(self, food: DogFood):
        print(f"Dog eats {food.dog_nutrients()}")

def feed_animal(animal: Animal[Food]):
    regular_food = Food()
    animal.eat(regular_food)

# This will work
general_animal = Animal[Food]()
feed_animal(general_animal)

# This will cause a type error at compile-time
dog = Dog()
# feed_animal(dog)  # Uncommenting this line will show a type error

This implementation:

1. Uses generics with a contravariant type variable `F`.
2. Makes `Animal` a generic class that can work with any subtype of `Food`.
3. Specifies `Dog` as `Animal[DogFood]`, meaning it can only eat `DogFood`.
4. The `feed_animal` function expects `Animal[Food]`, which can accept any `Animal` that can eat `Food` or its supertype.

Now, trying to pass a `Dog` to `feed_animal` will cause a type error at compile-time, preventing runtime errors. This approach maintains type safety while allowing `Dog` to be specific about what it eats.

To use a `Dog` safely, you would need to provide it with `DogFood`:

def feed_dog(dog: Dog):
    dog_food = DogFood()
    dog.eat(dog_food)

dog = Dog()
feed_dog(dog)  # This works fine

This structure respects the contravariant relationship between `Animal` and `Food` types, ensuring type safety and proper object-oriented design.

## Simple Method Inheritance vs Generic Inheritance

While generic inheritance is powerful, it's not always necessary. Let's compare two scenarios to understand when to use simple method inheritance and when to opt for generic inheritance.

### Simple Method Inheritance: File Handler Example

Consider this practical example of file handling:

class FileHandler:
    def read_file(self, file: str) -> str:
        with open(file, 'r') as f:
            return f.read()

class SmartFileHandler(FileHandler):
    def read_file(self, file: str | bytes) -> str:
        if isinstance(file, str):
            return super().read_file(file)
        elif isinstance(file, bytes):
            return file.decode('utf-8')

In this case, simple method inheritance works well because:

1. The child class (`SmartFileHandler`) extends functionality while maintaining compatibility with the parent class (`FileHandler`).
2. The wider argument type in the child class (`str | bytes`) includes the parent class's type (`str`).
3. Any code expecting a `FileHandler` can work with a `SmartFileHandler` without issues.

### Generic Inheritance: Animal and Food Example

Now, let's revisit our animal and food example using generic inheritance:

from typing import TypeVar, Generic

F = TypeVar('F', contravariant=True)

class Animal(Generic[F]):
    def eat(self, food: F):
        print("Animal eats food")

class Dog(Animal[DogFood]):
    def eat(self, food: DogFood):
        print(f"Dog eats {food.dog_nutrients()}")

class Food:
    pass

class DogFood(Food):
    def dog_nutrients(self):
        return "Special dog nutrients"

In this scenario, generic inheritance is necessary because:

1. We want to enforce type safety at a more granular level.
2. Different animal types might have specific dietary requirements.
3. We need to prevent runtime errors when feeding animals with incompatible food types.

### Why the Difference?

The file handler example uses simple inheritance because:
- The child class's functionality is a superset of the parent's.
- The child class can handle all inputs the parent can, plus more.
- There's no risk of runtime errors due to incompatible types.

The animal/food example requires generic inheritance because:
- Different animal subclasses might have incompatible dietary needs.
- We want to catch type mismatches at compile-time rather than runtime.
- The relationship between animal types and food types is more complex and needs to be explicitly defined.

In summary, use simple inheritance when extending functionality in a way that maintains full compatibility with the parent class. Use generic inheritance when you need to enforce more complex type relationships and prevent potential runtime errors due to type mismatches.