# Exception hierarchy & error handling

###### tags: `cycle 15`

- Shaped by: @petiaccja
- Appetite (FTEs, days): ???
- Developers: @petiaccja
- Available for discussion/review: @egparedes

## Problem

Error handling in GT4Py at the moment is confusing both for users and developers:

For developers:
- The same exception classes are redefined in multiple submodules
- FieldOperatorSyntaxError & ProgramSyntaxError: these two classes serve the same purpose, they are just used in different modules.
- Solution: share the exception classes accross modules
- Exception classes are named after the call stack they're raised from and not the error that signal
- FieldOperatorTypeDeductionError, ProgramTypeError: these class names are specific to a certain module, but cover a very wide range of errors.
- Solution: name exception classes based on the specific error they represent, not based on which module raises them.
- Complementary error handling utilities, like formatting all messages uniformly or getting the matching line from the sources is not available

For users, resulting from poor infrastructure for developers:
- Difficult to interpret the error messages:
- The following is shown when adding trying to add a field and a tuple: `gt4py.next.common.GTTypeError: Can not unambiguosly extract data type from tuple[Field[[IDim, JDim, KDim], int64], Field[[IDim, JDim, KDim], int64]]!`. No source location, no reference to the addition operation, data type is not applicable to tuples.
- Solution: use a meaningful and clearly formatted error message: `gt4py.next.errors.IncompatibleArgumentsError: my/file.py(132:14) : error: incompatible arguments to binary operator +: tuple[Field[[IDim, JDim, KDim], int64], Field[[IDim, JDim, KDim], int64]], Field[[IDim, JDim, KDim], int64]`
- Unexpected failures without any explanation:
- When compilation fails with a common exception (like ValueError or KeyError) that gives you no message about the validity of the source code
- Solution: check the validity of the input IR and throw a meaningful error instead of proceeding without having met the preconditions for the compiler pass
- Inconsistent formatting of messages
- The above type deduction error does not have source location information
- An error produced for an undefined symbol does have source location, but it does not print line numbers, it only prints the faulty line itself
- Solution: all error messages should have source location information and they should follow the same format

## References

We should try follow the standard conventions to deal with exceptions in Python. As a first step, here is a collection of relevant references to read and think about:

1. The [_"Programming Recommendations"_](https://peps.python.org/pep-0008/#programming-recommendations) section of **PEP 8** says the following about exceptions:

- Derive exceptions from `Exception` rather than `BaseException`. Direct inheritance from `BaseException` is reserved for exceptions where catching them is almost always the wrong thing to do.
- Design exception hierarchies based on the distinctions that code *catching* the exceptions is likely to need, rather than the locations where the exceptions are raised. Aim to answer the question “What went wrong?” programmatically, rather than only stating that “A problem occurred” (see [PEP 3151](https://peps.python.org/pep-3151) for an example of this lesson being learned for the builtin exception hierarchy).
- Class naming conventions apply here, although you should add the suffix “Error” to your exception classes if the exception is an error. Non-error exceptions that are used for non-local flow control or other forms of signaling need no special suffix.
- Use exception chaining appropriately. `raise X from Y` should be used to indicate explicit replacement without losing the original traceback. When deliberately replacing an inner exception (using `raise X from None`), ensure that relevant details are transferred to the new exception (such as preserving the attribute name when converting `KeyError` to `AttributeError`, or embedding the text of the original exception in the new exception message).

2. The [_Section 2.4 Exceptions_](https://google.github.io/styleguide/pyguide.html#s2.4-exceptions) of the _Google Python Style Guide_ (adopted in our coding conventions) says:

- Make use of built-in exception classes when it makes sense...
- Libraries or packages may define their own exceptions. When doing so they must inherit from an existing exception class. Exception names should end in `Error` and should not introduce repetition (`foo.FooError`).

Other references which might be interesting for this task are:

3. [StackExchange - What is considered best practice for custom exception classes?](https://softwareengineering.stackexchange.com/questions/310415/what-is-considered-best-practice-for-custom-exception-classes)

It basically follows PEP8 recommendations:
- Errors that must be handled the same way should be the same class.
- Errors that you don't see a good reason to handle separately can be the same class until you find a reason.
- Errors that a user might sometimes have good reason to distinguish should be distinct classes.
+ If one error is a special case of a more general error that you have, make the former a subclass of the latter.
+ If they're just different errors, don't have a subclass relationship between them.

4. [Should we use custom exceptions in Python?](https://towardsdatascience.com/should-we-use-custom-exceptions-in-python-b4b4bca474ac)

It's hard to summarize because it discusses different approaches and topics related to exceptions (e.g. `raise from`), but it feels more or less aligned with the previous references.

5. [PEP 654 - Exception Groups and except*](https://peps.python.org/pep-0654/). This is mostly only relevant for the topic of grouping exceptions in python3.11, which seems to be out of scope for this task, but I think it's anyway worth it to take a look to the PEP. Note that this feature originates from the [trio](https://trio.readthedocs.io/en/stable/) async library which first implemented the [MultiError](https://github.com/python-trio/trio/blob/master/trio/_core/_multierror.py) class. A library backporting similar functionality from 3.11 to previous Python versions is also available ([exceptiongroup](https://pypi.org/project/exceptiongroup/)). [PEP 678 – Enriching Exceptions with Notes](https://peps.python.org/pep-0678/) is a later addition to `ExceptionGroups` which allows adding custom notes to the exceptions inside the group.

6. [Beautiful tracebacks in Trio v0.7.0](https://vorpus.org/blog/beautiful-tracebacks-in-trio-v070/) is an interesting example of how to design useful and concise error messages and tracebacks for exceptions.

Out of scope for this project, but worth mentioning for future enhancements, it's worth mentioning exception hooks:

7. [Creating Beautiful Tracebacks with Python's Exception Hooks](https://martinheinz.dev/blog/66)

## Solution

The primary goal of this project is to create an infrastructure that supports error handling accross the codebase, the details of which are outlined in the subsequent points.

### Error handling module

An error handling module should be created on the top-level of GT4Py(`.next`). This module provides infrastructure for both compiler diagnostics and general error handling, however the focus is on the compiler diagnostics.

Find a good module name:
- `errors`?
- `exceptions`?
- Whatever other libraries use, maybe there is a convention

### Refactoring dependent structures

Certain data structures (e.g. source location) are shared between IRs and error handling. In case these are defined in the IR, they have to be moved to a globally accessible module. This project should find and resolve these issues if there are any.

### Facilities shared through error handling

The following facilities need to be implemented or reused if already available:
- Data structure for source location
- Use `SourceLocation` from `eve`
- Formatting:
- Source locations
- Types in the IR: use a concise, consistent formatting
- Finding source files and reading in lines at location
- Consider using Python's SyntaxError to handle formatting
- The main goal is to have consistent and centralized formatting, the actual formatting itself is second in importance
- (optional, low-priority) User-accessible config to customize error message formatting
- Some people might want to see error messages formatted like GCC or Clang to better interface with their IDEs or their users

### Exception hierarchy

The following serves as a basis, but should be extended during implementation as necessary:
- `builtin.Exception/builtin.SyntaxError`
- `errors.CompilationError`
- `errors.UndefinedSymbolError`
- `errors.IncorrectArgumentCountError`
- `errors.IncorrectArgumentTypeError`
- `errors.IncompatibleArgumentsError`
- ...

#### Rationale

Best practices for exception handling:
1. **Create a specific exception class for a specific error condition:**<br>For example, if a file does not exist, create a `FileNotFoundError` class for it. If the process does not have sufficient privileges to open the file, create a `FilePrivilegeError` class for it. This way, the code that handles the error can easily differentiate between the two error conditions, and inform the user that the file does not exist or prompt the user to elevate privileges.
2. **Include relevant information in the exception class:**<br>Both file handling error classes above should include the path to the file, and the privilege error can also include the privilege. This information may be useful when handling the error.
3. **Include error messages in the exception class:**<br>In case the exception has to be displayed as a message to the user or to the developer, let the exception class create a meaningful message. The `FileNotFoundError`, using the file path stored in it, may convert itself to a string like `"file not found: /my/file.txt"`. The advantage of having the error message provided by the class is that you don't have to type it every time you raise that error, and the message will always be the same for the same error for whoever reads it.
4. **Do not group exception classes by their source component**:<br>The same type of error may occur in multiple modules, you would have to duplicate the exception class' code in each. Example (computer game): do not do `raise GraphicsError("key not found in dictionary)"`, `raise PhysicsError("key not found in dictionary)"`, just use `KeyError` in both modules.

There are several other concerns too, like using exceptions as control flow, but they are not prevalent in our codebase.

Sources:
- https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#S-errors
- https://howtodoinjava.com/java/exception-handling/best-practices-for-for-exception-handling/
- Uncle Bob's *Clean Code*

#### Documentation, coding guidelines

A section on exceptions could be a useful addition to the GT4Py coding guidelines. This section could explain the general best practices for exceptions, contain links to external references, and explain how exceptions are used within GT4Py. Writing this is a secondary goal of this project.

## Non-goals

This project does **not** aim to:
- Implement any sort of logging
- Implement compiler warnings
- Implement aggregate handling of multiple compiler errors/warnings

### Rabbit holes

#### Fixing existing error handling code

The primary goal of this project is to create the infrastructure for well-organized error handling. Modifying existing code to use the new infrastructure is only a secondary goal:
- The examples in the document are to provide context on the shortcomings of the current system and guidance on how to build the infrastrcture, they are not existing code that must be fixed in this project
- It's great if you fix existing code and make it use the new exceptions, but don't spend too much time on fixing existing code that's complicated
- If you choose not to fix existing code, you can still create the appropriate exception classes to use in the future, but only if you are absolutely certain the exception classes will be useful (see https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it)