# Haskell AutoBench quick tutorial

**First, check out the README** and clone the repository: https://github.com/mathandley/AutoBench.
Autobench should be manually compiled from sources. The compilation itself should not be hard. After compiling try to run the test example given in the repository:

If everything works, you are lucky. If not, see the "Issues" section.

## How to run?
Run the benchmark with the following command:
`stack exec -- AutoBench ""<path-to-file>.hs"`

## What is a benchmark?
Benchmark is a Haskell module, which consists of benchmarkable programs, benchmark entrypoint and configs.
If program's names are not provided explicitly via configuration, AutoBench will consider each function inside the module as benchmarkable.
Benchmarkable programs should be of exact same types.
Documentation for the entry point and configuration can be found either in repository readme or in the source code in form of comments.

## How to allow automated benchmarking?
To create automatic benchmarks, AutoBench needs to know, how to generate inputs for benchmarkable programs.
To make such generation possible, you need to make an instance of `Arbitrary` for input types and `NFData` for output types.

## Can I plot a graph of several inputs (i.e. for function `Int -> Int -> Int`)? 
No. AutoBench does not allow such things yet. Use tuple: `(Int, Int) ->`. Graph will be 2-dimensional, but atleast it *will be* :).



## Issues.

1. If you are getting the following error message
![](https://i.imgur.com/j8DmzqU.jpg)
delete file `HSinteger-gmp-1.0.2.0.o` (the one that causes the problem, details [here](https://gitlab.haskell.org/ghc/ghc/-/issues/15105)). Then you should be able to atleast run the benchmark without errors.


2. **On MacOS** the problem you might encounter is a log that says *"No valid test suites."*:
![](https://i.imgur.com/TmmStKm.jpg)
This is tricky, because the problem is the runtime error which is not logged normally. By debugging AutoBench itself (or by running `stack repl`, creds to @fizruk31337), you can find out that GHC cannot load native *Accelerate* library: `GhcException "can't load framework: Accelerate (not found)"`. 
This issue is [reported](https://gitlab.haskell.org/ghc/ghc/-/issues/15105) and resolved in newer GHC versions, but in case you cannot upgrade for any reason, [here is a fix](https://github.com/yairchu/macos11-haskell-workaround). Clone the repo and follow the README. 
After completing the steps, try running AutoBench again:
`DYLD_INSERT_LIBRARIES=`pwd`/macos11ghcwa.dylib stack exec -- AutoBench <benchmark_filename>.hs`