---
# System prepended metadata

title: CS4248 G11 Group Project

---

# CS4248 G11 Group Project
The implementation of this project is based on Frustratingly Easy System Combination for Grammatical Error Correction (ESC). [Github](https://github.com/nusnlp/esc).

>Frustratingly Easy System Combination for Grammatical Error Correction
>[Muhammad Reza Qorib](https://mrqorib.github.io/), [Seung-Hoon Na](https://nlp.jbnu.ac.kr/~nash/faculty.html), and [Hwee Tou Ng](https://www.comp.nus.edu.sg/~nght/)
>2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) ([PDF](https://mrqorib.github.io/assets/pdf/ESC.pdf))



## Installation
This code should be run with Python 3.6. The reason Python 3.6 is needed is because the ERRANT version that is used in the BEA-2019 shared task (v2.0.0) is not compatible with Python >= 3.7

Install this code dependencies by running:
```.bash
pip install -r requirements.txt
python -m spacy download en
wget https://www.comp.nus.edu.sg/~nlp/sw/m2scorer.tar.gz
tar -xf m2scorer.tar.gz
```
## Execution

### Method 3.1: Soft activation during training

The correspoding file is `run_soft_disc.ipynb`.

Directly execute it under the environment with ipykernel.

### Method 3.2: Consider base models’ accuracy for each edit type

The correspoding file is `run_typeacc.py`.

For the CoNLL-2014 experiment, run: `export EXP_DIR=conll-exp` .

For the BEA-2019 experiment, run: `export EXP_DIR=bea-exp` .
1. Run the training command: 
```.bash
python run_typeacc.py --train --data_dir $EXP_DIR/dev-text --m2_dir $EXP_DIR/dev-m2 --model_path $EXP_DIR/models --vocab_path $EXP_DIR/vocab.idx
```
2. Get the prediction on BEA-2019 Dev:
```.bash
python run_typeacc.py --test --data_dir $EXP_DIR/dev-text --m2_dir $EXP_DIR/dev-m2 --model_path $EXP_DIR/models/model.pt --vocab_path $EXP_DIR/vocab.idx --output_path $EXP_DIR/outputs/dev.out
```
3. Get the F0.5 development score:
```.bash
errant_parallel -ori $EXP_DIR/dev-text/source.txt -cor $EXP_DIR/outputs/dev.out -out $EXP_DIR/outputs/dev.m2
errant_compare -ref bea-full-valid.m2 -hyp $EXP_DIR/outputs/dev.m2
```
4. Get the test prediction:
```.bash
python run_typeacc.py --test --data_dir $EXP_DIR/test-text --m2_dir $EXP_DIR/test-m2 --model_path $EXP_DIR/models/model.pt --vocab_path $EXP_DIR/vocab.idx --output_path $EXP_DIR/outputs/test.out
```
7. [Evaluate](#evaluation) the test prediction. Replace test_output with $EXP_DIR/outputs/test.out

### Method 3.3: Restricted Boltzmann machine

The correspoding file is `run_rbm.ipynb`.

Directly execute it under the environment with ipykernel.

### Method 3.4: Weight discount on logistic regression model

The correspoding file is `run_soft_disc.ipynb`.

Directly execute it under the environment with ipykernel.

## Evaluation
- For CoNLL-2014 (requires Python 2.x):
```.bash
python2 m2scorer/scripts/m2scorer.py test_output conll14st-test-corrected.m2
```
- For BEA-2019:
Compress the `test_output` into test.zip, then upload the zip file (only containing the prediction file without any folder or any other file) to https://codalab.lisn.upsaclay.fr/competitions/4057#participate
