---
# System prepended metadata

title: Ignota DSG Day 3

---

# Ignota DSG Day 3

## Transformer Models
- Created new transformer branch
- New scripts for molecular embeddings based on ChemBERTa-zinc250k-v1 pre-trained models from Hugging Face (https://huggingface.co/seyonec/ChemBERTa-zinc250k-v1) in "/src/representation/molecular_embeddings.py"
- Molecular embedding files have been exported and can be found in "/shared/". Each file is within the corresponding dataset file name under "embeddings_molecular_level_dataframe.csv"
- Scripts & functions for extracting SDF files for x,y, & z atom 3D coordinates in 

Discussion points for the report:
- The size of the embedding vector is 768. Hence, the transformer might work better for models using larger dataset & may not be very useable for smaller datasets
- Smile2vec: there is a proof of concept github repo available, but we would need more time to implement this. Smile2vec could be a possible future exploration, integrating other descriptive features.

Herg dataset with scaffold split:
- The following results use scaffold/fingerprint splits on fingerprint datasets. These are a single vector (i.e. NOT PROPER EMBEDDINGS).

![herg_scaffold_confusion](https://hackmd.io/_uploads/Hk_qbCaHp.png)

Test Accuracy: 0.7379
Test Precision: 0.0489
Test Recall: 0.2439
Test F1 Score: 0.0814

Herg dataset with fingerprint split:

![herg_fingerprint_confusion](https://hackmd.io/_uploads/ByrG9CaBT.png)


Test Accuracy: 0.7254
Test Precision: 0.1064
Test Recall: 0.2261
Test F1 Score: 0.1447


## Links
- [Github](https://github.com/alan-turing-institute/DSGDec2023Ig/tree/main/src/visulisations)
- [Report](https://www.overleaf.com/project/6565cf556736d9e50c1ad129)
- [Day 2 notes](https://hackmd.io/0qNBkU0HRXKf3Jrgsl7HiQ)
- [Day 1 notes](https://hackmd.io/a8bFPxfuQgm9HHT9rirXwg)


## Images
- Script that generates images based on pre-trained model
- Pre-trained model preprocessing removes salts and stereochemistry, our model will retain them
- Use weights from ImageMol as starting point and train on our data, since our data preprocessing is different
- Resolution 224 x 224
- All images for herg generated, found in /share
- In future work may be worth generating at images without stereochemistry for better compatibility with Imagemol


## Graphs
- Input data sets generated
- Won't have time to do both graphormer and EGNN
- Goal to finish by tomorrow lunch

## Transformers
- Pre-trained tokenizer to convert smiles to tokens
- Smiles2vec POC github repo
- Wrap up results for report and join graph team

## Smiley faces :)

:) :-) 


## GeoGNN

1. Created input SMILES datasets for converting into graphs and the following features:

