###### tags: `Sprint Diary`
# 4DN2 SPRINT DIARY - Q1S1
## 17 Jan. 2023 / 10:00 AM
## Timeline

Attendees:
* Nezar @nvictus
* Vedat @VOY
* Jiangyuan
* Johan
* Trevor

## Resources
* Sprint Slides for Figures: https://docs.google.com/presentation/d/15f1CvLSlRxAC3k5dvzhdoVkWNJnDLSb4ox_0HzxHKdk/edit?usp=sharing/
* Kanban Board: https://airtable.com/shrNznOkqe3JGugKK
* Github: https://github.com/abdenlab/4dn2-sprints.git
* Whiteboard: https://miro.com/app/board/uXjVPxut4LQ=/?share_link_id=732063246408
* Action Items:
* Sprint Goals:

## Agenda
Previous Items:
* Planned:
* Trajectories of RNAseq; Relationship to epigenetic compartmentalization.
* Insulation Scores and Short range interactions, TADs, Hi-C Low dimensional features
* ATAC-Seq peaks across the developmental stages.
* In Progress:
* Consolidated Epigenetic racks
* Improvements on IPG Clustering
* Dot calling for newly mapped Hi-C Lanes:
* RNAseq: Pairwise DEx
~~* RNAseq: Clustering on the latent Space, Results from Sashata needs details.~~
*
Sprint Items:
1. Dot calling for newly mapped Hi-C Lanes with Jiangyuan
2. Higlass and widgets with Trevor

Parking Lot:
1. No parked items

Notes:

## Plan

## Huddle Day Notes
### Morning
* Dot Calling (Discussion with Jiangyuan and Johan)
* Jiangyuan will do a comparison of the union lists prev vs current, e.g. Upset plots.
* Johan will make a megamap with all conditions together for standard dot calling.

* HiGlass (Trevor):
* polish up `hg` for release and convert the current `higlass-python` repo.
* Add boilerplate logic for jupyter-scatter integration to `hg`.
* Release notes for `anywidget`
* Expose higlass's Jupyter widget using `anywidget` and deprecate `higlass-widget`

### EOD (around 4pm)
* Trevor: Blog post with widgets are on the way.
* Jiangyuan: Dot-calling will compare union dots, For RNAseq DEx; some NF-core output was not integers. Might round up to the nearest integer.

# Sprint Notes
### Trevor
Released `anywidget` and blog post!!! CONGRATS!!! https://anywidget.dev/blog/introducing-anywidget

### Jiangyuan
Figures on [Slack](https://4dn2-csg.slack.com/archives/C03S6H86WCT/p1674232991733099) and [Sprint Slides](https://docs.google.com/presentation/d/15f1CvLSlRxAC3k5dvzhdoVkWNJnDLSb4ox_0HzxHKdk/edit?usp=sharing/).
He focused on clusters of enriched pixels at 10 and 5 kb resolutions separately (left and middle plots (first two)). Pixels in each cluster could be enriched in one or more conditions. I just combined all their enriched conditions to represent each cluster.
The right plot is the original one. After I merged 5 and 10kb dot calls, I used bioframe.cluster(min_dist=None).
7089 common dots across all 5 stages vs 1537 is quite an improvement!
Next step is to know how to select the brightest pixel in each cluster to represent the whole cluster as a dot (e.g., max sum or just max of max across 5 stages.

#### Nezar's suggestion
This is very encouraging! With the new 10kb clusters, the highest overlap is across all 5 stages. 7089 common dots vs 1537 before.
I think it's also worth trying the approach @Johan Gibcus mentioned on Tuesday that was used in the Reed et al paper: standard dot calling on a composite Hi-C map off all conditions.
@Johan Gibcus - were you able to make a mega-merged map from all 5 stages?
This **might work best** because it directly yields a "union list" of dots after which we can score those individual dot positions across all conditions to decide if a dot is enriched in a condition or not.