IDT
===
Sunday
---

Saturday
---

### Usps DTN
BN: Epoch 10 62.68 and up to 64
Transnorm: Epoch 10 45.10 epoch 9 53.02
#### GroupNorm Dead:
GN-4-start-0-idt-16-before: Epoch 10 56.33
GN-4-start-0.1-idt-16-before: Epoch 10 61.64
GN-4-start-0.1-idt-128-before: Epoch 10: 58.93 run2: 61.72****

GN-4-start-0-idt-16-after: Epoch 20 60.36 Epoch 10: 60.33
GN-4-start-0.1-idt-16-after: Epoch 10: 61.54 run2: 58.92
GN-4-start-0.1-idt-128-after: Epoch 10: 60.24 run2: 59.65
GN-4-start-0.1-idt-3-after: Epoch 10: 58.12 run2: 58.98

#### Standard Norm
Group-4-start-0.1-idt-16-Before: Epoch 10: 59.93 run2: 57.40
Group-4-start-5-idt-16-Before: Epoch 10: 62.05 run2: 59.46

Group-4-start-0.1-idt-16-After: Epoch 10: 59.86 run2: 56.96

#### StandardNorm Align2p
Group-4-start-5-idt-16-Before-l2p: Epoch 10: 56.17 run2: 55.39

#### sort
before: 46.21% 42.58%
before group0:
after: 48.56%
after group0: 39.78%

batch 4 gn: 40.56%
normal group 48.56%

mix 50%: 54.13%
mix 10% idted: 64.32%

What can be an intermediate way of the alignment****

Friday
---
### SVHN exp
BN: 52% 22%

AdaBN: Epoch 3: 62% Epoch 20: 52%
AdaBN: Run2 Epoch 3: Epoch 10: 52%

IDT-direct: 28%
Interp-hist-5-95-32: Epoch 3: 57% Epoch 10: 56%
Interp-hist-5-95-32-group4: Epoch 3: 60% Epoch 10: 58%
Interp-direct-space8: Epoch 3: 57% Epoch 10:
Interp-direct-space32: Epoch 3: 61% Epoch 10: 58%

Interp-direct-

Interp-direct-p::8::16: 62% 57%

Interp-direct-p::8::16: Use the same stats: 60% 60%

IDT-interp ::32 src2tar: using same BN stats: 61.78% 61.84%
IDT-interp ::128 src2tar: using same BN stats:
IDT-interp ::16 src2tar: using same BN stats:

IDT-interp-percentile-32 src2tar: using same BN stats: 63.27%

IDT
---

Think
---
Which are the ones that we shall sort
+ HxWxB as it is what it is in BN
+ HxW as it is what it is in GN

conv0layer0
---
### AdaBN
AW 0.74214 0.74717 0.73333 0.73962 0.74465
AD 0.80321 0.80924 0.81526 0.80723 0.81124
WA 0.57685 0.58502 0.58963 0.58289 0.58218

### IDT (with shortcut disabled)

BN Everything (mixed)
---
AW 0.72704
AD 0.79518
WA 0.59709

Layer0
---
### AdaBN
AW 0.73585
AD 0.79719
WA 0.55201 (AdaBN has weird performance on WA for layer0)

### conv0-onlyidt0123 (with conv0 = adabnq)
AD 0.72289

Conv0
---
### AdaBN
AW 0.72453 0.72956 0.72956 0.72579 0.73082
AD 0.80321 0.80321 0.80120 0.80321 0.78916
WA 0.59389 0.59070 0.59922 0.59531 0.59247

### IDT64
AW 0.71321 0.72075 0.73082 0.69937 0.71321
AD 0.51606 0.48795 0.51205 0.47590 0.52008
WA 0.59744 0.59531 0.59815 0.59993 0.59425

### IDT64-group4
AW 0.72579 0.73208 0.72956 0.72956 0.71950
AD 0.81124 0.80723 0.80120 0.80723 0.80924
WA 0.58573 0.59531 0.59460 0.58786 0.59567

Block0
---
### AdaBN
AW 0.73711 0.74214 0.73962 0.73962 0.72956
AD 0.78313 0.79116 0.79518 0.79719 0.79518
WA 0.54065 0.56195 0.52645 0.52716 0.54526

### IDT64
AW 0.74340 0.74340 0.73962 0.73962 0.74717
AD 0.79920 0.79116 0.78112 0.78112 0.78715
WA 0.59957 0.60596 0.60561 0.60277 0.60561

blockall
---
### AdaBN
AW 0.74591 0.74591 0.74214 0.74465 0.74717
AD 0.81325 0.81124 0.81727 0.80924 0.81325
WA 0.57792 0.57650 0.58005 0.58573 0.58218

###

block0123
---
### AdaBN
AW 0.74843 0.74465 0.74465 0.74088 0.73962
AD 0.81928 0.81124 0.81325 0.81526 0.81325
WA 0.38019 0.38481 0.41356 0.34150 0.44054

### fairc4-95-fix-0123 20%
AW 0.74591 0.75346
AD 0.79719 0.80321
WA 0.58928 0.58360

block123
---
### AdaBN
AW 0.74843 0.72579 0.73208 0.72327 0.73208
AD 0.81928 0.81526 0.80924 0.81526 0.81727
WA 0.57969 0.58005 0.58537 0.58147 0.57898

### fairc4-95-fix- 20 20% 128
AW 0.74843 0.74340
AD 0.80924 0.81325
WA 0.58502 0.58111 0.58644

### fairc4-95-fix-123-512 20% 128
AW 0.74591
AD 0.81124
WA 0.57827 0.58715

### fairc4-95-fix-123-64 5% 64
AW 0.75597 0.74340
AD 0.81928 0.81928
WA 0.59385

block23
---
### AdaBN
AW 0.73585 0.73082 0.74214 0.72956 0.73082
AD 0.80924 0.80321 0.80321 0.80522 0.79920
WA 0.58502 0.58431 0.59318 0.58111 0.58750
ClipPro 0.56477 0.56387 0.56026

### fairc4-last-all-improved-15-23 15%
AW 0.73459 0.74088 0.75094 0.74340 0.74717
AD 0.80522 0.80522 0.80321 0.81124 0.81124
WA 0.58253 0.58431 0.58360 0.58644 0.58999

### fairc4-95-fix-123-64 5%
AW 0.74214 0.73836 0.74843 0.74591 0.74214
AD 0.80120 0.80522 0.81124 0.79719 0.81124
WA 0.59141 0.58750 0.58821 0.58821 0.58715
ClipPro 0.56477 0.55959

block3
---
### AdaBN
AW 0.74717 0.74340 0.74717 0.73962 0.73585
AD 0.80120 0.80522 0.80321 0.79518 0.79518
WA 0.59141 0.59389 0.58715 0.59395 0.59283
### Grup1
AW 0.73836 0.73585 0.74843 0.73082 0.74088
AD 0.80120 0.78916 0.79719 0.80522 0.81124
### Grup4
AW 0.74214 0.74088
AD 0.80522 0.79719 0.81124

1-3block
---
### AdaBN
AW 0.74969 0.74214 0.74340 0.74214 0.73711
WA 0.58821 0.58821 0.58253 0.59105 0.58928
### Group2
AW 0.74969 0.74969 0.73962 0.74340
WA 0.59318 0.59218 0.59141 0.59283 0.59567
### Group4
AW 0.74340 0.75346 0.74843 0.74465 0.74214
WA 0.59638 0.59104 0.59034 0.59780 0.58786
### Group8
AW 0.74088 0.73836 0.74843 0.74214 0.74465
WA 0.59070 0.58892 0.58750 0.59176 0.59531
### Group1
No good
tar2src
---
### block34 ad
### block1
### block1 - accumulate
### block2

###
Block0 and conv0
---
group = 1
group = 4
group = all

group = all | before BN


group = all | before BN (5000)

group = all | batch seperate
Too slow; STOP

group = 4 | batch seperate

group = 1 | batch seperate

group = 1 | batch seperate (5000)
 achieved 0.6 but drops

group = 1 | batch seperate (5000, random)
trial 2
trial 3
trial 4
trial 5

group = 1 | batch seperate (5000, layer34)

Block 0 only
---
group = all

group = 1 | batch seperate

Block 0 and conv0
---

Clipart
---
geoup = 1 | batch seperate(5000, random)

group = 4 | batch seperate (5000, random)
Clipart Art

Clipart Realworld