---
# System prepended metadata

title: IDT

---

IDT
===
Sunday
---
![](https://i.imgur.com/sQmxC5B.png)


Saturday
---
![](https://i.imgur.com/3kqAH39.png)


### Usps DTN
BN: Epoch 10 62.68 and up to 64
Transnorm: Epoch 10 45.10 epoch 9 53.02
#### GroupNorm Dead:
GN-4-start-0-idt-16-before: Epoch 10 56.33
GN-4-start-0.1-idt-16-before: Epoch 10 61.64
GN-4-start-0.1-idt-128-before:  Epoch 10: 58.93   run2: 61.72****

GN-4-start-0-idt-16-after: Epoch 20 60.36 Epoch 10: 60.33
GN-4-start-0.1-idt-16-after:  Epoch 10: 61.54 run2: 58.92
GN-4-start-0.1-idt-128-after:  Epoch 10: 60.24   run2: 59.65
GN-4-start-0.1-idt-3-after:  Epoch 10:  58.12  run2: 58.98

#### Standard Norm
Group-4-start-0.1-idt-16-Before:  Epoch 10: 59.93   run2: 57.40
Group-4-start-5-idt-16-Before:  Epoch 10:  62.05  run2:  59.46

Group-4-start-0.1-idt-16-After:  Epoch 10:  59.86  run2: 56.96

#### StandardNorm Align2p
Group-4-start-5-idt-16-Before-l2p:  Epoch 10:  56.17  run2:  55.39

#### sort
before:   46.21% 42.58%
before group0: 
after: 48.56% 
after group0: 39.78%

batch 4 gn:  40.56%
normal group 48.56% 

mix 50%: 54.13%
mix 10% idted: 64.32%

What can be an intermediate way of the alignment****


Friday
---
### SVHN exp
BN: 52% 22%

AdaBN: Epoch 3: 62%  Epoch 20: 52%
AdaBN: Run2 Epoch 3:   Epoch 10: 52%

IDT-direct: 28%
Interp-hist-5-95-32: Epoch 3: 57% Epoch 10: 56%
Interp-hist-5-95-32-group4: Epoch 3: 60% Epoch 10: 58%
Interp-direct-space8: Epoch 3: 57% Epoch 10: 
Interp-direct-space32: Epoch 3: 61% Epoch 10: 58%

Interp-direct-

Interp-direct-p::8::16: 62% 57%

Interp-direct-p::8::16: Use the same stats: 60% 60%



IDT-interp ::32 src2tar: using same BN stats:  61.78% 61.84%
IDT-interp ::128  src2tar: using same BN stats:  
IDT-interp ::16  src2tar: using same BN stats:  

IDT-interp-percentile-32  src2tar: using same BN stats:  63.27%


IDT
---


Think
---
Which are the ones that we shall sort
+ HxWxB as it is what it is in BN
+ HxW as it is what it is in GN

conv0layer0
---
### AdaBN
AW 0.74214 0.74717 0.73333 0.73962 0.74465
AD 0.80321 0.80924 0.81526 0.80723 0.81124
WA 0.57685 0.58502 0.58963 0.58289 0.58218

### IDT (with shortcut disabled)

BN Everything (mixed)
---
AW 0.72704 
AD 0.79518 
WA 0.59709

Layer0
---
### AdaBN
AW 0.73585
AD 0.79719
WA 0.55201    (AdaBN has weird performance on WA for layer0)

### conv0-onlyidt0123 (with conv0 = adabnq)
AD 0.72289

Conv0
---
### AdaBN
AW 0.72453 0.72956 0.72956 0.72579 0.73082
AD 0.80321 0.80321 0.80120 0.80321 0.78916
WA 0.59389 0.59070 0.59922 0.59531 0.59247

### IDT64
AW 0.71321 0.72075 0.73082 0.69937 0.71321
AD 0.51606 0.48795 0.51205 0.47590 0.52008
WA 0.59744 0.59531 0.59815 0.59993 0.59425

### IDT64-group4
AW 0.72579 0.73208 0.72956 0.72956 0.71950
AD 0.81124 0.80723 0.80120 0.80723 0.80924
WA 0.58573 0.59531 0.59460 0.58786 0.59567

Block0
---
### AdaBN
AW 0.73711 0.74214 0.73962 0.73962 0.72956
AD 0.78313 0.79116 0.79518 0.79719 0.79518
WA 0.54065 0.56195 0.52645 0.52716 0.54526

### IDT64
AW 0.74340 0.74340 0.73962 0.73962 0.74717
AD 0.79920 0.79116 0.78112 0.78112 0.78715
WA 0.59957 0.60596 0.60561 0.60277 0.60561

blockall
---
### AdaBN
AW 0.74591 0.74591 0.74214 0.74465 0.74717
AD 0.81325 0.81124 0.81727 0.80924 0.81325
WA 0.57792 0.57650 0.58005 0.58573 0.58218

### 

block0123
---
### AdaBN
AW 0.74843 0.74465 0.74465 0.74088 0.73962
AD 0.81928 0.81124 0.81325 0.81526 0.81325
WA 0.38019 0.38481 0.41356 0.34150 0.44054

### fairc4-95-fix-0123 20%
AW 0.74591 0.75346
AD 0.79719 0.80321
WA 0.58928 0.58360


block123
---
### AdaBN
AW 0.74843 0.72579 0.73208 0.72327 0.73208
AD 0.81928 0.81526 0.80924 0.81526 0.81727
WA 0.57969 0.58005 0.58537 0.58147 0.57898

### fairc4-95-fix- 20   20% 128
AW 0.74843 0.74340 
AD 0.80924 0.81325 
WA 0.58502 0.58111 0.58644

### fairc4-95-fix-123-512  20% 128
AW 0.74591
AD 0.81124
WA 0.57827 0.58715

### fairc4-95-fix-123-64 5% 64
AW 0.75597 0.74340
AD 0.81928 0.81928
WA 0.59385 

block23
---
### AdaBN
AW 0.73585 0.73082 0.74214 0.72956 0.73082
AD 0.80924 0.80321 0.80321 0.80522 0.79920
WA 0.58502 0.58431 0.59318 0.58111 0.58750
ClipPro 0.56477 0.56387 0.56026

### fairc4-last-all-improved-15-23 15%
AW 0.73459 0.74088 0.75094 0.74340 0.74717
AD 0.80522 0.80522 0.80321 0.81124 0.81124
WA 0.58253 0.58431 0.58360 0.58644 0.58999


### fairc4-95-fix-123-64 5%
AW 0.74214 0.73836 0.74843 0.74591 0.74214
AD 0.80120 0.80522 0.81124 0.79719 0.81124
WA 0.59141 0.58750 0.58821 0.58821 0.58715
ClipPro 0.56477  0.55959 

block3
---
### AdaBN
AW 0.74717 0.74340 0.74717 0.73962 0.73585
AD 0.80120 0.80522 0.80321 0.79518 0.79518
WA 0.59141 0.59389 0.58715 0.59395 0.59283
### Grup1
AW 0.73836 0.73585 0.74843 0.73082 0.74088
AD 0.80120 0.78916 0.79719 0.80522 0.81124
### Grup4
AW 0.74214 0.74088 
AD 0.80522 0.79719 0.81124 

1-3block
---
### AdaBN
AW 0.74969 0.74214  0.74340 0.74214 0.73711
WA 0.58821 0.58821  0.58253 0.59105 0.58928
### Group2
AW 0.74969 0.74969  0.73962 0.74340
WA 0.59318 0.59218  0.59141 0.59283 0.59567
### Group4
AW 0.74340 0.75346   0.74843 0.74465 0.74214
WA 0.59638 0.59104   0.59034 0.59780 0.58786
### Group8
AW 0.74088 0.73836  0.74843  0.74214  0.74465
WA 0.59070 0.58892  0.58750  0.59176  0.59531
### Group1 
No good 
tar2src
---
### block34 ad
![](https://i.imgur.com/9RhsX2W.png)
### block1
![](https://i.imgur.com/QvALRDo.png)
### block1 - accumulate 
![](https://i.imgur.com/HZJW8dy.png)
### block2


###
Block0 and conv0
---
group = 1
![](https://i.imgur.com/PKtmPle.png)
group = 4
![](https://i.imgur.com/XZQX8gJ.png)
group = all
![](https://i.imgur.com/JIMfYFS.png)

group = all | before BN

![](https://i.imgur.com/NhWZIAL.png)

group = all | before BN (5000)
![](https://i.imgur.com/NMGPCgl.png)

group = all | batch  seperate
Too slow; STOP


group = 4 | batch  seperate
![](https://i.imgur.com/fk0tEbp.png)


group = 1 | batch  seperate
![](https://i.imgur.com/pTSt4AP.png)


group = 1 | batch  seperate (5000)
![](https://i.imgur.com/KHnQlgg.png) achieved 0.6 but drops



group = 1 | batch  seperate (5000, random)
![](https://i.imgur.com/k3YJkcY.png)
trial 2
![](https://i.imgur.com/as6AzyK.png)
trial 3
![](https://i.imgur.com/nCcqSW3.png)
trial 4
![](https://i.imgur.com/X7a1c29.png)
trial 5



group = 1 | batch  seperate (5000, layer34)
![](https://i.imgur.com/AbE1Ciy.png)



Block 0 only
---
group = all
![](https://i.imgur.com/n7QoYmp.png)

group = 1 | batch  seperate
![](https://i.imgur.com/nDRsIrf.png)


Block 0 and conv0
---




Clipart
---
geoup = 1 | batch seperate(5000, random)
![](https://i.imgur.com/U5sdyzo.png)

group = 4 | batch  seperate (5000, random)
Clipart Art
![](https://i.imgur.com/BVubAA9.png)

Clipart Realworld
![](https://i.imgur.com/SrTAVhU.png)

