Longformer BigBird

Reason is the light and the light of life.

Jerry Su Feb 22, 2022 2 mins

allenai/longformer-large-4096

epoch 3

with pretrained
Lead 0.7826552462526767
Position 0.6857142857142857
Claim 0.6016325707951224
Evidence 0.6062992125984252
Concluding Statement 0.7744827586206896
Counterclaim 0.5159301130524152
Rebuttal 0.43537414965986393

Overall 0.6288697623847826

========================================

epoch4

witout pretrained
Lead 0.7926960257787325
Position 0.6743119266055045
Claim 0.5527019174898314
Evidence 0.6058080479229067
Concluding Statement 0.7251962883654532
Counterclaim 0.4868686868686869
Rebuttal 0.39381153305203936

Overall 0.6044849180118792

with pretrained
Lead 0.7948164146868251
Position 0.6745484400656815
Claim 0.5881818181818181
Evidence 0.5861433087460485
Concluding Statement 0.7867698803659395
Counterclaim 0.5420207743153919
Rebuttal 0.43478260869565216

Overall 0.6296090350081938

========================================

epoch5

witout pretrained
Lead 0.7926565874730022
Position 0.6712629269821373
Claim 0.5932255111382362
Evidence 0.6297068563718876
Concluding Statement 0.7207586933614331
Counterclaim 0.48604860486048607
Rebuttal 0.42297650130548303

Overall 0.6166622402132379 online: 0.612

with pretrained
Lead 0.7887168141592921
Position 0.6821855803427094
Claim 0.6035402004691832
Evidence 0.5819283456269758
Concluding Statement 0.7747941281775869
Counterclaim 0.5489833641404805
Rebuttal 0.4293193717277487

Overall 0.6299239720919967 online: 0.613

========================================

google/bigbird-roberta-large

epoch4

witout pretrained
Lead 0.7881040892193308
Position 0.6803787136793993
Claim 0.5627563576702215
Evidence 0.6255405749173238
Concluding Statement 0.7199727798570943
Counterclaim 0.49209486166007904
Rebuttal 0.37621023513139695

Overall 0.6064368017335493

with pretrained
Lead 0.6620111731843575
Position 0.6253630203291385
Claim 0.5566195834330859
Evidence 0.5972862957937585
Concluding Statement 0.7845871559633028
Counterclaim 0.5284474445515911
Rebuttal 0.4489247311827957

Overall 0.6004627720625757

========================================

epoch5

witout pretrained
Lead 0.7944915254237288
Position 0.666058394160584
Claim 0.5928319030792529
Evidence 0.6387524557956779
Concluding Statement 0.7400275103163686
Rebuttal 0.38944723618090454
Counterclaim 0.4888093106535363

Overall 0.6157740479442932 online 0.621

with pretrained
Lead 0.684285040113261
Position 0.6341315550166717
Claim 0.5885818772856563
Evidence 0.606223588167499
Concluding Statement 0.7843278217109992
Counterclaim 0.5590277777777778
Rebuttal 0.47116564417177914

Overall 0.6182490434633777 online 0.602

========================================

google/bigbird-roberta-base

without pretrained
epoch 2:
Lead 0.7543398211467649
Position 0.6481126039667306
Claim 0.5375384942125943
Evidence 0.5703115146928995
Concluding Statement 0.662747979426892
Counterclaim 0.35459183673469385
Rebuttal 0.1670061099796334

Overall 0.5278069085943154

with pretrained
Lead 0.741280583029672
Position 0.6520376175548589
Claim 0.5612002658816827
Evidence 0.5893333333333334
Concluding Statement 0.6658461538461539
Counterclaim 0.44222222222222224
Rebuttal 0.34755134281200634

Overall 0.5713530740971328

========================================

without pretrained
epoch 3:

Lead 0.7664603263927968
Position 0.6520833333333333
Claim 0.49025932137788675
Evidence 0.5754781890738957
Concluding Statement 0.7298008267568583
Counterclaim 0.3768844221105528
Rebuttal 0.20384615384615384

Overall 0.5421160818416397

with pretrained
Lead 0.7558711086837794
Position 0.6555819477434679
Claim 0.5490921706950388
Evidence 0.6131916547881301
Concluding Statement 0.7357811375089993
Counterclaim 0.461211477151966
Rebuttal 0.338368580060423

Overall 0.5870140109474006

========================================

without pretrained
epoch 4:
Lead 0.7712561711464618
Position 0.6572368421052631
Claim 0.5434508093215474
Evidence 0.5744189207557429
Concluding Statement 0.7106097990835389
Counterclaim 0.3940435280641466
Rebuttal 0.2513274336283186

Overall 0.5574776434435742

with pretrained
Lead 0.7700592353257943
Position 0.6617834394904458
Claim 0.5798239100456136
Evidence 0.6244618890858445
Concluding Statement 0.7447639332623358
Counterclaim 0.48526522593320237
Rebuttal 0.38526912181303113

Overall 0.6073466792794668


Read more:

Related posts: