BasicOptimizer.scala:89 executed in 56.69 seconds (1.022 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2804551396561
Reset training subject: 2805020387457
Adding measurement 4843fe34 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-5.590723168648693;dx=-0.0018440221886971682
New Minimum: -5.590723168648693 > -5.590775824942668
WOLFE (weak): th(2.154434690031884)=-5.590775824942668; dx=-0.0018430207562136454 evalInputDelta=5.265629397488425E-5
New Minimum: -5.590775824942668 > -5.590827752783747
WOLFE (weak): th(4.308869380063768)=-5.590827752783747; dx=-0.0018425183133173742 evalInputDelta=1.0458413505443076E-4
New Minimum: -5.590827752783747 > -5.591042253334041
WOLFE (weak): th(12.926608140191302)=-5.591042253334041; dx=-0.0018405878219773745 evalInputDelta=3.1908468534780354E-4
New Minimum: -5.591042253334041 > -5.59198508463994
WOLFE (weak): th(51.70643256076521)=-5.59198508463994; dx=-0.0018357986010143223 evalInputDelta=0.0012619159912468092
New Minimum: -5.59198508463994 > -5.596237738296379
WOLFE (weak): th(258.53216280382605)=-5.596237738296379; dx=-0.0018316573058658036 evalInputDelta=0.005514569647686152
New Minimum: -5.596237738296379 > -5.631758961816276
WOLFE (weak): th(1551.1929768229563)=-5.631758961816276; dx=-0.0017522276361464932 evalInputDelta=0.0410357931675831
New Minimum: -5.631758961816276 > -6.114720106103019
END: th(10858.350837760694)=-6.114720106103019; dx=-0.001253907740916692 evalInputDelta=0.5239969374543261
Fitness changed from -5.590723168648693 to -6.114720106103019
Iteration 1 complete. Error: -6.114720106103019 Total: 5.3399; Orientation: 0.0181; Line Search: 3.9649
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 22555705 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-6.114720106103019;dx=-0.0020054022318691292
New Minimum: -6.114720106103019 > -7.244442763261641
END: th(23393.607721408407)=-7.244442763261641; dx=-8.049024473881679E-4 evalInputDelta=1.1297226571586219
Fitness changed from -6.114720106103019 to -7.244442763261641
Iteration 2 complete. Error: -7.244442763261641 Total: 1.4003; Orientation: 0.0169; Line Search: 0.9314
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 10203bf9 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-7.244442763261641;dx=-0.0022039393392504283
New Minimum: -7.244442763261641 > -11.433781414268292
END: th(50400.00000000001)=-11.433781414268292; dx=-3.6056367883985916E-4 evalInputDelta=4.1893386510066515
Fitness changed from -7.244442763261641 to -11.433781414268292
Iteration 3 complete. Error: -11.433781414268292 Total: 1.5903; Orientation: 0.0162; Line Search: 0.9475
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 32a2b515 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.142e+03, gradient 4.966e-02, dot -0.188; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.433781414268292, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 32a2b515 to history. Total: 3
th(0)=-11.433781414268292;dx=-0.0024650860803970046
New Minimum: -11.433781414268292 > -13.69525513313118
END: th(108583.50837760697)=-13.69525513313118; dx=-1.676704648290417E-4 evalInputDelta=2.261473718862888
Fitness changed from -11.433781414268292 to -13.69525513313118
Iteration 4 complete. Error: -13.69525513313118 Total: 1.8285; Orientation: 0.4131; Line Search: 0.9763
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 30b42a04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.505e+03, gradient 4.366e-02, dot -0.134; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.69525513313118, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 30b42a04 to history. Total: 3
th(0)=-13.69525513313118;dx=-0.0019025975551301415
New Minimum: -13.69525513313118 > -16.17155039038628
END: th(233936.07721408413)=-16.17155039038628; dx=-6.256353581259474E-5 evalInputDelta=2.4762952572550994
Fitness changed from -13.69525513313118 to -16.17155039038628
Iteration 5 complete. Error: -16.17155039038628 Total: 1.8322; Orientation: 0.4129; Line Search: 0.9586
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 55eb77dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.992e+03, gradient 3.621e-02, dot -0.104; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.17155039038628, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 55eb77dc to history. Total: 3
th(0)=-16.17155039038628;dx=-0.001305312600427178
New Minimum: -16.17155039038628 > -18.015396386171968
END: th(504000.0000000002)=-18.015396386171968; dx=-2.5840514594613317E-5 evalInputDelta=1.8438459957856885
Fitness changed from -16.17155039038628 to -18.015396386171968
Iteration 6 complete. Error: -18.015396386171968 Total: 2.4311; Orientation: 0.4197; Line Search: 1.5429
Adding measurement 5afa2ae1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.729e+03, gradient 2.485e-02, dot -0.071; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.015396386171968, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 5afa2ae1 to history. Total: 3
th(0)=-18.015396386171968;dx=-6.162889072712646E-4
Armijo: th(1085835.0837760698)=-17.648656536463843; dx=-1.0970194043487504E-5 evalInputDelta=-0.3667398497081251
New Minimum: -18.015396386171968 > -18.88750986062282
END: th(542917.5418880349)=-18.88750986062282; dx=-3.954752340691088E-5 evalInputDelta=0.8721134744508525
Fitness changed from -18.015396386171968 to -18.88750986062282
Iteration 7 complete. Error: -18.88750986062282 Total: 2.3022; Orientation: 0.4195; Line Search: 1.4461
Adding measurement 7aa18131 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.466e+03, gradient 2.166e-02, dot -0.075; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.88750986062282, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 7aa18131 to history. Total: 3
th(0)=-18.88750986062282;dx=-4.671464665820351E-4
New Minimum: -18.88750986062282 > -19.925381088976202
END: th(1169680.3860704207)=-19.925381088976202; dx=-1.5670987673632E-5 evalInputDelta=1.0378712283533815
Fitness changed from -18.88750986062282 to -19.925381088976202
Iteration 8 complete. Error: -19.925381088976202 Total: 1.8293; Orientation: 0.4088; Line Search: 1.0026
Adding measurement 17312ad9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.593e+03, gradient 1.729e-02, dot -0.063; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.925381088976202, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 17312ad9 to history. Total: 3
th(0)=-19.925381088976202;dx=-2.982242753486141E-4
New Minimum: -19.925381088976202 > -21.177484013790465
END: th(2520000.000000001)=-21.177484013790465; dx=-3.993796970531723E-6 evalInputDelta=1.2521029248142632
Fitness changed from -19.925381088976202 to -21.177484013790465
Iteration 9 complete. Error: -21.177484013790465 Total: 1.8224; Orientation: 0.4088; Line Search: 0.9757
Adding measurement 340360d7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.001e+04, gradient 1.331e-02, dot -0.038; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.177484013790465, -7.244442763261641, -6.114720106103019, -5.
...skipping 3867 bytes...
529342)=-22.802515165148712; dx=-1.4703374148243627E-6 evalInputDelta=-5.208085790197298
Armijo: th(2436834.13764671)=-26.941516854344368; dx=-9.609574990389844E-6 evalInputDelta=-1.0690841010016428
New Minimum: -28.01060095534601 > -29.91819268481694
END: th(812278.0458822367)=-29.91819268481694; dx=-4.534469106228207E-5 evalInputDelta=1.90759172947093
Fitness changed from -28.01060095534601 to -29.91819268481694
Iteration 14 complete. Error: -29.91819268481694 Total: 2.9398; Orientation: 0.4741; Line Search: 2.0390
Adding measurement 36f03f61 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.922e+06, gradient 1.189e-02, dot -0.106; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.91819268481694, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 36f03f61 to history. Total: 3
th(0)=-29.91819268481694;dx=-1.4053326581933852E-4
New Minimum: -29.91819268481694 > -30.138584050870207
END: th(1750000.000000001)=-30.138584050870207; dx=-1.753365282864099E-5 evalInputDelta=0.22039136605326703
Fitness changed from -29.91819268481694 to -30.138584050870207
Iteration 15 complete. Error: -30.138584050870207 Total: 1.7554; Orientation: 0.4053; Line Search: 0.9270
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 4ff053a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.025e+05, gradient 1.144e-02, dot -0.095; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.138584050870207, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 4ff053a6 to history. Total: 3
th(0)=-30.138584050870207;dx=-1.30252932428237E-4
Armijo: th(3770260.707555799)=-26.879650743372327; dx=-4.155987657779212E-6 evalInputDelta=-3.2589333074978803
New Minimum: -30.138584050870207 > -30.29573416733024
END: th(1885130.3537778994)=-30.29573416733024; dx=-1.68358094836333E-5 evalInputDelta=0.15715011646003418
Fitness changed from -30.138584050870207 to -30.29573416733024
Iteration 16 complete. Error: -30.29573416733024 Total: 2.2889; Orientation: 0.4065; Line Search: 1.4355
Adding measurement 1261860e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.271e+07, gradient 1.142e-02, dot -0.096; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.29573416733024, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 1261860e to history. Total: 3
th(0)=-30.29573416733024;dx=-1.299593146158591E-4
Armijo: th(4061390.2294111843)=-27.886990028252647; dx=-3.7345710533935172E-6 evalInputDelta=-2.4087441390775943
New Minimum: -30.29573416733024 > -31.82188374684004
END: th(2030695.1147055922)=-31.82188374684004; dx=-1.539538910963923E-5 evalInputDelta=1.5261495795097986
Fitness changed from -30.29573416733024 to -31.82188374684004
Iteration 17 complete. Error: -31.82188374684004 Total: 2.2684; Orientation: 0.4012; Line Search: 1.4565
Adding measurement 1f3f3ffd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.872e+05, gradient 1.143e-02, dot -0.092; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.82188374684004, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement 1f3f3ffd to history. Total: 3
th(0)=-31.82188374684004;dx=-1.301445187358E-4
Armijo: th(4375000.000000003)=-28.741676890829055; dx=-2.9147034740156764E-6 evalInputDelta=-3.080206856010985
New Minimum: -31.82188374684004 > -32.44249365745096
END: th(2187500.0000000014)=-32.44249365745096; dx=-1.3073366127998679E-5 evalInputDelta=0.62060991061092
Fitness changed from -31.82188374684004 to -32.44249365745096
Iteration 18 complete. Error: -32.44249365745096 Total: 2.3895; Orientation: 0.4871; Line Search: 1.4741
Adding measurement fb9a8fa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.115e+05, gradient 1.079e-02, dot -0.085; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.44249365745096, -7.244442763261641, -6.114720106103019, -5.590723168648693
LBFGS Accumulation History: 3 points
Removed measurement fb9a8fa to history. Total: 3
th(0)=-32.44249365745096;dx=-1.1559788894268999E-4
Armijo: th(4712825.884444749)=-30.00303865811011; dx=-3.225251076576294E-6 evalInputDelta=-2.4394549993408496
New Minimum: -32.44249365745096 > -34.092647578118964
END: th(2356412.9422223745)=-34.092647578118964; dx=-1.369483370736666E-5 evalInputDelta=1.6501539206680036
Fitness changed from -32.44249365745096 to -34.092647578118964
Iteration 19 complete. Error: -34.092647578118964 Total: 2.8777; Orientation: 0.4027; Line Search: 2.0489
Adding measurement 6745fd2b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.278e+04, gradient 1.054e-02, dot 0.091; [83ca111c-2e88-41b2-a470-3565da9f2137 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-34.092647578118964;dx=-59.93679599708926
Armijo: th(5076737.786763981)=-27.96525454418695; dx=6.416849619699868E-7 evalInputDelta=-6.127393033932012
Armijo: th(2538368.8933819905)=-27.95645905485181; dx=1.2860719849576845E-6 evalInputDelta=-6.136188523267155
Armijo: th(846122.9644606635)=-27.948689086258753; dx=3.835144540132031E-6 evalInputDelta=-6.143958491860211
Armijo: th(211530.74111516587)=-27.940101247382234; dx=1.5411202440999265E-5 evalInputDelta=-6.15254633073673
Armijo: th(42306.148223033175)=-27.95926801225666; dx=7.672282687117566E-5 evalInputDelta=-6.1333795658623025
Armijo: th(7051.0247038388625)=-27.94905410414957; dx=4.5194361612137615E-4 evalInputDelta=-6.143593473969393
Armijo: th(1007.2892434055518)=-27.926797935870947; dx=0.003168085706342167 evalInputDelta=-6.165849642248016
Armijo: th(125.91115542569398)=-28.31256922053194; dx=0.02591693568604487 evalInputDelta=-5.780078357587023
Armijo: th(13.990128380632664)=-30.482087082010867; dx=0.20265869315077095 evalInputDelta=-3.6105604961080964
Armijo: th(1.3990128380632663)=-31.954617091157818; dx=-1.6152795466886825 evalInputDelta=-2.1380304869611457
Armijo: th(0.12718298527847874)=-33.52249601680928; dx=-24.395341257132735 evalInputDelta=-0.5701515613096859
Armijo: th(0.010598582106539895)=-34.04093557312893; dx=-32.73334126046482 evalInputDelta=-0.05171200499003703
Armijo: th(8.15275546656915E-4)=-34.08885531304258; dx=-33.979476730375666 evalInputDelta=-0.0037922650763846377
Armijo: th(5.8233967618351075E-5)=-34.09237762557864; dx=-34.00972107818349 evalInputDelta=-2.69952540321583E-4
Armijo: th(3.882264507890072E-6)=-34.092624657703304; dx=-33.921995928636434 evalInputDelta=-2.292041565965519E-5
Armijo: th(2.426415317431295E-7)=-34.092646304762944; dx=-34.03098413107297 evalInputDelta=-1.2733560197375482E-6
Armijo: th(1.4273031279007617E-8)=-34.09264375805076; dx=-34.02358759283091 evalInputDelta=-3.820068201321192E-6
Armijo: th(7.929461821670898E-10)=-34.092647578118964; dx=-34.02391402793953 evalInputDelta=0.0
MIN ALPHA (4.173400958774157E-11): th(0.0)=-34.092647578118964
Fitness changed from -34.092647578118964 to -34.092647578118964
Static Iteration Total: 10.7416; Orientation: 0.4577; Line Search: 9.8712
Iteration 20 failed. Error: -34.092647578118964
Previous Error: 0.0 -> -34.092647578118964
Retrying iteration 20
Reset training subject: 2859325768928
Adding measurement 139f9e00 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-34.092647578118964;dx=-1.105795515222906E-4
END: th(2.1544346900318838E-10)=-34.092647578118964; dx=-8.433135423418749E-5 evalInputDelta=0.0
Fitness changed from -34.092647578118964 to -34.092647578118964
Static Iteration Total: 1.9101; Orientation: 0.0171; Line Search: 1.0329
Iteration 21 failed. Error: -34.092647578118964
Previous Error: 0.0 -> -34.092647578118964
Optimization terminated 21
Final threshold in iteration 21: -34.092647578118964 (> -Infinity) after 56.684s (< 3600.000s)
Returns
-34.092647578118964