BasicOptimizer.scala:89 executed in 61.23 seconds (1.166 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 154323089253400
Reset training subject: 154324040011800
Adding measurement 7e37489e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-242.5502122628426;dx=-0.024244119071091412
New Minimum: -242.5502122628426 > -242.6090122244019
WOLFE (weak): th(2.154434690031884)=-242.6090122244019; dx=-0.0235127432223243 evalInputDelta=0.058799961559316216
New Minimum: -242.6090122244019 > -242.66796695794028
WOLFE (weak): th(4.308869380063768)=-242.66796695794028; dx=-0.023484944278849205 evalInputDelta=0.11775469509768755
New Minimum: -242.66796695794028 > -242.9038599039752
WOLFE (weak): th(12.926608140191302)=-242.9038599039752; dx=-0.023524528974111135 evalInputDelta=0.3536476411325964
New Minimum: -242.9038599039752 > -243.96488124574407
WOLFE (weak): th(51.70643256076521)=-243.96488124574407; dx=-0.023518070132085574 evalInputDelta=1.4146689829014747
New Minimum: -243.96488124574407 > -249.62847393676867
WOLFE (weak): th(258.53216280382605)=-249.62847393676867; dx=-0.023521624960138706 evalInputDelta=7.078261673926079
New Minimum: -249.62847393676867 > -283.3214412332748
END: th(1551.1929768229563)=-283.3214412332748; dx=-0.02127457865519409 evalInputDelta=40.77122897043222
Fitness changed from -242.5502122628426 to -283.3214412332748
Iteration 1 complete. Error: -283.3214412332748 Total: 9.2883; Orientation: 0.0144; Line Search: 6.6055
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5c85d48a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-283.3214412332748;dx=-0.02467156549010514
New Minimum: -283.3214412332748 > -365.12754620817094
END: th(3341.943960201201)=-365.12754620817094; dx=-0.018311765733383206 evalInputDelta=81.80610497489613
Fitness changed from -283.3214412332748 to -365.12754620817094
Iteration 2 complete. Error: -365.12754620817094 Total: 2.6394; Orientation: 0.0149; Line Search: 1.7173
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement c5ad9a8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-365.12754620817094;dx=-0.024187931195091335
New Minimum: -365.12754620817094 > -514.7701853740172
END: th(7200.000000000001)=-514.7701853740172; dx=-0.013579625650146965 evalInputDelta=149.64263916584622
Fitness changed from -365.12754620817094 to -514.7701853740172
Iteration 3 complete. Error: -514.7701853740172 Total: 2.7105; Orientation: 0.0146; Line Search: 1.7496
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6578c3de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.747e+04, gradient 1.681e-01, dot -0.468; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -514.7701853740172, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 6578c3de to history. Total: 3
th(0)=-514.7701853740172;dx=-0.028162369783820765
New Minimum: -514.7701853740172 > -689.6176703435027
END: th(15511.929768229566)=-689.6176703435027; dx=-0.005456163901770582 evalInputDelta=174.84748496948555
Fitness changed from -514.7701853740172 to -689.6176703435027
Iteration 4 complete. Error: -689.6176703435027 Total: 3.2214; Orientation: 0.3014; Line Search: 1.7477
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 13e33cee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.314e+04, gradient 1.914e-01, dot -0.310; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -689.6176703435027, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 13e33cee to history. Total: 3
th(0)=-689.6176703435027;dx=-0.03653388547192647
New Minimum: -689.6176703435027 > -912.8089887813331
END: th(33419.43960201202)=-912.8089887813331; dx=-0.0012874065413554202 evalInputDelta=223.19131843783043
Fitness changed from -689.6176703435027 to -912.8089887813331
Iteration 5 complete. Error: -912.8089887813331 Total: 2.8929; Orientation: 0.2858; Line Search: 1.6986
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 55546bed to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.626e+04, gradient 1.708e-01, dot -0.215; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -912.8089887813331, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 55546bed to history. Total: 3
th(0)=-912.8089887813331;dx=-0.029104422905642628
New Minimum: -912.8089887813331 > -1375.1414472700617
END: th(72000.00000000003)=-1375.1414472700617; dx=-5.706401164485924E-6 evalInputDelta=462.33245848872855
Fitness changed from -912.8089887813331 to -1375.1414472700617
Iteration 6 complete. Error: -1375.1414472700617 Total: 2.9530; Orientation: 0.2826; Line Search: 1.7349
Adding measurement 7f587dfd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.243e+04, gradient 1.722e-01, dot -0.170; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1375.1414472700617, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 7f587dfd to history. Total: 3
th(0)=-1375.1414472700617;dx=-0.02958506934054009
New Minimum: -1375.1414472700617 > -1609.1503192679047
WOLF (strong): th(155119.2976822957)=-1609.1503192679047; dx=0.002388095674602705 evalInputDelta=234.00887199784302
New Minimum: -1609.1503192679047 > -2067.098933287906
WOLF (strong): th(77559.64884114785)=-2067.098933287906; dx=0.0016839503573554265 evalInputDelta=691.9574860178445
END: th(25853.216280382618)=-2028.041358059544; dx=-0.009516286458212805 evalInputDelta=652.8999107894824
Fitness changed from -1375.1414472700617 to -2067.098933287906
Iteration 7 complete. Error: -2067.098933287906 Total: 4.6366; Orientation: 0.2650; Line Search: 3.4177
Adding measurement 5e40ce03 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.872e+04, gradient 1.947e-01, dot -0.162; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2067.098933287906, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 5e40ce03 to history. Total: 3
th(0)=-2067.098933287906;dx=-0.03781930161456037
New Minimum: -2067.098933287906 > -2894.20800911061
WOLF (strong): th(55699.06600335338)=-2894.20800911061; dx=0.0012987752108384759 evalInputDelta=827.1090758227037
END: th(27849.53300167669)=-2846.209409021058; dx=-0.008788451591917795 evalInputDelta=779.1104757331518
Fitness changed from -2067.098933287906 to -2894.20800911061
Iteration 8 complete. Error: -2894.20800911061 Total: 3.6124; Orientation: 0.2735; Line Search: 2.5437
Adding measurement 4797144d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.340e+05, gradient 1.875e-01, dot -0.230; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2894.20800911061, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 4797144d to history. Total: 3
th(0)=-2894.20800911061;dx=-0.035132000013454553
New Minimum: -2894.20800911061 > -3545.6686252384047
WOLF (strong): th(60000.00000000004)=-3545.6686252384047; dx=0.0033304737481467893 evalInputDelta=651.4606161277948
New Minimum: -3545.6686252384047 > -3591.5182026546936
END: th(30000.00000000002)=-3591.5182026546936; dx=-0.007047002960236355 evalInputDelta=697.3101935440836
Fitness changed from -2894.20800911061 to -3591.5182026546936
Iteration 9 complete. Error: -3591.5182026546936 Total: 3.8799; Orientation: 0.2780; Line Search: 2.7737
Adding measurement 590ec731 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.461e+05, gradient 1.652e-01, dot 0.312; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-3591.5182026546936;dx=-7536.910575580609
Armijo: th(64633.04070095655)=-995.866080354181; dx=0.012307831836761724 evalInputDelta=-2595.6521223005125
Armijo: th(32316.520350478277)=-995.8425139046836; dx=0.024633111609785104 evalInputDelta=-2595.67568875001
Armijo: th(10772.173450159426)=-996.0016629002182; dx=0.0738581816766481 evalInputDelta=-2595.516539754475
Armijo: th(2693.0433625398564)=-997.1738188500539; dx=0.2948919145788964 evalInputDelta=-2594.3443838046396
Armijo: th(538.6086725079713)=-1003.6384122282287; dx=1.475248472656415 evalInputDelta=-2587.8797904264648
Armijo: th(89.76811208466188)=-1016.6390706637238; dx=8.824235246159702 evalInputDelta=-2574.8791319909697
Armijo: th(12.824016012094555)=-1066.1657938613487; dx=62.72127108046428 evalInputDelta=-2525.3524087933447
Armijo: th(1.6030020015118194)=-1462.6784506514068; dx=525.6351510374858 evalInputDelta=-2128.839752003287
Armijo: th(0.17811133350131325)=-3243.1341322862363; dx=2401.2336048721263 evalInputDelta=-348.3840703684573
New Minimum: -3591.5182026546936 > -3661.652768900951
END: th(0.017811133350131327)=-3661.652768900951; dx=-1976.881965523548 evalInputDelta=70.13456624625724
Fitness changed from -3591.5182026546936 to -3661.652768900951
Iteration 10 complete. Error: -3661.652768900951 Total: 10.6645; Orientation: 0.2690; Line Search: 9.5589
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 60bb808d to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.218e+05, gradient 1.679e-01, dot -0.233; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.652768900951, -3591.5182026546936, -365.12754620817094, -283.3214412332748, -242.5502122628426
Rejected: LBFGS Orientation magnitude: 1.452e+05, gradient 1.679e-01, dot -0.266; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.652768900951, -3591.5182026546936, -365.12754620817094, -283.3214412332748
LBFGS Accumulation History: 3 points
Removed measurement 60bb808d to history. Total: 4
Removed measurement 590ec731 to history. Total: 3
th(0)=-3661.652768900951;dx=-0.02813773915567159
New Minimum: -3661.652768900951 > -3661.653588203013
END: th(0.03837292355830673)=-3661.653588203013; dx=-0.017285647984569667 evalInputDelta=8.193020621547475E-4
Fitness changed from -3661.652768900951 to -3661.653588203013
Iteration 11 complete. Error: -3661.653588203013 Total: 3.2367; Orientation: 0.6326; Line Search: 1.7101
Adding measurement 63169941 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.582e+05, gradient 1.679e-01, dot -0.225; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.653588203013, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 63169941 to history. Total: 3
th(0)=-3661.653588203013;dx=-0.028157203841192118
New Minimum: -3661.653588203013 > -3661.655499907112
END: th(0.08267195767195773)=-3661.655499907112; dx=-0.017293269501846807 evalInputDelta=0.0019117040988021472
Fitness changed from -3661.653588203013 to -3661.655499907112
Iteration 12 complete. Error: -3661.655499907112 Total: 2.7913; Orientation: 0.2730; Line Search: 1.7163
Adding measurement 5dfb455a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.626e+05, gradient 1.679e-01, dot -0.225; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.655499907112, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 5dfb455a to history. Total: 3
th(0)=-3661.655499907112;dx=-0.028167422622763456
New Minimum: -3661.655499907112 > -3661.660006062824
END: th(0.17811133350131328)=-3661.660006062824; dx=-0.017317294989724508 evalInputDelta=0.0045061557120789075
Fitness changed from -3661.655499907112 to -3661.660006062824
Iteration 13 complete. Error: -3661.660006062824 Total: 3.0558; Orientation: 0.2708; Line Search: 1.9406
Adding measurement 6b10b77e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.271e+05, gradient 1.678e-01, dot -0.225; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.660006062824, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 6b10b77e to history. Total: 3
th(0)=-3661.660006062824;dx=-0.028117639828591348
New Minimum: -3661.660006062824 > -3661.669428006848
END: th(0.3837292355830674)=-3661.669428006848; dx=-0.017287484714252982 evalInputDelta=0.009421944024325057
Fitness changed from -3661.660006062824 to -3661.669428006848
Iteration 14 complete. Error: -3661.669428006848 Total: 2.7625; Orientation: 0.2652; Line Search: 1.7004
Adding measurement 79ab7806 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.444e+05, gradient 1.679e-01, dot -0.225; [829d4c65-0ffa-42fd-ae0e-c6184ef591ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3661.669428006848, -365.12754620817094, -283.3214412332748, -242.5502122628426
LBFGS Accumulation History: 3 points
Removed measurement 79ab7806 to history. Total: 3
th(0)=-3661.669428006848;dx=-0.028151887568027667
New Minimum: -3661.669428006848 > -3661.689227665286
END: th(0.8267195767195775)=-3661.689227665286; dx=-0.01729741064156838 evalInputDelta=0.01979965843793252
Fitness changed from -3661.669428006848 to -3661.689227665286
Iteration 15 complete. Error: -3661.689227665286 Total: 2.7807; Orientation: 0.2716; Line Search: 1.7246
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3661.689227665286 (> -Infinity) after 61.226s (< 5400.000s)
Returns
-3661.689227665286