BasicOptimizer.scala:89 executed in 136.24 seconds (1.735 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 75566685987200
Reset training subject: 75569445726800
Adding measurement 49e5cfb7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-20.173722467680307;dx=-8.385579891400413E-7
New Minimum: -20.173722467680307 > -20.173724387037293
END: th(2.154434690031884)=-20.173724387037293; dx=-7.082938021058853E-7 evalInputDelta=1.9193569862352433E-6
Fitness changed from -20.173722467680307 to -20.173724387037293
Iteration 1 complete. Error: -20.173724387037293 Total: 13.0362; Orientation: 0.0589; Line Search: 5.2026
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 392e71b7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-20.173724387037293;dx=-7.064025137547592E-7
New Minimum: -20.173724387037293 > -20.1737275859652
WOLFE (weak): th(4.641588833612779)=-20.1737275859652; dx=-7.058804911312042E-7 evalInputDelta=3.1989279065669507E-6
New Minimum: -20.1737275859652 > -20.173732064463415
WOLFE (weak): th(9.283177667225559)=-20.173732064463415; dx=-7.060792148392724E-7 evalInputDelta=7.677426122398856E-6
New Minimum: -20.173732064463415 > -20.173746779521984
WOLFE (weak): th(27.849533001676676)=-20.173746779521984; dx=-7.059775320410073E-7 evalInputDelta=2.2392484691380332E-5
New Minimum: -20.173746779521984 > -20.173813956826933
WOLFE (weak): th(111.3981320067067)=-20.173813956826933; dx=-7.05719008718625E-7 evalInputDelta=8.956978964036466E-5
New Minimum: -20.173813956826933 > -20.174165834293436
WOLFE (weak): th(556.9906600335335)=-20.174165834293436; dx=-7.055369241310423E-7 evalInputDelta=4.4144725614359004E-4
New Minimum: -20.174165834293436 > -20.176373565396673
WOLFE (weak): th(3341.9439602012008)=-20.176373565396673; dx=-7.052678566160883E-7 evalInputDelta=0.0026491783593804996
New Minimum: -20.176373565396673 > -20.192222345532095
WOLFE (weak): th(23393.607721408407)=-20.192222345532095; dx=-7.023355297534261E-7 evalInputDelta=0.018497958494801736
New Minimum: -20.192222345532095 > -20.318758751170595
WOLFE (weak): th(187148.86177126726)=-20.318758751170595; dx=-6.73294913729425E-7 evalInputDelta=0.14503436413330206
New Minimum: -20.318758751170595 > -21.14953404609573
END: th(1684339.7559414054)=-21.14953404609573; dx=-3.327279345080164E-7 evalInputDelta=0.975809659058438
Fitness changed from -20.173724387037293 to -21.14953404609573
Iteration 2 complete. Error: -21.14953404609573 Total: 29.2201; Orientation: 0.0584; Line Search: 26.0250
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4a2a6dad to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-21.14953404609573;dx=-7.106036524853357E-7
New Minimum: -21.14953404609573 > -22.112052081779364
END: th(3628800.0000000005)=-22.112052081779364; dx=-1.0728047072858402E-7 evalInputDelta=0.9625180356836331
Fitness changed from -21.14953404609573 to -22.112052081779364
Iteration 3 complete. Error: -22.112052081779364 Total: 8.1853; Orientation: 0.0605; Line Search: 5.1466
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 67453b7f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.230e+03, gradient 8.775e-04, dot -0.532; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.112052081779364, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 67453b7f to history. Total: 3
th(0)=-22.112052081779364;dx=-7.691078443136413E-7
New Minimum: -22.112052081779364 > -23.3015212318818
WOLF (strong): th(7818012.603187701)=-23.3015212318818; dx=1.2746612029123839E-8 evalInputDelta=1.1894691501024361
New Minimum: -23.3015212318818 > -23.544090418708326
WOLF (strong): th(3909006.3015938504)=-23.544090418708326; dx=5.4061109694450016E-8 evalInputDelta=1.4320383369289615
END: th(1303002.1005312835)=-23.1017896011713; dx=-5.180280772486538E-7 evalInputDelta=0.9897375193919373
Fitness changed from -22.112052081779364 to -23.544090418708326
Iteration 4 complete. Error: -23.544090418708326 Total: 14.2737; Orientation: 1.0322; Line Search: 10.3040
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 33eb0150 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.475e+03, gradient 9.056e-04, dot -0.334; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.544090418708326, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 33eb0150 to history. Total: 3
th(0)=-23.544090418708326;dx=-8.199464817274871E-7
New Minimum: -23.544090418708326 > -24.814440436663187
WOLF (strong): th(2807232.926569009)=-24.814440436663187; dx=9.812251969424882E-8 evalInputDelta=1.2703500179548612
END: th(1403616.4632845046)=-24.61564643480496; dx=-4.2873409783999166E-7 evalInputDelta=1.0715560160966362
Fitness changed from -23.544090418708326 to -24.814440436663187
Iteration 5 complete. Error: -24.814440436663187 Total: 11.7914; Orientation: 1.0016; Line Search: 7.6579
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3c0e88b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.651e+03, gradient 8.463e-04, dot -0.455; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.814440436663187, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 3c0e88b8 to history. Total: 3
th(0)=-24.814440436663187;dx=-7.159759448235376E-7
New Minimum: -24.814440436663187 > -25.433645983927928
WOLF (strong): th(3024000.0000000005)=-25.433645983927928; dx=1.9366116990377262E-7 evalInputDelta=0.6192055472647411
New Minimum: -25.433645983927928 > -25.62985823055114
END: th(1512000.0000000002)=-25.62985823055114; dx=-1.1083479978364748E-7 evalInputDelta=0.8154177938879528
Fitness changed from -24.814440436663187 to -25.62985823055114
Iteration 6 complete. Error: -25.62985823055114 Total: 11.5887; Orientation: 0.9720; Line Search: 7.5703
Adding measurement 24d9adac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.490e+04, gradient 6.799e-04, dot -0.632; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.62985823055114, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 24d9adac to history. Total: 3
th(0)=-25.62985823055114;dx=-4.6199858146701667E-7
New Minimum: -25.62985823055114 > -26.01615522683521
WOLF (strong): th(3257505.251328209)=-26.01615522683521; dx=1.7907024977734327E-8 evalInputDelta=0.3862969962840701
New Minimum: -26.01615522683521 > -26.03349640848977
END: th(1628752.6256641045)=-26.03349640848977; dx=-4.026016388510931E-8 evalInputDelta=0.40363817793863177
Fitness changed from -25.62985823055114 to -26.03349640848977
Iteration 7 complete. Error: -26.03349640848977 Total: 11.4613; Orientation: 1.1046; Line Search: 7.8064
Adding measurement 30e77d9e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.494e+04, gradient 7.522e-04, dot -0.547; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.03349640848977, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 30e77d9e to history. Total: 3
th(0)=-26.03349640848977;dx=-5.653611376759658E-7
New Minimum: -26.03349640848977 > -26.22007558198702
WOLF (strong): th(3509041.158211262)=-26.22007558198702; dx=7.96280718169729E-8 evalInputDelta=0.1865791734972504
New Minimum: -26.22007558198702 > -26.430544383232174
WOLF (strong): th(1754520.579105631)=-26.430544383232174; dx=8.727990162351364E-8 evalInputDelta=0.39704797474240294
END: th(584840.1930352104)=-26.346169648629107; dx=-3.5259562522875886E-7 evalInputDelta=0.3126732401393362
Fitness changed from -26.03349640848977 to -26.430544383232174
Iteration 8 complete. Error: -26.430544383232174 Total: 13.8475; Orientation: 1.0046; Line Search: 10.4325
Adding measurement 2c81de0c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.508e+04, gradient 8.081e-04, dot -0.484; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.430544383232174, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 2c81de0c to history. Total: 3
th(0)=-26.430544383232174;dx=-6.525996432949307E-7
New Minimum: -26.430544383232174 > -26.956814414066212
WOLF (strong): th(1260000.0000000005)=-26.956814414066212; dx=2.6089456544756713E-8 evalInputDelta=0.5262700308340378
END: th(630000.0000000002)=-26.826873477807062; dx=-4.224875824138005E-7 evalInputDelta=0.3963290945748881
Fitness changed from -26.430544383232174 to -26.956814414066212
Iteration 9 complete. Error: -26.956814414066212 Total: 11.3752; Orientation: 1.1069; Line Search: 7.7952
Adding measurement 7593a97a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.844e+04, gradient 6.727e-04, dot -0.609; [dd2da9b2-e651-4b60-9803-3e9f08d4b1d4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.956814414066212, -21.14953404609573, -20.173724387037293, -20.173722467680307
LBFGS Accumulation History: 3 points
Removed measurement 7593a97a to history. Total: 3
th(0)=-26.956814414066212;dx=-4.521972651803097E-7
New Minimum: -26.956814414066212 > -27.258000088434837
WOLF (strong): th(1357293.8547200873)=-27.258000088434837; dx=7.75287484950972E-9 evalInputDelta=0.3011856743686252
END: th(678646.9273600436)=-27.206984033775697; dx=-1.8418623501509726E-7 evalInputDelta=0.25016961970948515
Fitness changed from -26.956814414066212 to -27.258000088434837
Iteration 10 complete. Error: -27.258000088434837 Total: 10.9802; Orientation: 0.9535; Line Search: 7.5953
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -27.258000088434837 (> -Infinity) after 136.234s (< 5400.000s)
Returns
-27.258000088434837