BasicOptimizer.scala:89 executed in 93.57 seconds (1.397 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 289280490768200
Reset training subject: 289282538468500
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -59.352238170927485 > -59.41887681164697
th(0)=-59.41887681164697;dx=-4.562458723533691E-6
New Minimum: -59.41887681164697 > -59.418883120078824
END: th(2.154434690031884)=-59.418883120078824; dx=-3.3057041639599076E-6 evalInputDelta=6.3084318568940034E-6
Fitness changed from -59.352238170927485 to -59.418883120078824
Iteration 1 complete. Error: -59.418883120078824 Total: 9.4529; Orientation: 0.0371; Line Search: 3.7358
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 944406a to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-59.418883120078824;dx=-3.3124757862113873E-6
New Minimum: -59.418883120078824 > -59.41889889115554
WOLFE (weak): th(4.641588833612779)=-59.41889889115554; dx=-3.3124143874938113E-6 evalInputDelta=1.577107671835165E-5
New Minimum: -59.41889889115554 > -59.418914662228076
WOLFE (weak): th(9.283177667225559)=-59.418914662228076; dx=-3.312412113339407E-6 evalInputDelta=3.154214925160659E-5
New Minimum: -59.418914662228076 > -59.41897774647635
WOLFE (weak): th(27.849533001676676)=-59.41897774647635; dx=-3.3124145718169115E-6 evalInputDelta=9.462639752655377E-5
New Minimum: -59.41897774647635 > -59.41925216217699
WOLFE (weak): th(111.3981320067067)=-59.41925216217699; dx=-3.3123858280827633E-6 evalInputDelta=3.690420981641296E-4
New Minimum: -59.41925216217699 > -59.420731461767424
WOLFE (weak): th(556.9906600335335)=-59.420731461767424; dx=-3.31225231580772E-6 evalInputDelta=0.0018483416885999304
New Minimum: -59.420731461767424 > -59.42995338616035
WOLFE (weak): th(3341.9439602012008)=-59.42995338616035; dx=-3.3114503435319277E-6 evalInputDelta=0.011070266081524949
New Minimum: -59.42995338616035 > -59.49630337578309
WOLFE (weak): th(23393.607721408407)=-59.49630337578309; dx=-3.305892759637658E-6 evalInputDelta=0.07742025570426847
New Minimum: -59.49630337578309 > -60.03222480242311
WOLFE (weak): th(187148.86177126726)=-60.03222480242311; dx=-3.2394649980693156E-6 evalInputDelta=0.6133416823442843
New Minimum: -60.03222480242311 > -64.44445973365677
END: th(1684339.7559414054)=-64.44445973365677; dx=-2.655464746952164E-6 evalInputDelta=5.025576613577947
Fitness changed from -59.418883120078824 to -64.44445973365677
Iteration 2 complete. Error: -64.44445973365677 Total: 21.3222; Orientation: 0.0587; Line Search: 19.0262
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6a7b3add to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-64.44445973365677;dx=-2.419856484451011E-6
New Minimum: -64.44445973365677 > -71.69340649359216
END: th(3628800.0000000005)=-71.69340649359216; dx=-1.5762316389309587E-6 evalInputDelta=7.248946759935393
Fitness changed from -64.44445973365677 to -71.69340649359216
Iteration 3 complete. Error: -71.69340649359216 Total: 6.1730; Orientation: 0.0638; Line Search: 3.9211
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7c0d6bfe to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-71.69340649359216;dx=-1.506940406180917E-6
New Minimum: -71.69340649359216 > -80.5698785951399
END: th(7818012.603187701)=-80.5698785951399; dx=-7.965446144688251E-7 evalInputDelta=8.876472101547733
Fitness changed from -71.69340649359216 to -80.5698785951399
Iteration 4 complete. Error: -80.5698785951399 Total: 6.2365; Orientation: 0.0583; Line Search: 4.1091
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 785131ca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.448e+04, gradient 1.041e-03, dot -0.745; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.5698785951399, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement 785131ca to history. Total: 3
th(0)=-80.5698785951399;dx=-1.083192415249439E-6
New Minimum: -80.5698785951399 > -89.57735744671136
END: th(1.6843397559414055E7)=-89.57735744671136; dx=-1.996033354967087E-7 evalInputDelta=9.007478851571463
Fitness changed from -80.5698785951399 to -89.57735744671136
Iteration 5 complete. Error: -89.57735744671136 Total: 6.9890; Orientation: 1.0372; Line Search: 3.7666
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 74f0e2c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.344e+04, gradient 1.228e-03, dot -0.473; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -89.57735744671136, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement 74f0e2c7 to history. Total: 3
th(0)=-89.57735744671136;dx=-1.507401691301E-6
Armijo: th(3.628800000000001E7)=-85.00792658878922; dx=3.7643398259387294E-7 evalInputDelta=-4.569430857922143
New Minimum: -89.57735744671136 > -92.90607010335862
WOLF (strong): th(1.8144000000000004E7)=-92.90607010335862; dx=4.6117482700110557E-7 evalInputDelta=3.328712656647255
New Minimum: -92.90607010335862 > -95.42609691417906
END: th(6048000.000000001)=-95.42609691417906; dx=-3.9076638213914475E-7 evalInputDelta=5.848739467467695
Fitness changed from -89.57735744671136 to -95.42609691417906
Iteration 6 complete. Error: -95.42609691417906 Total: 10.6683; Orientation: 1.0049; Line Search: 7.5028
Adding measurement 9da5b03 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.463e+04, gradient 8.812e-04, dot -0.635; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -95.42609691417906, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement 9da5b03 to history. Total: 3
th(0)=-95.42609691417906;dx=-7.765447044804844E-7
New Minimum: -95.42609691417906 > -100.18677985750723
END: th(1.3030021005312836E7)=-100.18677985750723; dx=-8.04140249834106E-8 evalInputDelta=4.760682943328177
Fitness changed from -95.42609691417906 to -100.18677985750723
Iteration 7 complete. Error: -100.18677985750723 Total: 6.7613; Orientation: 1.0954; Line Search: 3.8582
Adding measurement 3fc7d637 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.589e+04, gradient 1.078e-03, dot -0.474; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -100.18677985750723, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement 3fc7d637 to history. Total: 3
th(0)=-100.18677985750723;dx=-1.1610317261633512E-6
Armijo: th(2.8072329265690096E7)=-97.49279967095087; dx=3.433061624252755E-7 evalInputDelta=-2.6939801865563595
New Minimum: -100.18677985750723 > -102.8828962637397
WOLF (strong): th(1.4036164632845048E7)=-102.8828962637397; dx=3.819494736994182E-7 evalInputDelta=2.6961164062324627
New Minimum: -102.8828962637397 > -104.00623593529986
END: th(4678721.544281683)=-104.00623593529986; dx=-4.231448315195337E-7 evalInputDelta=3.8194560777926228
Fitness changed from -100.18677985750723 to -104.00623593529986
Iteration 8 complete. Error: -104.00623593529986 Total: 10.3920; Orientation: 1.0232; Line Search: 7.5734
Adding measurement 19a011eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.300e+04, gradient 7.163e-04, dot -0.685; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -104.00623593529986, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement 19a011eb to history. Total: 3
th(0)=-104.00623593529986;dx=-5.130816743927771E-7
New Minimum: -104.00623593529986 > -107.65826751839451
END: th(1.0080000000000004E7)=-107.65826751839451; dx=-2.385984927745865E-7 evalInputDelta=3.6520315830946544
Fitness changed from -104.00623593529986 to -107.65826751839451
Iteration 9 complete. Error: -107.65826751839451 Total: 6.8106; Orientation: 1.0328; Line Search: 3.9553
Adding measurement a4873c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.467e+04, gradient 7.790e-04, dot -0.594; [5e655928-e088-44bf-8a83-a7adfd2efc6c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.65826751839451, -71.69340649359216, -64.44445973365677, -59.418883120078824
LBFGS Accumulation History: 3 points
Removed measurement a4873c7 to history. Total: 3
th(0)=-107.65826751839451;dx=-6.068412351286645E-7
New Minimum: -107.65826751839451 > -110.14994369708332
WOLF (strong): th(2.1716701675521396E7)=-110.14994369708332; dx=7.94532075729036E-8 evalInputDelta=2.491676178688806
New Minimum: -110.14994369708332 > -110.54500382245514
END: th(1.0858350837760698E7)=-110.54500382245514; dx=-3.432046920659148E-8 evalInputDelta=2.8867363040606335
Fitness changed from -107.65826751839451 to -110.54500382245514
Iteration 10 complete. Error: -110.54500382245514 Total: 8.2870; Orientation: 0.9734; Line Search: 5.5519
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -110.54500382245514 (> -Infinity) after 93.570s (< 5400.000s)
Returns
-110.54500382245514