BasicOptimizer.scala:89 executed in 130.72 seconds (2.091 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 33678194299199
Reset training subject: 33680919625100
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -70.01819195985328 > -70.03415643969525
th(0)=-70.03415643969525;dx=-4.566162873507647E-6
New Minimum: -70.03415643969525 > -70.03416621175474
END: th(2.154434690031884)=-70.03416621175474; dx=-3.405718473867995E-6 evalInputDelta=9.772059485158024E-6
Fitness changed from -70.01819195985328 to -70.03416621175474
Iteration 1 complete. Error: -70.03416621175474 Total: 12.7482; Orientation: 0.0393; Line Search: 5.1518
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1994a8c9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-70.03416621175474;dx=-3.3903585389413017E-6
New Minimum: -70.03416621175474 > -70.03418575586959
WOLFE (weak): th(4.641588833612779)=-70.03418575586959; dx=-3.3897114678153117E-6 evalInputDelta=1.954411484916818E-5
New Minimum: -70.03418575586959 > -70.03420041395216
WOLFE (weak): th(9.283177667225559)=-70.03420041395216; dx=-3.389623729489527E-6 evalInputDelta=3.420219742622521E-5
New Minimum: -70.03420041395216 > -70.03427370431898
WOLFE (weak): th(27.849533001676676)=-70.03427370431898; dx=-3.3869996581550656E-6 evalInputDelta=1.0749256423991937E-4
New Minimum: -70.03427370431898 > -70.03460106702104
WOLFE (weak): th(111.3981320067067)=-70.03460106702104; dx=-3.3862431669205147E-6 evalInputDelta=4.3485526630604454E-4
New Minimum: -70.03460106702104 > -70.03633557520826
WOLFE (weak): th(556.9906600335335)=-70.03633557520826; dx=-3.3857687585437488E-6 evalInputDelta=0.002169363453518258
New Minimum: -70.03633557520826 > -70.04717649957537
WOLFE (weak): th(3341.9439602012008)=-70.04717649957537; dx=-3.3853281169972847E-6 evalInputDelta=0.013010287820634403
New Minimum: -70.04717649957537 > -70.12517777751822
WOLFE (weak): th(23393.607721408407)=-70.12517777751822; dx=-3.378711035892605E-6 evalInputDelta=0.09101156576348046
New Minimum: -70.12517777751822 > -70.75496955165679
WOLFE (weak): th(187148.86177126726)=-70.75496955165679; dx=-3.307949704212217E-6 evalInputDelta=0.7208033399020479
New Minimum: -70.75496955165679 > -75.92727940141637
END: th(1684339.7559414054)=-75.92727940141637; dx=-2.6794873656768748E-6 evalInputDelta=5.8931131896616336
Fitness changed from -70.03416621175474 to -75.92727940141637
Iteration 2 complete. Error: -75.92727940141637 Total: 28.5248; Orientation: 0.0635; Line Search: 25.4590
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 77232431 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-75.92727940141637;dx=-2.6387133276181456E-6
New Minimum: -75.92727940141637 > -84.63459204891141
END: th(3628800.0000000005)=-84.63459204891141; dx=-1.6365011945369304E-6 evalInputDelta=8.707312647495044
Fitness changed from -75.92727940141637 to -84.63459204891141
Iteration 3 complete. Error: -84.63459204891141 Total: 8.8040; Orientation: 0.0680; Line Search: 5.5903
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 18444a22 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-84.63459204891141;dx=-1.706573761333797E-6
New Minimum: -84.63459204891141 > -95.63526244192325
END: th(7818012.603187701)=-95.63526244192325; dx=-7.996629832830585E-7 evalInputDelta=11.00067039301183
Fitness changed from -84.63459204891141 to -95.63526244192325
Iteration 4 complete. Error: -95.63526244192325 Total: 8.3128; Orientation: 0.0633; Line Search: 5.0944
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 16535b91 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.134e+04, gradient 1.231e-03, dot -0.620; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -95.63526244192325, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 16535b91 to history. Total: 3
th(0)=-95.63526244192325;dx=-1.5128861383864226E-6
New Minimum: -95.63526244192325 > -100.86955307578964
WOLF (strong): th(1.6843397559414055E7)=-100.86955307578964; dx=2.1877535759353732E-7 evalInputDelta=5.234290633866394
New Minimum: -100.86955307578964 > -101.8927097044105
END: th(8421698.779707028)=-101.8927097044105; dx=-1.1721672168335212E-7 evalInputDelta=6.257447262487261
Fitness changed from -95.63526244192325 to -101.8927097044105
Iteration 5 complete. Error: -101.8927097044105 Total: 12.5731; Orientation: 1.0831; Line Search: 7.7976
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1d8cf49a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.202e+04, gradient 1.478e-03, dot -0.462; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.8927097044105, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 1d8cf49a to history. Total: 3
th(0)=-101.8927097044105;dx=-2.1840875269607173E-6
Armijo: th(1.8144000000000004E7)=-96.38529960348676; dx=7.850433772000487E-7 evalInputDelta=-5.5074101009237495
New Minimum: -101.8927097044105 > -105.48860839862934
WOLF (strong): th(9072000.000000002)=-105.48860839862934; dx=7.654168557759944E-7 evalInputDelta=3.595898694218832
New Minimum: -105.48860839862934 > -107.22860241142271
END: th(3024000.0000000005)=-107.22860241142271; dx=-8.220038995810778E-7 evalInputDelta=5.335892707012206
Fitness changed from -101.8927097044105 to -107.22860241142271
Iteration 6 complete. Error: -107.22860241142271 Total: 14.4809; Orientation: 1.0743; Line Search: 10.4552
Adding measurement 732a0187 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.581e+04, gradient 9.667e-04, dot -0.701; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.22860241142271, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 732a0187 to history. Total: 3
th(0)=-107.22860241142271;dx=-9.326601377227186E-7
New Minimum: -107.22860241142271 > -112.04928775718422
END: th(6515010.502656418)=-112.04928775718422; dx=-4.289630499687294E-7 evalInputDelta=4.820685345761504
Fitness changed from -107.22860241142271 to -112.04928775718422
Iteration 7 complete. Error: -112.04928775718422 Total: 8.4821; Orientation: 1.0564; Line Search: 5.0041
Adding measurement 1da85626 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.913e+04, gradient 1.031e-03, dot -0.609; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -112.04928775718422, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 1da85626 to history. Total: 3
th(0)=-112.04928775718422;dx=-1.0601079025312958E-6
New Minimum: -112.04928775718422 > -114.9735189875832
WOLF (strong): th(1.4036164632845048E7)=-114.9735189875832; dx=1.6945516348124813E-7 evalInputDelta=2.9242312303989877
New Minimum: -114.9735189875832 > -115.76160546697582
END: th(7018082.316422524)=-115.76160546697582; dx=-6.532336676852493E-8 evalInputDelta=3.7123177097916056
Fitness changed from -112.04928775718422 to -115.76160546697582
Iteration 8 complete. Error: -115.76160546697582 Total: 11.1720; Orientation: 1.0662; Line Search: 7.6422
Adding measurement 23760e11 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.987e+04, gradient 1.191e-03, dot -0.494; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -115.76160546697582, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 23760e11 to history. Total: 3
th(0)=-115.76160546697582;dx=-1.417886153306989E-6
Armijo: th(1.5120000000000006E7)=-113.59449900356127; dx=6.183026395081487E-7 evalInputDelta=-2.167106463414555
New Minimum: -115.76160546697582 > -118.9056629425665
WOLF (strong): th(7560000.000000003)=-118.9056629425665; dx=4.2435164084391754E-7 evalInputDelta=3.144057475590671
END: th(2520000.000000001)=-118.84106713471971; dx=-7.193253037364859E-7 evalInputDelta=3.0794616677438853
Fitness changed from -115.76160546697582 to -118.9056629425665
Iteration 9 complete. Error: -118.9056629425665 Total: 13.7313; Orientation: 1.0396; Line Search: 10.1950
Adding measurement 7a4ebf79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.700e+04, gradient 1.372e-03, dot -0.393; [4bb6cbcd-1f06-4c27-a314-fa771d004424 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -118.9056629425665, -84.63459204891141, -75.92727940141637, -70.03416621175474
LBFGS Accumulation History: 3 points
Removed measurement 7a4ebf79 to history. Total: 3
th(0)=-118.9056629425665;dx=-1.8803506832281474E-6
New Minimum: -118.9056629425665 > -123.74050248399335
WOLF (strong): th(5429175.418880349)=-123.74050248399335; dx=3.2703881311358937E-7 evalInputDelta=4.8348395414268595
END: th(2714587.7094401745)=-123.18513016344414; dx=-8.178640310619142E-7 evalInputDelta=4.279467220877649
Fitness changed from -118.9056629425665 to -123.74050248399335
Iteration 10 complete. Error: -123.74050248399335 Total: 11.4349; Orientation: 1.0229; Line Search: 7.7744
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -123.74050248399335 (> -Infinity) after 130.716s (< 5400.000s)
Returns
-123.74050248399335